- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents ARROW, a feed-forward framework for joint 3D reconstruction and point tracking from arbitrary sets of RGB observations. Its key contribution is an order-invariant querying mechanism that repurposes image-conditioned camera tokens from a pretrained DA3 encoder as identity embeddings, enabling flexible source, target, and reference selection without explicit temporal or camera indexing. By training on diverse static image collections, dynamic videos, and multi-view sequences, ARROW generalizes beyond temporally continuous video. It achieves state-of-the-art results on several 3D tracking benchmarks and outperforms dedicated multi-view trackers under an RGB-only setting, while remaining competitive in 3D reconstruction and camera pose estimation.
Critical strengths
-
The method is clearly presented and easy to understand.
-
The motivation is well aligned with the method design, and the distinction between order-aware and order-invariant representations is well motivated.
-
I appreciate the unified framework for 3D reconstruction and both monocular and multi-view tracking.
-
The method achieves state-of-the-art performance on several benchmarks.
Critical weaknesses
- Lack of video demonstrations. The paper would benefit from video demos that showcase the method's performance in dynamic scenes, particularly its ability to maintain temporally consistent 2D and 3D point tracks.
Questions for authors
-
Since the method predicts UV correspondences, a more comprehensive evaluation of 2D tracking would strengthen the paper. Both quantitative results and qualitative visualizations of 2D point trajectories would help clarify the effectiveness of the correspondence prediction mechanism.
-
Track4World and SpatialTrackerV2 also support 2D and 3D tracking. Qualitative comparisons with these methods, particularly visualizations of 2D trajectories and 3D point tracks on the same video sequences, would help better illustrate the advantages and limitations of the proposed method. Such comparisons could provide a clearer understanding of its tracking quality, temporal consistency, and robustness in challenging scenarios.
We thank the reviewer for the positive assessment of the motivation, unified framework, and benchmark results. We agree that video demonstrations would provide additional evidence of the reconstruction and tracking quality.
Qualitative demonstrations and comparisons. Our anonymous website provides a video overview, interactive reconstruction and tracking examples, and predicted 2D trajectories illustrations. We also provide the website source as supplementary material (
ARROW-supplementary.zip) for local viewing.2D tracking. Following D4RT, we supervise image-space coordinates as an auxiliary objective, while focusing on consistent 3D reconstruction and tracking. The UV loss coefficient accounts for only 5% of the loss weights (Appendix B.1). Similarly, we do not claim state-of-the-art 2D tracking performance. For completeness, we report below the monocular TAP-Vid-DAVIS results. Since D4RT does not report 2D tracking accuracy, we include OpenD4RT as an available implementation of D4RT-style querying, alongside Point4D, which also utilizes a similar auxiliary 2D supervision in its training.
Under the same evaluation protocol, ARROW outperforms OpenD4RT and Point4D on all four metrics, while trailing dedicated 2D trackers and methods that rely on explicit 2D motion estimation for 3D tracking. These results illustrate the remaining gap in this domain to dedicated 2D trackers, consistent with the local-precision limitation discussed in Appendix E.
The aforementioned website also provides illustration of the uv-regressed predictions on video sequences from the TAP-Vid-DAVIS benchmark.
Comparisons with Track4World and SpatialTrackerV2. Fig. 12 adds qualitative 3D tracking comparisons with Track4World, SpatialTrackerV2, and Open-D4RT on the same three video sequences. These complement the existing comparisons in Fig. 4, which we further extend, as well as the reconstruction and tracking results provided in Appendix G.
We hope that the additionally provided results and qualitative illustration address the raised concerns.