acceptodds
Under review as a conference paper at ICLR 2027

GeoMVT: Rethinking Multi-View Point Tracking through Geometric Reasoning

Abstract

We introduce GeoMVT, a depth-free multi-view point tracker for accurate 2D tracking and reliable 3D motion recovery. Our key idea is to leverage calibrated 3D camera geometry to bridge multi-view 2D tracking and accurate 3D motion recovery by guiding cross-view interactions during trajectory estimation. To this end, we propose a geometry-guided cross-view reasoning framework with two complementary stages: correlation aggregation and point-state updates. We first design epipolar-guided geometric transform attention (EpiGTA) to aggregate local correlation features through relative geometric alignment and adaptive epipolar guidance. Building on these geometry-enhanced features, a geometry-guided transformer combines spatio-temporal context with geometrically aligned cross-view interactions. Together, the two stages form a recurrent loop, where geometry-enhanced correlations guide refinement and updated trajectories determine subsequent correlation sampling and geometric compatibility. We construct a synthetic multi-view dataset of 12K sequences using Kubric for training. Experiments on four standard benchmarks demonstrate substantial improvements over existing multi-view 2D trackers and effective use of additional views across different camera configurations. Moreover, simple triangulation yields competitive 3D tracking, surpassing dedicated 3D trackers on multiple real-world benchmarks without depth input.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Point tracking, Multi-view geometry

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_xQKaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces GeoMVT, a depth-free multi-view point tracking framework for accurate 2D tracking and 3D motion recovery. It proposes an epipolar-guided geometric transform attention mechanism for cross-view correlation aggregation and a geometry-guided transformer for iterative trajectory refinement, enabling 3D trajectory recovery through weighted triangulation. Experiments on four benchmarks show that GeoMVT consistently outperforms existing multi-view 2D trackers and achieves better 3D tracking results than MVTracker on DexYCB and Harmony4D, without requiring depth input.

Critical strengths
  1. Practical Relevance. The proposed method enables multi-view 2D point tracking and 3D trajectory recovery without requiring depth input, which is useful for applications such as robotics and 3D motion reconstruction.

  2. Strong Empirical Performance. GeoMVT consistently outperforms existing multi-view 2D trackers on all four benchmarks. It also achieves better 3D tracking results than MVTracker on DexYCB and Harmony4D without using depth input.

  3. Comprehensive Ablation Studies. The paper provides extensive ablation studies to examine the effectiveness of the proposed components.

Critical weaknesses
  1. Unclear Contribution Attribution. EpiGTA builds on existing geometric attention and epipolar constraints. In Table 4, adding GTA to epipolar-guided attention yields only marginal gains. Although the full framework performs well, the individual contributions of geometric transformation, epipolar guidance, and the two-stage design remain insufficiently distinguished.

  2. Insufficient Direct Evidence for Improved 3D Geometric Consistency. The paper claims that geometry-guided reasoning improves cross-view consistency and leads to more accurate 3D tracking. While Figure 7 provide supporting evidence by comparing 3D errors within matched 2D error bins, direct geometric consistency metrics, such as epipolar or reprojection errors, are still missing. Moreover, the ablation studies primarily focus on 2D tracking, leaving the individual contributions of CCA and GIU to 3D performance unclear. Additional 3D ablations and evaluations of cross-view geometric consistency would help substantiate the paper's central claims.

  3. Reliance on Cross-View Correspondences and Accurate Camera Calibration. The method assumes known point correspondences across cameras and accurate camera calibration, which may be difficult to obtain in real-world scenarios. The paper does not evaluate robustness to incorrect cross-view correspondences.

Questions for authors
  1. Could the authors further clarify the individual contribution of GTA-based geometric transformations? Table 4 shows only marginal gains from adding GTA to epipolar-guided attention. It would be helpful to evaluate a full-model variant that retains epipolar guidance in CCA but replaces GTA-based transformations with standard cross-view attention, ablating the geometric transformations in CCA and GIU separately. This would help better isolate the contributions of the proposed geometric reasoning mechanisms.

  2. Could the authors provide more direct evidence that GeoMVT improves cross-view geometric consistency? While Figure 7 and Appendix B.9 show lower 3D errors within matched 2D error bins, additional metrics such as epipolar errors or reprojection errors would help substantiate this claim. Moreover, reporting 3D AJ and MTE for the CCA-only, GIU-only, and full-model variants would clarify how each component contributes to 3D tracking performance.

  3. The method assumes that query points corresponding to the same physical point are already identified across camera views. How would GeoMVT behave when these cross-view correspondences are imperfect or partially incorrect? An evaluation with perturbed or mismatched cross-view correspondences would help clarify the robustness of the proposed framework and its applicability to practical scenarios.

  4. Appendix B.1 provides a useful analysis of robustness to camera calibration errors in terms of 2D tracking accuracy. However, calibration errors may also directly affect the subsequent 3D triangulation. Could the authors further evaluate 3D tracking performance under calibration perturbations, particularly in terms of 3D AJ and MTE? This would help clarify how sensitive the reported 3D improvements are to calibration accuracy.

3 comments
AuthorsOpenReview ↗

Response to Reviewer xQKa (Part 1/3)

W1 & Q1: Detailed contribution ablations

Thank you for this suggestion. We have expanded the component ablations with a variant that retains epipolar guidance in CCA but removes its geometric transformations, while keeping GIU unchanged. Table R1 (Table 3 in the manuscript) reports both 2D and 3D tracking results. Adding geometric transformations in CCA further improves the epipolar-guided variant, while coupling CCA with GIU yields higher AJ than either stage alone. Together, the 2D and 3D results further support the effectiveness of geometric transformations, epipolar guidance, and the coupled two-stage design.

Table R1. Ablation of the main components on 2D and 3D point tracking. Additional results is in appendix Table B.3.

CCAGIU2D3DGTAEPI00××✓✓×××✓✓✓00××✓✓✓Panoptic StudioDexYCBAJδavgOA44.067.175.246.269.276.848.770.580.049.671.481.050.972.481.5AJδavgOA46.463.678.549.066.981.449.767.583.651.369.084.951.469.084.4Panoptic StudioDexYCBAJδavgOAMTE82.687.797.14.985.690.696.34.188.392.597.63.689.593.497.73.390.193.897.93.1AJδavgOAMTE70.779.092.91.874.482.892.61.576.984.793.01.477.685.693.01.378.486.393.01.2\scriptsize\begin{array}{c|c|c|c}\text{CCA}&\text{GIU}&\text{2D}&\text{3D}\\\\\hline \begin{array}{cc}\text{GTA}&\text{EPI}\\\\\hline \phantom{0}&\phantom{0}\\\\\times&\times\\\\\checkmark&\checkmark\\\\\times&\times\\\\\times&\checkmark\\\\\checkmark&\checkmark\end{array}&\begin{array}{c}\phantom{0}\\\\\phantom{0}\\\\\times\\\\\times\\\\\checkmark\\\\\checkmark\\\\\checkmark\end{array}&\begin{array}{c|c}\text{Panoptic Studio}&\text{DexYCB}\\\\\begin{array}{ccc}\text{AJ}&\delta_{\mathrm{avg}}&\text{OA}\\\\\hline 44.0&67.1&75.2\\\\46.2&69.2&76.8\\\\48.7&70.5&80.0\\\\49.6&71.4&81.0\\\\\mathbf{50.9}&\mathbf{72.4}&\mathbf{81.5}\end{array}&\begin{array}{ccc}\text{AJ}&\delta_{\mathrm{avg}}&\text{OA}\\\\\hline 46.4&63.6&78.5\\\\49.0&66.9&81.4\\\\49.7&67.5&83.6\\\\51.3&\mathbf{69.0}&\mathbf{84.9}\\\\\mathbf{51.4}&\mathbf{69.0}&84.4\end{array}\end{array}&\begin{array}{c|c}\text{Panoptic Studio}&\text{DexYCB}\\\\\begin{array}{cccc}\text{AJ}&\delta_{\mathrm{avg}}&\text{OA}&\text{MTE}\\\\\hline 82.6&87.7&97.1&4.9\\\\85.6&90.6&96.3&4.1\\\\88.3&92.5&97.6&3.6\\\\89.5&93.4&97.7&3.3\\\\\mathbf{90.1}&\mathbf{93.8}&\mathbf{97.9}&\mathbf{3.1}\end{array}&\begin{array}{cccc}\text{AJ}&\delta_{\mathrm{avg}}&\text{OA}&\text{MTE}\\\\\hline 70.7&79.0&92.9&1.8\\\\74.4&82.8&92.6&1.5\\\\76.9&84.7&\mathbf{93.0}&1.4\\\\77.6&85.6&\mathbf{93.0}&1.3\\\\\mathbf{78.4}&\mathbf{86.3}&\mathbf{93.0}&\mathbf{1.2}\end{array}\end{array}\end{array}

W2.2 & Q2.2: Additional 3D ablations

We have extended Table R1 with 3D tracking metrics. The 3D results largely mirror the 2D trends, with the coupled model achieving the highest AJ and lowest MTE on both datasets. Together with the reprojection analysis, these results further support the effectiveness of our geometric design in improving cross-view consistency and 3D trajectory recovery.

W2.1 & Q2.1: Additional evidence of cross-view geometric consistency

Thank you for this suggestion. We triangulate 2D tracks using the same weighted procedure and measure mean pixel distances between their reprojections and predicted 2D positions. Table R2 (Appendix Table B.20) shows that GeoMVT consistently achieves lower reprojection error than MV-TAP [1] in every available bin across all four datasets. Our multi-view tracks therefore agree more closely with a common 3D trajectory, providing further evidence of improved cross-view geometric consistency.

Table R2. Reprojection errors within matched 2D error bins. 2D error is the mean pixel error at ground-truth visible positions.

2D errorPanoptic StudioDexYCBMV-KubricHarmony4D0[0,1)[1,2)[2,3)[3,4)[4,5)[5,6)[6,7)[7,8)MV-TAPOursΔ1.180.75−0.432.181.46−0.722.951.98−0.973.312.44−0.874.422.93−1.493.922.75−1.175.823.76−2.066.063.64−2.42MV-TAPOursΔ0.610.41−0.201.871.21−0.662.521.75−0.774.022.67−1.345.103.28−1.825.083.73−1.366.814.87−1.946.214.45−1.76MV-TAPOursΔ3.971.62−2.345.882.27−3.619.402.66−6.7425.583.35−22.236.773.29−3.4812.475.36−7.127.784.53−3.249.225.85−3.37MV-TAPOursΔ−−−1.141.04−0.111.981.56−0.432.581.92−0.663.462.42−1.043.692.76−0.934.652.96−1.695.993.23−2.75\scriptsize\begin{array}{c|c|c|c|c}\text{2D error}&\text{Panoptic Studio}&\text{DexYCB}&\text{MV-Kubric}&\text{Harmony4D}\\\\\hline \begin{array}{c}\phantom{0}\\\\\text{[0,1)}\\\\\text{[1,2)}\\\\\text{[2,3)}\\\\\text{[3,4)}\\\\\text{[4,5)}\\\\\text{[5,6)}\\\\\text{[6,7)}\\\\\text{[7,8)}\end{array}&\begin{array}{ccc}\text{MV-TAP}&\text{Ours}&\Delta\\\\\hline 1.18&\mathbf{0.75}&-0.43\\\\2.18&\mathbf{1.46}&-0.72\\\\2.95&\mathbf{1.98}&-0.97\\\\3.31&\mathbf{2.44}&-0.87\\\\4.42&\mathbf{2.93}&-1.49\\\\3.92&\mathbf{2.75}&-1.17\\\\5.82&\mathbf{3.76}&-2.06\\\\6.06&\mathbf{3.64}&-2.42\end{array}&\begin{array}{ccc}\text{MV-TAP}&\text{Ours}&\Delta\\\\\hline 0.61&\mathbf{0.41}&-0.20\\\\1.87&\mathbf{1.21}&-0.66\\\\2.52&\mathbf{1.75}&-0.77\\\\4.02&\mathbf{2.67}&-1.34\\\\5.10&\mathbf{3.28}&-1.82\\\\5.08&\mathbf{3.73}&-1.36\\\\6.81&\mathbf{4.87}&-1.94\\\\6.21&\mathbf{4.45}&-1.76\end{array}&\begin{array}{ccc}\text{MV-TAP}&\text{Ours}&\Delta\\\\\hline 3.97&\mathbf{1.62}&-2.34\\\\5.88&\mathbf{2.27}&-3.61\\\\9.40&\mathbf{2.66}&-6.74\\\\25.58&\mathbf{3.35}&-22.23\\\\6.77&\mathbf{3.29}&-3.48\\\\12.47&\mathbf{5.36}&-7.12\\\\7.78&\mathbf{4.53}&-3.24\\\\9.22&\mathbf{5.85}&-3.37\end{array}&\begin{array}{ccc}\text{MV-TAP}&\text{Ours}&\Delta\\\\\hline -&-&-\\\\1.14&\mathbf{1.04}&-0.11\\\\1.98&\mathbf{1.56}&-0.43\\\\2.58&\mathbf{1.92}&-0.66\\\\3.46&\mathbf{2.42}&-1.04\\\\3.69&\mathbf{2.76}&-0.93\\\\4.65&\mathbf{2.96}&-1.69\\\\5.99&\mathbf{3.23}&-2.75\end{array}\end{array}

References.

  • [1] MV-TAP: Tracking Any Point in Multi-View Videos, CVPR 2026.
AuthorsOpenReview ↗

Response to Reviewer xQKa (Part 2/3)

W3 & Q3: Robustness to incorrect cross-view correspondences

Thank you for this insightful suggestion. To evaluate realistic initialization errors, we retain the ground-truth query in the first valid view and perturb queries in the remaining valid views by 1–3 px. Table R3 (Appendix Table B.21) shows that both methods lose accuracy as correspondence errors increase, but GeoMVT consistently outperforms MV-TAP [1] across all three metrics on all four datasets. Even at 2 px, GeoMVT retains 93.1–95.0% of its unperturbed AJ. These results support robustness to imperfect initial cross-view correspondences without requiring accurate queries in every view. This robustness suggests that GeoMVT could be paired with recent correspondence methods such as RoMa v2 [2] or VGGT [3] to initialize queries across views and enable accurate multi-view tracking.

Table R3. 2D tracking with imperfect initial cross-view correspondences.

$ \scriptsize\begin{array}{c|c|c|c|c|c}\text{Error (px)}&\text{Method}&\text{Panoptic Studio}&\text{DexYCB}&\text{MV-Kubric}&\text{Harmony4D}\\&&\begin{array}{ccc}\text{AJ}&\delta_{\mathrm{avg}}&\text{OA}\end{array}&\begin{array}{ccc}\text{AJ}&\delta_{\mathrm{avg}}&\text{OA}\end{array}&\begin{array}{ccc}\text{AJ}&\delta_{\mathrm{avg}}&\text{OA}\end{array}&\begin{array}{ccc}\text{AJ}&\delta_{\mathrm{avg}}&\text{OA}\end{array}\\\hline 0&\begin{array}{l}\text{MV-TAP}\\\text{Ours}\end{array}&\begin{array}{ccc}43.5&67.2&74.8\\\mathbf{50.4}&\mathbf{72.2}&\mathbf{81.4}\end{array}&\begin{array}{ccc}44.9&62.7&77.7\\\mathbf{49.8}&\mathbf{68.5}&\mathbf{83.4}\end{array}&\begin{array}{ccc}49.5&68.8&81.9\\\mathbf{58.0}&\mathbf{77.2}&\mathbf{86.9}\end{array}&\begin{array}{ccc}38.9&71.3&69.3\\\mathbf{44.4}&\mathbf{75.0}&\mathbf{72.3}\end{array}\\\hline 1&\begin{array}{l}\text{MV-TAP}\\\text{Ours}\end{array}&\begin{array}{ccc}42.6&66.4&74.5\\\mathbf{49.8}&\mathbf{71.5}&\mathbf{81.3}\end{array}&\begin{array}{ccc}44.4&62.1&77.8\\\mathbf{49.2}&\mathbf{67.9}&\mathbf{83.3}\end{array}&\begin{array}{ccc}48.6&67.9&81.9\\\mathbf{57.1}&\mathbf{76.3}&\mathbf{86.8}\end{array}&\begin{array}{ccc}38.1&70.3&69.2\\\mathbf{43.7}&\mathbf{74.2}&\mathbf{72.1}\end{array}\\\hline 2&\begin{array}{l}\text{MV-TAP}\\\text{Ours}\end{array}&\begin{array}{ccc}40.5&63.8&74.2\\\mathbf{47.9}&\mathbf{69.0}&\mathbf{81.1}\end{array}&\begin{array}{ccc}41.5&58.5&77.5\\\mathbf{46.5}&\mathbf{64.2}&\mathbf{83.2}\end{array}&\begin{array}{ccc}45.3&64.0&81.3\\\mathbf{54.0}&\mathbf{72.8}&\mathbf{86.5}\end{array}&\begin{array}{ccc}35.9&67.5&69.0\\\mathbf{41.8}&\mathbf{71.5}&\mathbf{71.9}\end{array}\\\hline 3&\begin{array}{l}\text{MV-TAP}\\\text{Ours}\end{array}&\begin{array}{ccc}37.8&60.5&73.6\\\mathbf{45.3}&\mathbf{66.0}&\mathbf{80.6}\end{array}&\begin{array}{ccc}39.1&55.5&77.3\\\mathbf{43.9}&\mathbf{61.1}&\mathbf{82.8}\end{array}&\begin{array}{ccc}41.5&59.9&80.5\\\mathbf{50.4}&\mathbf{68.8}&\mathbf{85.9}\end{array}&\begin{array}{ccc}33.2&63.9&68.7\\\mathbf{39.0}&\mathbf{68.2}&\mathbf{71.4}\end{array}\\\hline \end{array} $

References.

  • [1] MV-TAP: Tracking Any Point in Multi-View Videos, CVPR 2026.
  • [2] RoMa v2: Harder Better Faster Denser Feature Matching, ECCV 2026.
  • [3] VGGT: Visual Geometry Grounded Transformer, CVPR 2025.
AuthorsOpenReview ↗

Response to Reviewer xQKa (Part 3/3)

Q4: 3D tracking performance under calibration perturbations

Thank you for this insightful suggestion. We separately perturb camera rotations with fixed centers and camera positions with fixed rotations, applying the perturbed extrinsics to both tracking and triangulation.

(i) Rotation perturbations. Table R4 shows that both methods lose AJ and incur larger MTE as rotation errors increase. Nevertheless, GeoMVT retains higher AJ and lower MTE than MV-TAP [1] across all four datasets throughout the tested range.

(ii) Translation perturbations. Table R5 shows that our advantage also persists throughout the tested translation-error range. These results demonstrate that our geometric design retains its comparative benefit under calibration perturbations, although inaccurate cameras affect both tracking and 3D recovery.

The corresponding curves are provided in Appendix Figs. B.4–B.7.

Table R4. 3D tracking under camera rotation perturbations.

Rotation error (°)Panoptic StudioDexYCBMV-KubricHarmony4D0000.5125MV-TAPOursAJMTE81.65.181.15.379.75.873.67.251.313.2AJMTE90.13.189.43.587.34.378.56.153.112.8MV-TAPOursAJMTE69.42.069.22.068.02.263.12.550.54.2AJMTE78.41.277.41.374.61.666.62.151.94.0MV-TAPOursAJMTE29.410.728.110.825.411.120.412.08.217.3AJMTE48.43.744.64.038.84.629.16.110.611.9MV-TAPOursAJMTE55.112.854.712.853.113.248.914.834.323.1AJMTE63.69.262.39.560.610.052.812.934.522.8\scriptsize\begin{array}{c|c|c|c|c}\text{Rotation error (°)}&\text{Panoptic Studio}&\text{DexYCB}&\text{MV-Kubric}&\text{Harmony4D}\\\\\hline \begin{array}{c}\phantom{0}\\\\\phantom{0}\\\\0\\\\0.5\\\\1\\\\2\\\\5\end{array}&\begin{array}{c|c}\text{MV-TAP}&\text{Ours}\\\\\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline 81.6&5.1\\\\81.1&5.3\\\\79.7&5.8\\\\73.6&7.2\\\\51.3&13.2\end{array}&\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline \mathbf{90.1}&\mathbf{3.1}\\\\\mathbf{89.4}&\mathbf{3.5}\\\\\mathbf{87.3}&\mathbf{4.3}\\\\\mathbf{78.5}&\mathbf{6.1}\\\\\mathbf{53.1}&\mathbf{12.8}\end{array}\end{array}&\begin{array}{c|c}\text{MV-TAP}&\text{Ours}\\\\\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline 69.4&2.0\\\\69.2&2.0\\\\68.0&2.2\\\\63.1&2.5\\\\50.5&4.2\end{array}&\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline \mathbf{78.4}&\mathbf{1.2}\\\\\mathbf{77.4}&\mathbf{1.3}\\\\\mathbf{74.6}&\mathbf{1.6}\\\\\mathbf{66.6}&\mathbf{2.1}\\\\\mathbf{51.9}&\mathbf{4.0}\end{array}\end{array}&\begin{array}{c|c}\text{MV-TAP}&\text{Ours}\\\\\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline 29.4&10.7\\\\28.1&10.8\\\\25.4&11.1\\\\20.4&12.0\\\\8.2&17.3\end{array}&\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline \mathbf{48.4}&\mathbf{3.7}\\\\\mathbf{44.6}&\mathbf{4.0}\\\\\mathbf{38.8}&\mathbf{4.6}\\\\\mathbf{29.1}&\mathbf{6.1}\\\\\mathbf{10.6}&\mathbf{11.9}\end{array}\end{array}&\begin{array}{c|c}\text{MV-TAP}&\text{Ours}\\\\\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline 55.1&12.8\\\\54.7&12.8\\\\53.1&13.2\\\\48.9&14.8\\\\34.3&23.1\end{array}&\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline \mathbf{63.6}&\mathbf{9.2}\\\\\mathbf{62.3}&\mathbf{9.5}\\\\\mathbf{60.6}&\mathbf{10.0}\\\\\mathbf{52.8}&\mathbf{12.9}\\\\\mathbf{34.5}&\mathbf{22.8}\end{array}\end{array}\end{array}

Table R5. 3D tracking under camera translation perturbations.

\scriptsize\begin{array}{c|c|c|c|c}\text{Translation error (\\%)}&\text{Panoptic Studio}&\text{DexYCB}&\text{MV-Kubric}&\text{Harmony4D}\\\\\hline \begin{array}{c}\phantom{0}\\\\\phantom{0}\\\\0\\\\0.25\\\\0.5\\\\1\\\\2\end{array}&\begin{array}{c|c}\text{MV-TAP}&\text{Ours}\\\\\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline 81.6&5.1\\\\81.3&5.2\\\\80.8&5.4\\\\79.2&6.0\\\\72.1&7.4\end{array}&\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline \mathbf{90.1}&\mathbf{3.1}\\\\\mathbf{89.9}&\mathbf{3.3}\\\\\mathbf{89.5}&\mathbf{3.5}\\\\\mathbf{87.8}&\mathbf{4.2}\\\\\mathbf{78.6}&\mathbf{6.0}\end{array}\end{array}&\begin{array}{c|c}\text{MV-TAP}&\text{Ours}\\\\\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline 69.4&2.0\\\\69.4&2.0\\\\69.5&2.0\\\\69.2&2.1\\\\65.9&2.3\end{array}&\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline \mathbf{78.4}&\mathbf{1.2}\\\\\mathbf{78.2}&\mathbf{1.3}\\\\\mathbf{77.7}&\mathbf{1.3}\\\\\mathbf{76.3}&\mathbf{1.5}\\\\\mathbf{70.4}&\mathbf{1.8}\end{array}\end{array}&\begin{array}{c|c}\text{MV-TAP}&\text{Ours}\\\\\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline 29.4&10.7\\\\29.3&10.7\\\\28.6&10.7\\\\26.7&11.0\\\\22.8&11.7\end{array}&\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline \mathbf{48.4}&\mathbf{3.7}\\\\\mathbf{47.9}&\mathbf{3.8}\\\\\mathbf{46.1}&\mathbf{3.9}\\\\\mathbf{41.4}&\mathbf{4.3}\\\\\mathbf{33.1}&\mathbf{5.4}\end{array}\end{array}&\begin{array}{c|c}\text{MV-TAP}&\text{Ours}\\\\\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline 55.1&12.8\\\\55.1&12.8\\\\54.7&12.9\\\\53.5&13.2\\\\49.3&14.6\end{array}&\begin{array}{cc}\text{AJ}&\text{MTE}\\\\\hline \mathbf{63.6}&\mathbf{9.2}\\\\\mathbf{63.3}&\mathbf{9.2}\\\\\mathbf{62.9}&\mathbf{9.3}\\\\\mathbf{60.7}&\mathbf{10.2}\\\\\mathbf{54.7}&\mathbf{11.9}\end{array}\end{array}\end{array}

References.

  • [1] MV-TAP: Tracking Any Point in Multi-View Videos, CVPR 2026.

We sincerely thank you again for the valuable comments provided during this review.

Reviewer_8qrxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a new architecture for multi-view point tracking. The crucial part is the integration of group transform ideas (GTA; Miyato et al., 2024) into the existing state-of-the-art multi-view 2D tracking transformer (MV-TAP; Koo et al., 2026), which makes direct use of the calibrated camera transforms to map tokens from one view to another directly inside the attention mechanism. In the same vein, epipolar constraints are used to guide the attention. This leads to 2D tracks that are geometrically consistent across views, so that simple triangulation produces 3D trajectories matching or exceeding depth-based 3D trackers without using depth input. The paper also contributes a synthetic multi-view dataset for point tracking. The overall results suggest consistent improvement over the baselines, with clear ablations showing isolated positive effects of the core mechanisms.

Critical strengths
  • Clear idea of integrating existing mechanisms in a meaningful way. Group transforms and epipolar constraints are built into the cross-view attention itself, so calibration acts as a structural prior rather than additional input. The ablations with capacity-matched baseline convincingly isolate this as the source of the main gains.
  • Strong quantitative results. The method shows consistent 2D improvements over MV-TAP (current state-of-the-art) on all four benchmarks at matched training data. Most notably, on real world DexYCB dataset, simple triangulation of the 2D tracks reaches 80.6 AJ / 1.1 cm MTE without depth, surpassing MVTracker (73.3 / 1.7 cm) which uses ground truth depth, directly validating the central claim that geometry-consistent 2D tracks make 3D recovery easy.
Critical weaknesses
  • Modest novelty and slight overclaim.. The method composes existing components (group transforms + epipolar attention bias + MV-TAP backbone). This is acceptable for a well-executed integration, but Fig. 1 claims “the first depth-free multi-view point tracker,” which the paper’s own Tables 1–2 contradict (MV-TAP and LAPA are both depth-free). The positioning should be corrected.
Questions for authors
  • Why is GeoMVT so clearly worse than MVTracker in 3D on Multi-View Kubric (56.6 vs 68.9 AJ, 19.6 vs 10.1 cm MTE) when Kubric is the actual training domain? Also, could the authors provide any insight into why the method clearly outperforms MVTracker on DexYCB but is roughly on par with it on Panoptic Studio?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 8qrx (Part 1/2)

W1: Modest novelty and slight overclaim

Novelty. We thank the reviewer for the constructive feedback. Our contribution centers on how calibrated 3D camera geometry structures cross-view interactions throughout iterative multi-view 2D tracking, rather than on the individual geometric operators. Specifically, our framework integrates three complementary aspects:

  1. Conceptual insight. We move beyond auxiliary camera encoding by incorporating geometry directly into cross-view feature comparison and aggregation, conditioned on evolving trajectory hypotheses.

  2. Stage-specific geometric reasoning. We tailor geometric guidance to observation aggregation and trajectory correction: epipolar compatibility regulates correlation exchange, while state refinement retains geometric transformations without additional epipolar penalties to preserve cross-view correction pathways.

  3. Coupled two-stage architecture. Geometry-enhanced correlation features inform point-state updates, while refined positions refresh correlation sampling and geometric compatibility for subsequent aggregation, coupling geometric reasoning with trajectory refinement.

Together, these designs embed geometric reasoning within 2D trajectory estimation, producing accurate, cross-view-consistent tracks that support 3D recovery through simple triangulation without depth input.

Overclaim. Thank you for pointing this out. We have replaced the first with a in the Fig. 1 caption to avoid overstating priority.

AuthorsOpenReview ↗

Response to Reviewer 8qrx (Part 2/2)

Q1: Tracking performance across datasets

Thank you for raising this question. We examine depth range, input depth quality, and 3D motion between evaluation frames. Note that we have corrected the MTE scale for MV-Kubric.

Depth and motion analysis. Table R1 summarizes tracked-point camera-Z depths, absolute input-depth residuals at ground-truth pixels, and 3D displacements between adjacent evaluation frames. MV-Kubric quantities use the approximate conversion of one scene unit to 13 cm. These statistics suggest different tradeoffs across datasets:

(i) MV-Kubric. Its simulated depth has the lowest median residual, benefiting MVTracker's depth-based 3D representation [1]. Meanwhile, its largest median 3D displacement challenges correspondence estimation, with errors in the resulting 2D tracks affecting triangulated trajectories. Training on Kubric does not eliminate this distinction between direct 3D tracking and triangulation. However, when depth noise reaches 5 cm, MVTracker's AJ on MV-Kubric drops from 68.9 to 48.6, highlighting its reliance on accurate depth (Table R2).

(ii) DexYCB. Its input depth is less accurate, whereas its shorter depth range and smaller median 3D displacement favor our 2D-to-3D formulation. These conditions favor our 2D tracking approach, consistent with its larger advantage over MVTracker.

(iii) Panoptic Studio. Points lie farther from the cameras, increasing triangulation's sensitivity to 2D localization errors, and median 3D displacement is larger than on DexYCB. Meanwhile, its relatively accurate depth benefits MVTracker, narrowing the performance gap between the methods.

Table R1. Depth and motion statistics across datasets.

\begin{array}{c|c|c|c} \text{Dataset} & \text{GT depth (cm)} & \text{Depth residual (cm)} & \text{3D displacement (cm/frame)} \\\\ \hline \begin{array}{l} \phantom{\text{P50}} \\\\ \text{Panoptic Studio} \\\\ \text{DexYCB} \\\\ \text{MV-Kubric} \end{array} & \begin{array}{ccc} \text{P5} & \text{P50} & \text{P95} \\\\ \hline 179.29 & 283.71 & 368.65 \\\\ 60.41 & 82.54 & 103.07 \\\\ 93.57 & 155.12 & 217.88 \end{array} & \begin{array}{cc} \text{P5} & \text{P50} \\\\ \hline <0.0001 & 0.8128 \\\\ 0.0774 & 1.0018 \\\\ 0.0167 & 0.3277 \end{array} & \begin{array}{cc} \text{P5} & \text{P50} \\\\ \hline 0.28 & 1.27 \\\\ 0.03 & 0.93 \\\\ 0.19 & 2.88 \end{array} \end{array}$$ **3D tracking under depth noise.** Table R2 (Appendix Table A.2) further isolates MVTracker's dependence on depth quality by adding Gaussian noise to its native depth inputs. Its AJ decreases and MTE increases as depth noise grows, whereas GeoMVT requires no depth input. For example, 2 cm depth noise reduces MVTracker's AJ on DexYCB from 73.3 to 64.2. These results substantiate the importance of depth quality to MVTracker and the practical benefit of our depth-free recovery. **Table R2. MVTracker's 3D tracking performance under input depth noise.** $$\scriptsize\begin{array}{c|c|c|c|c}\begin{array}{c}\vphantom{\begin{array}{c}\text{Panoptic Studio}\\\\\delta_{\mathrm{avg}}\uparrow\end{array}}\text{Method}\\\\\hline \phantom{0}\\\\\phantom{0}\\\\\phantom{0}\\\\\text{MVTracker}\\\\\phantom{0}\\\\\phantom{0}\\\\\phantom{0}\\\\\hline \text{GeoMVT (Ours)}\end{array}&\begin{array}{c}\vphantom{\begin{array}{c}\text{Panoptic Studio}\\\\\delta_{\mathrm{avg}}\uparrow\end{array}}\sigma\text{ (cm)}\\\\\hline 0\\\\1\\\\2\\\\5\\\\10\\\\20\\\\50\\\\\hline \text{--}\end{array}&\begin{array}{c}\text{Panoptic Studio}\\\\\begin{array}{cccc}\text{AJ}\uparrow&\delta_{\mathrm{avg}}\uparrow&\text{OA}\uparrow&\text{MTE}\downarrow\\\\\hline 91.4&94.6&98.3&3.2\\\\91.2&94.3&98.3&3.2\\\\90.3&93.5&98.3&3.5\\\\85.8&89.8&98.3&4.4\\\\78.9&83.8&98.3&5.9\\\\72.0&77.6&98.3&8.1\\\\67.5&73.5&98.3&9.6\\\\\hline 90.4&94.1&97.8&3.0\end{array}\end{array}&\begin{array}{c}\text{DexYCB}\\\\\begin{array}{cccc}\text{AJ}\uparrow&\delta_{\mathrm{avg}}\uparrow&\text{OA}\uparrow&\text{MTE}\downarrow\\\\\hline 73.3&81.0&93.0&1.6\\\\70.8&78.4&93.0&2.0\\\\64.2&72.1&93.0&2.8\\\\58.5&66.0&93.0&4.1\\\\46.0&52.3&93.0&8.3\\\\38.8&45.6&93.0&10.5\\\\33.7&40.7&93.0&12.2\\\\\hline 80.6&88.4&92.9&1.1\end{array}\end{array}&\begin{array}{c}\text{MV-Kubric}\\\\\begin{array}{cccc}\text{AJ}\uparrow&\delta_{\mathrm{avg}}\uparrow&\text{OA}\uparrow&\text{MTE}\downarrow\\\\\hline 68.9&80.7&90.7&1.3\\\\67.6&78.2&92.1&1.4\\\\61.9&72.7&91.5&1.7\\\\48.6&59.4&89.3&2.7\\\\38.1&48.4&87.3&4.4\\\\29.8&39.2&85.6&6.7\\\\23.7&32.2&84.6&9.8\\\\\hline 56.6&66.1&90.2&2.5\end{array}\end{array}\end{array}$$ **References.** - [1] Multi-View 3D Point Tracking, ICCV 2025. --- We sincerely thank you again for the valuable comments provided during this review.
Reviewer_npiZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GeoMVT, a depth-free multi-view point tracking framework that uses calibrated camera geometry to guide cross-view interactions during iterative trajectory estimation. The method introduces two components: (1) Epipolar-guided Geometric Transform Attention (EpiGTA), which aggregates view-local correlation features using relative geometric transformations and epipolar compatibility, and (2) a geometry-guided transformer that integrates cross-view information with temporal and spatial context to iteratively refine point trajectories.

The resulting 2D trajectories are triangulated to recover 3D motion without depth input. The authors construct a synthetic multi-view training dataset using Kubric and evaluate the method on DexYCB, Panoptic Studio, Multi-View Kubric, and Harmony4D. The experiments report improvements over existing multi-view 2D tracking methods and competitive 3D tracking performance.

Critical strengths

Improvements in multi-view 2D and 3D point tracking performance.

The method achieves improvements over existing multi-view 2D tracking approaches across four benchmarks. The improved 2D trajectories also lead to better 3D tracking results through triangulation, without requiring depth input.

Ablation studies examining the proposed geometric components.

The paper provides ablations on the geometric transformation and epipolar guidance components, as well as camera configurations. These experiments show that the proposed components can improve tracking performance, although they do not necessarily establish the mechanisms claimed in the motivation.

Critical weaknesses

The central motivation and the claimed geometric alignment are not directly validated.

In Lines 65–72, the authors argue that previous methods use calibrated camera geometry primarily as an auxiliary representation cue, which limits effective cross-view information exchange under wide baselines and asymmetric occlusions. However, the paper does not directly establish that the auxiliary use of geometry is responsible for these limitations.

In particular, the paper does not demonstrate that previous methods attend to geometrically inconsistent or unreliable cross-view evidence. The improvements obtained by introducing epipolar guidance do not necessarily imply that such incorrect information exchange was the underlying problem.

Similarly, although the proposed GTA-based module applies geometric transformations to feature representations, it is unclear whether these learned features actually exhibit the geometric transformation properties assumed by the method. Applying SE(3), SO(3), and SO(2) representations to projected correlation features does not by itself guarantee meaningful geometric alignment. The reported ablations establish performance improvements, but do not directly demonstrate that geometric alignment is achieved.

Furthermore, the motivation emphasizes the benefits of directly incorporating geometric constraints, whereas Table 5 shows that applying epipolar guidance to the iterative update module can degrade tracking performance. The authors attribute this to the suppression of useful information when current trajectory estimates are inaccurate. This suggests that explicit geometric guidance can also introduce limitations, which are not sufficiently discussed in the motivation.

The proposed epipolar guidance has important limitations that are not adequately analyzed.

The proposed EpiGTA uses the Sampson distance between current trajectory estimates to downweight geometrically inconsistent observations. However, epipolar compatibility is not equivalent to observation reliability.

First, epipolar constraints cannot directly identify occlusions. Even when a point is occluded in one view, its correct projected location can still satisfy the epipolar constraint. Therefore, an occluded view may contribute unreliable correlation features without receiving a geometric penalty.

Second, the Sampson distance only measures inconsistency relative to the epipolar line. An incorrect trajectory estimate can drift along the epipolar line while maintaining a small epipolar error.

Third, the attention mechanism may have difficulty identifying errors in the target-view trajectory itself. When the target-view estimate is incorrect, its epipolar distances to multiple source views may all increase. Since the attention weights are normalized using softmax, a common increase in the penalties does not necessarily suppress unreliable information.

These limitations raise questions about whether the proposed epipolar guidance can reliably distinguish useful cross-view evidence from unreliable observations. The paper does not directly analyze the relationship between epipolar compatibility, tracking errors, occlusions, and attention weights.

Moreover, although the method uses calibrated multi-view observations, it does not directly incorporate multi-view reprojection consistency during trajectory estimation. The motivation for choosing pairwise epipolar compatibility over alternative multi-view consistency measures is not sufficiently established.

Questions for authors

Q1. What evidence supports the claim that the auxiliary use of geometry is a major limitation of existing multi-view trackers?

In Lines 65–72, the authors attribute limitations in cross-view information exchange to previous methods using camera geometry primarily as an auxiliary representation cue.

However, the paper does not directly demonstrate this failure mechanism. Could the authors provide an analysis showing that existing approaches aggregate geometrically inconsistent cross-view evidence?

Additionally, could the authors compare no geometric encoding, Plücker-based encoding, and the proposed GTA-based encoding within the same architecture and training setup? This would help distinguish the effects of the geometric representation from other architectural changes.

Could the authors also clarify whether the R1 baseline in Tables 3–4 retains the Plücker-based geometry encoding from MV-TAP?

Q2. Does the proposed geometric transformation actually produce geometrically aligned feature representations?

The proposed GTA mechanism applies transformations based on camera poses and image coordinates to projected feature representations. However, it is unclear whether these features are actually learned to transform consistently under the corresponding geometric transformations.

Could the authors provide direct evidence of the claimed geometric alignment, rather than relying only on downstream tracking accuracy?

For example, an analysis of feature transformation consistency or experiments perturbing or shuffling camera extrinsics could help clarify how the model uses the geometric transformations.

Q3. How does epipolar guidance handle occlusions and incorrect trajectory estimates?

Since an occluded point can still satisfy the epipolar constraint, it is unclear how the proposed epipolar bias suppresses unreliable correlation features from occluded views.

Similarly, errors along the epipolar line may not be captured by the Sampson distance.

Could the authors analyze the effectiveness of epipolar guidance under different occlusion ratios and tracking error patterns? In particular, does a larger epipolar distance actually correlate with less reliable cross-view observations?

The method also predicts visibility, but the EpiGTA attention weights in Eq. (2) do not explicitly use the predicted visibility. Could the authors explain this design choice?

Q4. How does EpiGTA handle an inaccurate target-view trajectory?

In Eq. (2), the attention weights are computed by applying softmax over source views.

When the target-view trajectory estimate is inaccurate, the epipolar distances to several source views may simultaneously become large. Since softmax is invariant to a common additive shift in its logits, this does not necessarily indicate to the model that the target-view estimate itself is unreliable.

Could the authors clarify how this case is handled?

Additionally, does the source-view set include the target view itself? If so, how is self-attention affected when the target-view trajectory estimate is incorrect?

Q5. Why is pairwise epipolar compatibility preferred over direct multi-view geometric consistency?

The method uses pairwise Sampson distances to guide cross-view information exchange. However, given calibrated observations from multiple cameras, an alternative would be to triangulate a 3D point using other views and measure its reprojection error in the target view.

Such a formulation could potentially identify view-specific outliers and errors that remain consistent with an individual epipolar line.

Could the authors explain the motivation for using pairwise epipolar guidance rather than multi-view reprojection consistency? A comparison with leave-one-out reprojection errors or a reprojection consistency loss would help clarify this design choice.

Q6. How should the results in Table 5 be reconciled with the paper's motivation?

The paper motivates its approach by arguing that geometry should directly guide cross-view interactions rather than serve only as an auxiliary representation cue.

However, Table 5 indicates that introducing epipolar guidance into the iterative update module can reduce tracking performance, and the authors explain that inaccurate current estimates may cause the geometric constraint to suppress useful information.

Does this suggest that the benefit of explicit geometric guidance depends strongly on where and how it is applied? Could the authors elaborate on why epipolar guidance is beneficial during correlation aggregation but detrimental during point-state updates?

7 comments
AuthorsOpenReview ↗

Response to Reviewer npiZ (Part 1/7)

W1.1 & Q1.1: Core motivation for geometry-guided interactions

Motivation. Thank you for raising this point. Viewpoint-dependent appearance and visibility differences complicate the association and integration of complementary observations. Our motivation therefore centers on how geometry structures cross-view interactions by establishing a shared geometric reference within attention for feature comparison and aggregation. While MV-TAP [1] supplies camera geometry as auxiliary token context, GeoMVT directly transforms attention features according to relative camera geometry. Epipolar guidance further modulates correlation exchange according to the compatibility of current trajectory hypotheses. Together, these mechanisms make calibrated geometry an explicit basis for cross-view information exchange as trajectories evolve.

Experimental analysis. Our camera-layout experiment (Table R1, manuscript Table 8) evaluates this design under increasing viewpoint differences. From compact to wide layouts, AJ decreases from 45.9 to 43.9 for MV-TAP and from 46.6 to 44.5 for aggregation without geometric guidance. GeoMVT maintains 50.5–50.9 AJ across layouts, widening its advantage over the latter from 4.0 to 6.4 points. This supports the value of explicitly structuring interactions when appearance differences make cross-view integration more challenging.

Table R1. Effect of camera layout on Panoptic Studio.

Camera layoutMean baselineMedian angleMV-TAPw/o Geow/ GeoCompact1.9,m42.1∘45.946.650.6Medium2.9,m59.9∘43.044.950.5Wide3.8,m81.6∘43.944.550.9\scriptsize\begin{array}{cccccc}\text{Camera layout}&\text{Mean baseline}&\text{Median angle}&\text{MV-TAP}&\text{w/o Geo}&\text{w/ Geo}\\\\\hline \text{Compact}&1.9\\,\mathrm{m}&42.1^\circ&45.9&46.6&\mathbf{50.6}\\\\\text{Medium}&2.9\\,\mathrm{m}&59.9^\circ&43.0&44.9&\mathbf{50.5}\\\\\text{Wide}&3.8\\,\mathrm{m}&81.6^\circ&43.9&44.5&\mathbf{50.9}\end{array}

The view-count experiment provides complementary evidence (manuscript Fig. 7 and Table R2, Appendix Table B.18). With one view, GeoMVT and MV-TAP differ by at most 0.5 AJ across all four datasets. With eight views, GeoMVT gains 5.5–8.6 AJ, indicating that its advantage emerges from integrating additional observations rather than improving single-view tracking alone.

Together, these results support our central hypothesis: directly incorporating camera geometry into cross-view interactions enables more effective exploitation of multi-view complementarity.

Table R2. Tracking performance with different numbers of input views.

ViewsPanoptic StudioDexYCBMV-KubricHarmony4D012345678MV-TAPOursΔAJ39.038.9−0.139.243.44.140.245.45.242.246.94.743.348.14.944.849.95.243.450.36.943.950.97.0MV-TAPOursΔAJ35.836.30.537.539.92.441.745.13.441.645.74.142.546.84.244.248.94.845.050.35.345.951.45.5MV-TAPOursΔAJ46.145.9−0.248.251.53.249.354.14.749.255.36.150.657.46.851.358.77.451.159.18.050.959.58.6MV-TAPOursΔAJ32.032.50.534.737.22.434.637.93.335.739.94.137.541.94.439.943.83.938.944.05.138.544.76.2\scriptsize\begin{array}{c|c|c|c|c}\text{Views}&\text{Panoptic Studio}&\text{DexYCB}&\text{MV-Kubric}&\text{Harmony4D}\\\\\hline \begin{array}{c}\phantom{0}\\\\1\\\\2\\\\3\\\\4\\\\5\\\\6\\\\7\\\\8\end{array}&\begin{array}{ccc}\text{MV-TAP}&\text{Ours}&\Delta\text{AJ}\\\\\hline 39.0&38.9&-0.1\\\\39.2&43.4&4.1\\\\40.2&45.4&5.2\\\\42.2&46.9&4.7\\\\43.3&48.1&4.9\\\\\mathbf{44.8}&49.9&5.2\\\\43.4&50.3&6.9\\\\43.9&\mathbf{50.9}&\mathbf{7.0}\end{array}&\begin{array}{ccc}\text{MV-TAP}&\text{Ours}&\Delta\text{AJ}\\\\\hline 35.8&36.3&0.5\\\\37.5&39.9&2.4\\\\41.7&45.1&3.4\\\\41.6&45.7&4.1\\\\42.5&46.8&4.2\\\\44.2&48.9&4.8\\\\45.0&50.3&5.3\\\\\mathbf{45.9}&\mathbf{51.4}&\mathbf{5.5}\end{array}&\begin{array}{ccc}\text{MV-TAP}&\text{Ours}&\Delta\text{AJ}\\\\\hline 46.1&45.9&-0.2\\\\48.2&51.5&3.2\\\\49.3&54.1&4.7\\\\49.2&55.3&6.1\\\\50.6&57.4&6.8\\\\\mathbf{51.3}&58.7&7.4\\\\51.1&59.1&8.0\\\\50.9&\mathbf{59.5}&\mathbf{8.6}\end{array}&\begin{array}{ccc}\text{MV-TAP}&\text{Ours}&\Delta\text{AJ}\\\\\hline 32.0&32.5&0.5\\\\34.7&37.2&2.4\\\\34.6&37.9&3.3\\\\35.7&39.9&4.1\\\\37.5&41.9&4.4\\\\\mathbf{39.9}&43.8&3.9\\\\38.9&44.0&5.1\\\\38.5&\mathbf{44.7}&\mathbf{6.2}\end{array}\end{array}

References.

  • [1] MV-TAP: Tracking Any Point in Multi-View Videos, CVPR 2026.
AuthorsOpenReview ↗

Response to Reviewer npiZ (Part 2/7)

Q1.2: Comparison of geometric encodings

Table R3 (manuscript Table 6 and Appendix Table B.8) shows that Plücker encoding [1] provides no consistent improvement over no encoding, whereas GTA improves all metrics on all four datasets. Plücker representations depend on the global coordinate frame and supply geometry only as auxiliary token context. GTA instead incorporates relative camera geometry directly into cross-view attention, supporting more effective iterative state updates.

Table R3. Geometric encoding ablation in GIU.

EncodingPanoptic StudioDexYCBMV-KubricHarmony4D0w/o encodingw/ Plu¨ckerw/ GTAAJ↑δavg↑OA↑44.067.175.243.967.474.848.770.580.0AJ↑δavg↑OA↑46.463.678.545.963.377.749.767.583.6AJ↑δavg↑OA↑51.570.082.750.969.382.357.375.586.5AJ↑δavg↑OA↑39.171.468.738.571.069.043.173.972.2\scriptsize\begin{array}{c|c|c|c|c}\text{Encoding}&\text{Panoptic Studio}&\text{DexYCB}&\text{MV-Kubric}&\text{Harmony4D}\\\\\hline \begin{array}{l}\phantom{0}\\\\\text{w/o encoding}\\\\\text{w/ Plücker}\\\\\text{w/ GTA}\end{array}&\begin{array}{ccc}\text{AJ}\uparrow&\delta_{\mathrm{avg}}\uparrow&\text{OA}\uparrow\\\\\hline 44.0&67.1&75.2\\\\43.9&67.4&74.8\\\\\mathbf{48.7}&\mathbf{70.5}&\mathbf{80.0}\end{array}&\begin{array}{ccc}\text{AJ}\uparrow&\delta_{\mathrm{avg}}\uparrow&\text{OA}\uparrow\\\\\hline 46.4&63.6&78.5\\\\45.9&63.3&77.7\\\\\mathbf{49.7}&\mathbf{67.5}&\mathbf{83.6}\end{array}&\begin{array}{ccc}\text{AJ}\uparrow&\delta_{\mathrm{avg}}\uparrow&\text{OA}\uparrow\\\\\hline 51.5&70.0&82.7\\\\50.9&69.3&82.3\\\\\mathbf{57.3}&\mathbf{75.5}&\mathbf{86.5}\end{array}&\begin{array}{ccc}\text{AJ}\uparrow&\delta_{\mathrm{avg}}\uparrow&\text{OA}\uparrow\\\\\hline 39.1&71.4&68.7\\\\38.5&71.0&69.0\\\\\mathbf{43.1}&\mathbf{73.9}&\mathbf{72.2}\end{array}\end{array}

Q1.3: Use of Plücker-based geometry encoding

We do not retain Plücker-based geometry encoding in the R1 baseline in Tables 3–4 of the manuscript (Tables 3–4).

References.

  • [1] MV-TAP: Tracking Any Point in Multi-View Videos, CVPR 2026.
AuthorsOpenReview ↗

Response to Reviewer npiZ (Part 3/7)

W1.2 & Q2: Geometric alignment of feature representations

Thank you for the question. By geometric alignment, we refer to establishing a shared geometric reference within attention through relative transformations. This structures feature comparison and aggregation according to camera relations, facilitating cross view association and information exchange rather than imposing strict equivariance on the learned features [1]. We assess this mechanism through two complementary analyses.

Correspondence retrieval. To examine whether this geometric reference improves correspondence discrimination, we compare correlation aggregation with and without GTA, disabling epipolar guidance and GTA in the iterative transformer. Tables R4 and R5 (Appendix Tables B.1 and B.2) show improvements in both R@1 and R@5 across every reported viewing angle and relative baseline group. Overall R@1 increases from 7.49% to 17.37% on Panoptic Studio and from 7.77% to 19.14% on DexYCB. These gains support the effectiveness of relative geometric transformations in learning more discriminative cross view feature affinities.

Table R4. Q–K correspondence retrieval on Panoptic Studio.

GTAViewing angle (°)Relative baseline00×✓0–3030–6060–9090–120120–180R@1R@57.3718.1221.5539.12R@1R@55.3918.3215.3134.70R@1R@57.1622.5819.5842.79R@1R@58.3725.3018.0542.22R@1R@510.5128.1916.0638.680.5–11–1.5R@1R@56.1120.0416.7437.79R@1R@59.1526.2218.1141.56\scriptsize\begin{array}{c|c|c}\text{GTA}&\text{Viewing angle (°)}&\text{Relative baseline}\\\\\hline \begin{array}{c}\phantom{0}\\\\\phantom{0}\\\\\times\\\\\checkmark\end{array}&\begin{array}{ccccc}\text{0–30}&\text{30–60}&\text{60–90}&\text{90–120}&\text{120–180}\\\\\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 7.37&18.12\\\\\mathbf{21.55}&\mathbf{39.12}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 5.39&18.32\\\\\mathbf{15.31}&\mathbf{34.70}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 7.16&22.58\\\\\mathbf{19.58}&\mathbf{42.79}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 8.37&25.30\\\\\mathbf{18.05}&\mathbf{42.22}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 10.51&28.19\\\\\mathbf{16.06}&\mathbf{38.68}\end{array}\end{array}&\begin{array}{cc}\text{0.5–1}&\text{1–1.5}\\\\\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 6.11&20.04\\\\\mathbf{16.74}&\mathbf{37.79}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 9.15&26.22\\\\\mathbf{18.11}&\mathbf{41.56}\end{array}\end{array}\end{array}

Table R5. Q–K correspondence retrieval on DexYCB.

GTAViewing angle (°)Relative baseline00×✓0–3030–6060–9090–120120–180R@1R@55.0215.189.7925.00R@1R@56.3218.2515.9233.82R@1R@57.9121.6722.4444.57R@1R@59.2123.8419.7040.53R@1R@57.7022.4015.0035.690–0.50.5–11–1.5≥1.5R@1R@57.0220.4417.7938.96R@1R@56.7319.5117.7837.67R@1R@58.7523.1621.3242.60R@1R@59.1723.1215.8435.52\scriptsize\begin{array}{c|c|c}\text{GTA}&\text{Viewing angle (°)}&\text{Relative baseline}\\\\\hline \begin{array}{c}\phantom{0}\\\\\phantom{0}\\\\\times\\\\\checkmark\end{array}&\begin{array}{ccccc}\text{0–30}&\text{30–60}&\text{60–90}&\text{90–120}&\text{120–180}\\\\\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 5.02&15.18\\\\\mathbf{9.79}&\mathbf{25.00}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 6.32&18.25\\\\\mathbf{15.92}&\mathbf{33.82}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 7.91&21.67\\\\\mathbf{22.44}&\mathbf{44.57}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 9.21&23.84\\\\\mathbf{19.70}&\mathbf{40.53}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 7.70&22.40\\\\\mathbf{15.00}&\mathbf{35.69}\end{array}\end{array}&\begin{array}{cccc}\text{0–0.5}&\text{0.5–1}&\text{1–1.5}&\text{≥1.5}\\\\\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 7.02&20.44\\\\\mathbf{17.79}&\mathbf{38.96}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 6.73&19.51\\\\\mathbf{17.78}&\mathbf{37.67}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 8.75&23.16\\\\\mathbf{21.32}&\mathbf{42.60}\end{array}&\begin{array}{cc}\text{R@1}&\text{R@5}\\\\\hline 9.17&23.12\\\\\mathbf{15.84}&\mathbf{35.52}\end{array}\end{array}\end{array}

Camera geometry analysis. Controlled rotation and position perturbations further examine whether the framework depends on the supplied camera geometry (Appendix B.2, Figs. B.2 to B.7). GeoMVT’s tracking accuracy changes with calibration errors, whereas the baseline changes little, and GeoMVT retains its advantage under moderate perturbations.

Together, the retrieval and perturbation analyses provide complementary evidence: relative geometric transformations improve correspondence discrimination, and calibrated camera relations meaningfully influence the resulting tracking predictions.

W2.1: Epipolar compatibility and observation reliability

Appendix Fig. B.9 shows a strong association between epipolar distance and observation reliability on Panoptic Studio. A pair is unreliable if either point's localization error exceeds 4 px. Unreliable pairs increase from 33.8% at square-root Sampson distances of 0–0.5 px to 98.7% at 4–8 px. This supports epipolar compatibility as a reliability cue for suppressing inconsistent cross-view correlations.

References.

  • [1] GTA: A Geometry-Aware Attention Mechanism for Multi-View Transformers, ICLR 2024.

Comments on the paper

1 comment

Dear Reviewers,

We thank you for your careful evaluation and constructive feedback. We have revised the manuscript and added experiments and analyses to clarify GeoMVT's contributions, geometric mechanisms, and robustness. Revisions are highlighted in blue.

Alongside our detailed responses to each reviewer, we summarize the main updates below. The locations refer to the revised manuscript and Appendix.

  • Fixes to the main paper (npiZ, 8qrx, xQKa). We strengthened the motivation for using geometric transformations to establish a shared reference for cross-view interactions (lines 65–67), clarified baseline configurations, replaced “the first” with “a” in the Fig. 1 caption, and corrected the MV-Kubric MTE scale.

  • Effectiveness and contribution of geometric transformations (npiZ, xQKa). We expanded the component ablations to distinguish geometric transformations in CCA and GIU, epipolar guidance, and their two-stage combination, reporting both 2D and 3D metrics. We also compared no encoding, Plücker encoding, and GTA in GIU. See main Tables 3–4 and 6, and Appendix Tables B.3 and B.8.

  • Direct evidence of geometric transformations and geometry awareness (npiZ). We added Q–K correspondence retrieval across viewing angles and relative baselines, together with attention visualizations. We clarified that geometric alignment concerns geometry-aware comparison and aggregation within attention, rather than guaranteed equivariance of learned features. See Appendix Tables B.1–B.2 and Fig. B.1, with camera-geometry analyses in Appendix B.2.

  • Effectiveness and limitations of epipolar guidance (npiZ). We added analyses of observation reliability, occlusion, directional tracking errors, and inaccurate initialization. Attention-weight comparisons further explain why epipolar guidance suppresses unreliable sources in CCA but can restrict corrective information in GIU. See main Fig. 6, Appendix Figs. B.9–B.10, and Tables B.4–B.5 and B.11–B.13.

  • Cross-view geometric consistency and 3D tracking improvements (xQKa). We added reprojection-error comparisons on matched tracks within the same 2D error bins across four datasets. Together with the expanded 3D component ablations, these results provide further evidence of improved cross-view consistency and 3D recovery. See main Table 3 and Appendix Tables B.3 and B.20.

  • Robustness to calibration and correspondence errors (npiZ, xQKa). We added 3D AJ and MTE results under camera rotation and translation perturbations, and evaluated noisy cross-view queries while retaining one ground-truth anchor view. We also isolated EpiGTA's effectiveness under inaccurate initialization. See Appendix Figs. B.4–B.7 and Tables B.13 and B.21.

  • 3D tracking under depth noise (8qrx). We added controlled perturbations to MVTracker's native depth inputs on three datasets, showing how increasing depth noise degrades its 3D tracking performance and highlighting the benefit of depth-free recovery. Depth and motion statistics further contextualize the dataset-specific comparisons. See Appendix Table A.2.

  • Pairwise epipolar guidance versus reprojection consistency (npiZ). We compared epipolar guidance with a triangulation-based reprojection bias in CCA. The results support assessing pairwise compatibility without relying on an intermediate 3D reconstruction during correlation aggregation. See main Table 5 and Appendix Table B.7.

We hope these revisions and additional results clarify our contribution and resolve the reviewers' concerns, and thus help raise their scores accordingly. We welcome further discussion with the reviewers and would be happy to provide any additional clarification. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.