- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper addresses long-horizon 3D reconstruction from video using frozen feed-forward 3D backbones (e.g., VGGT-Omega, LingBot-MAP). Such backbones are limited to short windows, so long sequences are processed as overlapping chunks. Existing chunk-level Sim(3) alignment methods preserve each chunk’s internal camera configuration up to scale, which cannot resolve conflicting pose estimates for the same physical frame observed in multiple overlapping windows. The paper proposes HiP3R, a hierarchical pose-graph optimization framework. Stage I optimizes a chunk-level Sim(3) pose graph with overlap and loop constraints to recover a globally consistent layout and scale. Stage II fixes this coarse layout and optimizes a frame-level shared-camera SE(3) pose graph, assigning one camera variable per physical frame, with placement priors and within-chunk motion factors to reconcile conflicting predictions. The method also introduces progressive loop scheduling and bounded point sampling for stable and scalable optimization without retraining. Experiments on seven public benchmarks (61 sequences) show large trajectory and geometry improvements over strong baselines, e.g., reducing VGGT-Omega ATE from 18.555 m to 4.906 m and RPE_trans from 0.9878 m to 0.0622 m. Ablations demonstrate the benefits of shared-camera refinement, progressive loop scheduling, and bounded sampling.
Critical strengths
- The shared-camera SE(3) pose graph assigns one camera variable per physical frame and jointly optimizes placement priors and within-chunk motion factors. This directly addresses the limitation of chunk-level Sim(3) alignment and yields substantial reductions in ATE and relative pose errors across diverse indoor and outdoor benchmarks. The ablation in Table 5 clearly shows the benefit over single, unary, and motion-only variants.
- The method is evaluated on 61 sequences across seven public datasets, including long KITTI and VBR traversals. It integrates with two different frozen backends without retraining. The progressive loop scheduling and bounded point sampling reduce graph build and solve time by 83.62% with unchanged accuracy, making the approach practical for kilometer-scale sequences. The stage-wise analysis convincingly separates the roles of coarse layout recovery and fine trajectory refinement.
Critical weaknesses
- The method is a post-processing framework and inherits errors from the frozen feed-forward backbone and visual place recognition. While the paper uses robust losses and progressive scheduling, it does not systematically evaluate failure cases when loop closures are severely incorrect or when the backend produces grossly inconsistent chunks. The perturbation experiment in Appendix K is a good start, but more analysis on real failure modes would strengthen the claims.
Questions for authors
-
How does HiP3R behave when a significant fraction of loop closures are false positives or have large scale errors? Can you provide more quantitative failure analysis or a detection/rejection mechanism beyond the current robust weighting?
-
For sequences longer than 2000 frames, you use a stride-5 keyframe subgraph with interpolated corrections. What is the trade-off in fine-grained motion accuracy compared with a dense graph on even longer sequences (e.g., VBR)? Can you report more dense-vs-keyframe comparisons?
-
Table 12 reports stage-wise timings, but what is the total wall-clock time of HiP3R compared with VGGT-Long and GlueMap on the same hardware, including loop detection and graph construction?
We thank Reviewer 79xj for the positive assessment and constructive feedback. We appreciate the recognition of the effectiveness of our shared-camera refinement, the breadth of our evaluation, and the efficiency and scalability of our framework for long-horizon reconstruction. We address the concerns point by point below.
W1 / Q1: Behaviour under false and missing loop closures
Thanks for this insightful suggestion. We have evaluated wrong-endpoint and missing-loop conditions with fixed predictions on VBR sequences. The results show that wrong accepted constraints can substantially distort the coarse layout, which the fixed-layout refinement cannot independently restore.
These results motivate more conservative loop selection. Future work can prioritize filtering low-confidence loop constraints before optimization to reduce the risk of global layout distortion.
W2 / Q2: Dense versus stride-5 keyframe refinement
On the seven VBR sequences, stride-5 preserves global accuracy with a small trade-off in relative pose accuracy and lower memory use. It reduces solver time, but keyframe construction and interpolation offset this saving in our current implementation.
W3 / Q3: Total wall-clock time
We measured total runtime on a 320-frame Oxford sequence, including startup, fresh loop detection, neural inference, graph construction, optimization, and result writing. All variants use a single GPU. For fair comparison, serial variants use one CPU worker/thread. HiP3R also supports CPU-parallel acceleration.
We hope our responses have addressed the reviewer's concerns. If anything remains unclear or if you have further questions, we'd be happy to clarify and discuss them in more detail.