acceptodds
Under review as a conference paper at ICLR 2027

HiP3R: Hierarchical Pose Graphs for Long-Horizon 3D Reconstruction

Abstract

Feed-forward models accurately estimate camera motion and 3D geometry from short video windows. However, computational and memory costs prevent direct inference on sequences containing thousands of frames. Long videos must therefore be processed as overlapping chunks. This division produces independent local reconstructions with inconsistent coordinate frames and duplicate estimates of shared cameras. Achieving global consistency requires propagating constraints from adjacent overlaps and distant revisits across the sequence. This requirement naturally motivates pose graph optimization. However, the choice of graph level creates a trade-end. Chunk-level graphs efficiently resolve scale drift and close large loops. Yet, their rigid transformations cannot correct internal errors or conflicting frame predictions. Conversely, frame-level optimization offers local flexibility, but scaling it globally is computationally expensive and can break metric scale. HiP3R resolves this trade-off without retraining the underlying models, using a hierarchical pose graph. HiP3R first optimizes a robust chunk-level graph to establish the global layout and scale. It then holds this layout fixed and estimates one shared camera per frame, turning overlapping window predictions into a single trajectory. As a post-processing method that does not retrain the backend, HiP3R applies to both chunk-based and streaming models. Across seven benchmarks, HiP3R reduces VGGT-Omega average trajectory error by , from to m, and relative tracking drift by , from to m. It also decreases 3D surface reconstruction error by , from to m. When paired with LingBot-MAP, HiP3R reduces trajectory error by , from to m, and achieves the lowest overall error. It executes faster than existing alignment methods on long sequences.

Primary area:
generative models
Keywords:
Long-Horizon 3D Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_79xjOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses long-horizon 3D reconstruction from video using frozen feed-forward 3D backbones (e.g., VGGT-Omega, LingBot-MAP). Such backbones are limited to short windows, so long sequences are processed as overlapping chunks. Existing chunk-level Sim(3) alignment methods preserve each chunk’s internal camera configuration up to scale, which cannot resolve conflicting pose estimates for the same physical frame observed in multiple overlapping windows. The paper proposes HiP3R, a hierarchical pose-graph optimization framework. Stage I optimizes a chunk-level Sim(3) pose graph with overlap and loop constraints to recover a globally consistent layout and scale. Stage II fixes this coarse layout and optimizes a frame-level shared-camera SE(3) pose graph, assigning one camera variable per physical frame, with placement priors and within-chunk motion factors to reconcile conflicting predictions. The method also introduces progressive loop scheduling and bounded point sampling for stable and scalable optimization without retraining. Experiments on seven public benchmarks (61 sequences) show large trajectory and geometry improvements over strong baselines, e.g., reducing VGGT-Omega ATE from 18.555 m to 4.906 m and RPE_trans from 0.9878 m to 0.0622 m. Ablations demonstrate the benefits of shared-camera refinement, progressive loop scheduling, and bounded sampling.

Critical strengths
  1. The shared-camera SE(3) pose graph assigns one camera variable per physical frame and jointly optimizes placement priors and within-chunk motion factors. This directly addresses the limitation of chunk-level Sim(3) alignment and yields substantial reductions in ATE and relative pose errors across diverse indoor and outdoor benchmarks. The ablation in Table 5 clearly shows the benefit over single, unary, and motion-only variants.
  2. The method is evaluated on 61 sequences across seven public datasets, including long KITTI and VBR traversals. It integrates with two different frozen backends without retraining. The progressive loop scheduling and bounded point sampling reduce graph build and solve time by 83.62% with unchanged accuracy, making the approach practical for kilometer-scale sequences. The stage-wise analysis convincingly separates the roles of coarse layout recovery and fine trajectory refinement.
Critical weaknesses
  1. The method is a post-processing framework and inherits errors from the frozen feed-forward backbone and visual place recognition. While the paper uses robust losses and progressive scheduling, it does not systematically evaluate failure cases when loop closures are severely incorrect or when the backend produces grossly inconsistent chunks. The perturbation experiment in Appendix K is a good start, but more analysis on real failure modes would strengthen the claims.
Questions for authors
  1. How does HiP3R behave when a significant fraction of loop closures are false positives or have large scale errors? Can you provide more quantitative failure analysis or a detection/rejection mechanism beyond the current robust weighting?

  2. For sequences longer than 2000 frames, you use a stride-5 keyframe subgraph with interpolated corrections. What is the trade-off in fine-grained motion accuracy compared with a dense graph on even longer sequences (e.g., VBR)? Can you report more dense-vs-keyframe comparisons?

  3. Table 12 reports stage-wise timings, but what is the total wall-clock time of HiP3R compared with VGGT-Long and GlueMap on the same hardware, including loop detection and graph construction?

1 comment

We thank Reviewer 79xj for the positive assessment and constructive feedback. We appreciate the recognition of the effectiveness of our shared-camera refinement, the breadth of our evaluation, and the efficiency and scalability of our framework for long-horizon reconstruction. We address the concerns point by point below.

W1 / Q1: Behaviour under false and missing loop closures

Thanks for this insightful suggestion. We have evaluated wrong-endpoint and missing-loop conditions with fixed predictions on VBR sequences. The results show that wrong accepted constraints can substantially distort the coarse layout, which the fixed-layout refinement cannot independently restore.

Loops affected012345
Missing loops, ATE (m)22.4522.4022.4122.4222.4722.48
Wrong loops, ATE (m)22.4556.3553.8158.0677.0785.65

These results motivate more conservative loop selection. Future work can prioritize filtering low-confidence loop constraints before optimization to reduce the risk of global layout distortion.

W2 / Q2: Dense versus stride-5 keyframe refinement

On the seven VBR sequences, stride-5 preserves global accuracy with a small trade-off in relative pose accuracy and lower memory use. It reduces solver time, but keyframe construction and interpolation offset this saving in our current implementation.

MetricDenseStride-5
ATE (m)22.447722.4473
Translation RPE (m)0.057660.05910
Rotation RPE (deg)0.58310.5867
Solver time (s)0.9310.215
Fine graph construction, solve and interpolation (s)1.2481.955
CPU fine-replay peak RSS (MiB)474.2300.6

W3 / Q3: Total wall-clock time

We measured total runtime on a 320-frame Oxford sequence, including startup, fresh loop detection, neural inference, graph construction, optimization, and result writing. All variants use a single GPU. For fair comparison, serial variants use one CPU worker/thread. HiP3R also supports CPU-parallel acceleration.

WorkflowWall-clock (s)
HiP3R, single CPU worker/thread160.44
HiP3R, CPU-parallel implementation134.04
VGGT-Long, single CPU worker/thread404.15
GlueMap, single CPU worker/thread2317.10

We hope our responses have addressed the reviewer's concerns. If anything remains unclear or if you have further questions, we'd be happy to clarify and discuss them in more detail.

Reviewer_myEKOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents HiP3R, a two-stage pose graph method for long-sequence 3D reconstruction. The first stage aligns overlapping chunks through a Sim(3) graph with overlap and loop constraints. The second stage fixes the chunk transformations and refines one shared SE(3) camera pose per frame. This refinement combines placement priors with relative motion constraints within each chunk. The authors evaluate the method with frozen VGGT-Omega and LingBot-MAP models on seven datasets, reporting improvements in trajectory accuracy and dense reconstruction.

Critical strengths

The two-stage design is practical and effective. It first recovers the global layout through chunk alignment, then improves local pose accuracy through lightweight refinement, as shown in Table 6.

Critical weaknesses
  1. Limited technical contribution. Chunk alignment and global Sim(3) optimization already exist in VGGT-Long. HiP3R adds solver changes and shared-camera refinement. These additions provide practical gains, but their methodological significance remains unclear. In particular, Table 5 shows modest gains from shared-camera refinement over simple pose averaging. The paper provides little analysis of when this refinement is important or what broader insight it offers.

  2. The presentation limits assessment of the claims. Extensive implementation details obscure the main contribution, while some experimental settings remain unclear. For example, "Style Control" is not clearly defined, and Tables 13–15 contain unexplained numerical rows. Figures 3 and 4 compare only with VGGT-Omega, leaving the visual advantages over direct chunk-based competitors unclear. The supplementary videos also do not clearly connect the displayed results to the claimed contributions. These issues make the method and its reported advantages difficult to assess.

Questions for authors
  1. Tables 6 and 21 attribute most global gains to coarse alignment. Table 5 shows modest refinement gains over pose averaging. What evidence establishes the value of the hierarchy itself, beyond improvements to the coarse solver?

  2. Please clarify the meaning of the unlabeled numbers in Tables 13–15.

  3. Can you provide annotated comparisons with direct chunk-based competitors, beyond VGGT-Omega? Please also consolidate the many tables and highlight the results that support the central claims.

7 comments

We thank Reviewer myEK for the careful review and constructive feedback. We appreciate the recognition of the practicality and effectiveness of our two-stage design in recovering the global layout and improving local pose accuracy. We address the concerns point by point below.

W1 / Q1: Significance of shared-camera refinement

Unary robustly averages overlapping camera predictions independently for each frame. Changes in the contributing observations can leave jumps at chunk boundaries. Shared couples adjacent cameras through within-chunk motion constraints while placement priors anchor them to the recovered layout, reducing boundary discontinuities and improving local trajectory consistency. We have added boundary-level analysis in Appendix L. The table below reports translation RPE at chunk boundaries for VGGT-Omega:

DatasetUnary (m)Shared (m)Error reduction
ETH3D0.09650.056441.53%
7-Scenes0.02300.014337.77%
Neural RGB-D0.01580.007949.85%
Tanks and Temples0.03710.03495.99%
Oxford0.29460.151248.68%
KITTI0.38230.167756.13%
VBR0.25800.115055.43%

Motion-only constraints lower local error but worsen global ATE, while our joint objective preserves the layout and improves local motion.

W2 / Q2: Unlabeled numbers in Tables 13–15

Thank you for pointing this out. The unlabeled numbers came from an obsolete three-line LaTeX row whose final line was left uncommented. These remnants have been removed without changing the other reported results.

W2 / Q3: Annotated comparisons and table consolidation

''Style Control'' has been renamed ''coarse solver reference''. The coarse-only comparison uses fixed predictions, loop candidates, and initialization, with subsequent rows adding components cumulatively. Figures 3 and 4 now include VGGT-Long, and Figure 3 provides stage-wise comparisons showing the effects of layout optimization and trajectory refinement. Supplementary reconstruction results and illustrations are available on our demo page: https://hip3r-anonymous.github.io. We consolidated the keyframe comparisons into Table 8 and the per-dataset geometry comparisons across both backends into Table 16, and removed repeated results. Tables 1, 2, 4, 5, and 18 now mark the best and second-best results within comparable settings with bold and underlining, respectively. Tables 3 and 6 highlight the best results in bold.

We hope our responses have addressed the reviewer's concerns. If anything remains unclear or if you have further questions, we'd be happy to clarify and discuss them in more detail.

Thank you for the clarifications. After reviewing your responses alongside the other reviewers' comments, my concerns remain unresolved, and I also noticed several questionable experimental settings:

1. Questionable evaluation setup (Reviewer e6K5 W3/Q4; W2/Q3):

  • Macro-averaging across datasets: Unlike prior works (e.g., VGGT-Long) that directly report metrics on each long-horizon benchmark (KITTI, VBR, Oxford), the main table (Table 1) averages across seven very different datasets.
  • Single KITTI sequence with disproportionate weight: KITTI is evaluated on only 1 sequence (seq 00) (Table 10), yet it carries a full 1/7 weight in the Table 1 macro-average (while gaining only 4.5% over loop-enabled VGGT-Long in Table 2, as noted by Reviewer e6K5). Why was only one KITTI sequence evaluated when prior works like VGGT-Long evaluate across the full KITTI sequences?
  • Inactive sequences included in ETH3D: In ETH3D, 10 of the 11 sequences fit within a single chunk (<60 frames, Page 17), where the design of HiP3R is completely inactive, yet they are included in the benchmark evaluation.

2. Unclear novelty of the hierarchical design (Reviewer e6K5 W1/Q1; W1/Q1):

  • As Reviewer e6K5 also noted, the core graph components are already standard in VGGT-Long and PoseSLAM.
  • What is the actual methodological novelty beyond a better-tuned VGGT-Long stitcher plus local seam smoothing?

3. Demo videos disconnected from the core claims (W2/Q3):

  • Most of the demo videos in the submitted supplementary material are loop-free sequences where the design of HiP3R is never triggered, and these videos remain on the newly updated anonymous website. This makes it difficult to see a clear core story supporting the claimed contributions.

Thank you for your follow-up and additional feedback. We appreciate the opportunity to clarify these points and address each concern below.

1. Evaluation setup

Macro-averaging. Table 1 (Sec. 4.1, p. 6) provides a dataset-level macro-average to summarize performance across diverse domains without allowing datasets with more sequences to dominate the results. We also report per-dataset metrics in Tables 13–15 (Appendix G, pp. 19–20) and separate long-horizon comparisons in Table 2 (Sec. 4.2, p. 7). Under identical VGGT-Omega window settings and loop candidates, HiP3R consistently reduces ATE compared with loop-enabled VGGT-Long on Oxford (1.665 → 1.579 m), KITTI (10.19 → 9.730 m), and VBR (41.27 → 22.44 m).

Full KITTI evaluation. We have now evaluated all 11 KITTI odometry sequences (00–10) under the same matched setting. HiP3R improves all three mean trajectory metrics, reducing ATE, RPE-r, and RPE-t by 16.4%, 12.6%, and 14.9%, respectively, while reducing mean end-to-end reconstruction time by 74.3% (3.88× faster):

Method (VGGT-Omega backend, KITTI 00–10 mean)ATE ↓ (m)RPE-r ↓ (°)RPE-t ↓ (m)End-to-end time ↓ (s)
VGGT-Long (w/ loops)20.5980.19100.25622257.85
HiP3R17.2230.16690.2181581.26

Single-chunk ETH3D sequences. We kept the full ETH3D split so that all baselines are evaluated on the same data. As noted in Appendix E (p. 16), 10 of the 11 ETH3D sequences fit within a single chunk, for which HiP3R returns the input unchanged. These sequences are not used to demonstrate the benefits of hierarchical optimization. Instead, across the 51 multi-chunk sequences in our original benchmark, shared-camera refinement reduces chunk-boundary translation RPE relative to per-frame averaging in 50 cases (Table 19, Appendix L, p. 22).

2. Novelty beyond a tuned VGGT-Long stitcher plus seam smoothing

We agree that Sim(3) chunk graphs and pose graphs are established building blocks. Our contribution lies in the hierarchical formulation and optimization strategy that decouple global scale and layout recovery from shared-camera pose refinement. We clarify the contributions of both stages and provide a direct comparison with standard per-frame PGO.

  • Stage I: Progressive Sim(3) layout recovery. Activating long-span loops all at once on an unaligned chunk chain often traps the non-convex Sim(3) problem in poor minima. We introduce span-ordered progressive loop activation, adaptive continuation around dense revisits, residual-fit weighting, and active-bound restarts. Under the same chunk-graph setting (Table 4, Sec. 4.5, p. 8), these designs reduce ATE from 21.78 to 17.47 m and final solve time from 1,404.5 to 41.3 s. Stage I accounts for most of the global ATE gain (Table 6, Sec. 4.5, p. 9).
  • Stage II: Whole-trajectory shared-camera reconciliation. A per-chunk Sim(3) applies one similarity transform to each chunk, so it cannot resolve conflicting estimates of the same frame from overlapping chunks. Stage II assigns one shared SE(3) variable to each frame for the full sequence, constrained by all its predictions, within-chunk motion factors, and placement priors from Stage I. With the same coarse layout (Table 5, Sec. 4.5, p. 8), motion factors alone raise ATE from 4.91 to 11.68 m, and independent per-frame averaging leaves jumps at chunk boundaries. Combining the two preserves global accuracy and cuts boundary translation RPE by 55.4% on VBR and 56.1% on KITTI (Table 19, Appendix L, p. 22).
  • Direct comparison with standard per-frame PGO (PoseSLAM-style). To directly evaluate the benefit of our hierarchical design, we compare HiP3R against single-stage global per-frame Sim(3) PGO using identical VGGT-Omega predictions and loop pairs on KITTI sequences 00–10. HiP3R lowers ATE from 21.05 to 17.22 m and RPE-r and RPE-t by about 40%, with about 86× faster graph build and optimization (67.25 s vs. 0.785 s). The full table is:
Method (VGGT-Omega backend, KITTI 00–10 mean)ATE ↓ (m)RPE-r ↓ (°)RPE-t ↓ (m)Graph build & optimization ↓ (s)
Single-stage per-frame Sim(3) PGO21.0530.27930.363667.252
HiP3R (hierarchical PGO)17.2230.16690.21810.785
  • Computational efficiency. By resolving scale and global layout on a compact chunk-level graph, HiP3R enables efficient Stage II refinement on a sparse keyframe subgraph, requiring only 0.16 s of solver time on VBR sequences averaging 12,383 frames. In the end-to-end comparison on Oxford, HiP3R takes 160.44 s, compared with 404.15 s for VGGT-Long (Table 12, Appendix F, p. 18).
Reviewer_e6K5OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

HiP3R reconstructs long videos using overlapping-window predictions from a frozen reconstruction model. It first aligns the windows to correct global drift, then fixes that layout and refines one camera pose per frame using overlapping observations and relative-motion constraints. The paper evaluates VGGT-Omega and LingBot-MAP on seven indoor and outdoor benchmarks. With the same VGGT-Omega backend and 60/30 windows, VBR ATE decreases from 41.27 to 22.44 m relative to loop-enabled VGGT-Long.

Critical strengths
  1. The matched-backend improvement on VBR is convincing. Table 2 uses the same VGGT-Omega backend and 60/30 windows. Compared with loop-enabled VGGT-Long, VBR ATE decreases from 41.27 to 22.44 m, a 45.6% improvement. KITTI sequence 00 improves from 10.19 to 9.730 m, a more modest 4.5% gain.
  2. The two stages have experimentally supported roles. In Table 6, the coarse stage reduces KITTI ATE from 77.42 to 9.750 m. The fine stage only changes ATE to 9.730 m, but further reduces rotational and translational RPE by about 24% and 29%. This supports global drift correction followed by local motion refinement.
  3. The efficiency work is useful. Bounded point sampling reduces graph-build-and-solve time by about 83.6% with almost unchanged accuracy in Table 4. Sparse solvers and keyframes also make long-sequence processing more practical.
Critical weaknesses
  1. The contribution beyond existing pose-graph methods needs more precise positioning. Section 3.2 contrasts its weighted IRLS with 'unweighted Umeyama' while citing VGGT-Long. However, VGGT-Long also uses confidence-weighted IRLS with Huber reweighting and global pose-graph optimization [R1]. This contrast should be corrected. Representing a frame with one pose constrained by multiple observations is also standard pose-graph modeling [R2]. The contribution is better framed around the hierarchy, solver scheduling, and joint use of placement and motion constraints, rather than the shared variable itself.
  2. Point-cloud accuracy has not been compared with VGGT-Long. Table 2 compares HiP3R and VGGT-Long on trajectory error, but not point-cloud accuracy. Appendix M, Table 21, compares unaligned windows, HiP3R layout alignment alone, and full HiP3R. For VGGT-Omega, Chamfer changes from 0.027289 m after alignment to 0.027140 m for the full method, an additional gain of about 0.55%. This separates the two stages' effects, but does not establish the point-cloud improvement over VGGT-Long.
  3. Regressions on some scenes need an explanation. Table 13 already reports regressions after adding HiP3R to LingBot-MAP: ATE increases from 0.0789 to 0.0901 m on 7-Scenes, from 0.0554 to 0.126 m on Neural RGB-D, and from 0.518 to 1.026 m on Tanks and Temples. Stage-wise results on these datasets would clarify whether alignment or refinement causes the degradation. Appendix I separately discusses differing coarse layouts between the exploratory window sweeps and main benchmark, not the cause of these regressions. This motivates checking how refinement depends on coarse-layout quality.

[R1] Deng et al. VGGT-Long: Chunk It, Loop It, Align It—Pushing VGGT's Limits on Kilometer-scale Long RGB Sequences. arXiv:2507.16443v1. Sections 3.1 and 3.3.

[R2] Frank Dellaert and Michael Kaess. Factor Graphs for Robot Perception. Foundations and Trends in Robotics, 6(1–2):1–139, 2017. DOI:10.1561/2300000043. Section 6.2.4 (PoseSLAM).

Questions for authors
  1. What are the concrete additions over VGGT-Long, and which registration and loop-closure settings are retained in the matched baseline?
  2. Can the authors compare the final point-cloud accuracy (Chamfer and F1) of HiP3R and loop-enabled VGGT-Long using identical VGGT?
  3. Omega predictions and window settings? This would extend the trajectory-error comparison in Table 2 to point-cloud quality.
  4. Which stage causes the reported regressions, and how does coarse-layout quality affect refinement?
4 comments

We thank Reviewer e6K5 for the positive assessment and constructive feedback. We appreciate the recognition of our improvements under matched-backend comparisons, the complementary roles of coarse alignment and fine refinement, and the efficiency of our framework. We address the concerns point by point below.

W1 / Q1: Contribution beyond VGGT-Long

We thank the reviewer for the suggestion. We have revised Section 3.2 and acknowledge the shared registration and pose-graph components. Our contribution is the hierarchy and its optimization schedule. Progressive loop activation first recovers chunk placement and scale, followed by fixed-layout trajectory refinement using joint placement and motion constraints. Placement anchors preserve global structure, while motion factors reconcile local predictions. Sparse keyframe optimization, closed-form Jacobians, and interpolated corrections enable efficient refinement of sequences with over 10,000 frames. The camera corrections are then applied consistently to the dense pointmaps. We also provide supplementary reconstruction results and illustrations on our demo page to further highlight our contributions: https://hip3r-anonymous.github.io.

W2 / Q2 & Q3: Point-cloud accuracy against VGGT-Long

Yes, the matched-backend comparison in Table 2 uses identical VGGT-Omega predictions and matching window settings for both methods. We have additionally included a point-cloud comparison in Section 4.3 (Line 366-371): Chamfer distance / F1 is 0.05982 m / 84.44% for VGGT-Long and 0.02714 m / 91.29% for HiP3R. These results show that HiP3R also improves dense reconstruction quality.

W3 / Q4: Which stage causes the regression

We added the intermediate results for the three LingBot-MAP regressions to Table 11(b):

DatasetLingBot-MAPStage 1Stage 2
7-Scenes0.07890.09040.0901
Neural RGB-D0.05540.12630.1262
Tanks and Temples0.51821.02561.0265

Most of the gap to the baseline is already present before refinement. Stage 2 slightly improves the first two datasets and slightly worsens Tanks and Temples.

We hope these clarifications and additional experiments address the reviewer's concerns. We would be happy to discuss any remaining questions.

Thank you for the clarifications and additional results. The matched VGGT-Omega point-cloud comparison, using the same predictions and window settings, addresses my questions about point-cloud quality (Chamfer: 0.05982 m vs. 0.02714 m; F1: 84.44% vs. 91.29%). The stage-wise results also clarify that most LingBot-MAP regressions arise in Stage 1, with Stage 2 providing little recovery. The response further explains HiP3R’s hierarchical optimization and scheduling. My remaining concern is that the contribution beyond standard global per-frame PGO is still not isolated, and the matched-baseline registration and loop-closure settings should be stated explicitly. Overall, the added evidence addresses most of my questions, while the methodological distinction remains somewhat unclear.

Thank you for your positive feedback on our responses. We are glad the matched comparison and stage-wise results resolved most of your questions. We address the two remaining points below.

1. Contribution beyond standard global per-frame PGO

The key distinction lies in how scale and camera poses are optimized. Each feed-forward window has its own scale, and a direct per-frame Sim(3) formulation couples global scale correction to thousands of camera variables. HiP3R instead decouples these tasks: Stage I estimates chunk-level scales and global layout on a compact Sim(3) graph, while Stage II fixes the layout and refines a shared-camera SE(3) trajectory.

Direct comparison with per-frame PGO. To directly evaluate the benefit of this hierarchical design, we compare HiP3R against single-stage per-frame Sim(3) PGO using the same VGGT-Omega predictions and loop pairs on KITTI sequences 00–10.

Method (VGGT-Omega backend, KITTI 00–10 mean)ATE ↓ (m)RPE-r ↓ (°)RPE-t ↓ (m)Graph build & optimization ↓ (s)
Single-stage per-frame Sim(3) PGO21.0530.27930.363667.252
HiP3R (hierarchical PGO)17.2230.16690.21810.785

HiP3R reduces ATE by 18.2% and both RPE-r and RPE-t by approximately 40%, while achieving an 86× speedup in graph construction and optimization. Resolving scale and global layout at the chunk level before per-frame refinement yields both higher accuracy and substantially lower computational cost than directly optimizing them over per-frame variables.

Stage II ablation. Table 5 (Sec. 4.5, p. 8) further shows why both terms are necessary for shared-camera refinement once the global layout is fixed. Motion factors without placement priors raise ATE from 4.91 to 11.68 m, while per-frame averaging without motion factors leaves discontinuities at chunk boundaries. The combined formulation preserves global accuracy (4.906 m) and reduces boundary translation RPE by 50.6% overall (55.4% on VBR; Table 19, Appendix L, p. 22).

2. Matched-baseline settings with VGGT-Long

We clarify the experimental settings for the matched comparison between HiP3R and VGGT-Long as follows:

Shared settings: Both methods use identical cached VGGT-Omega predictions (C=60C=60, O=30O=30), 20-frame loop windows, and SALAD loop pairs (similarity 0.85, top-5, 25-frame NMS, one pair per chunk pair), as detailed in Table 9 (Appendix D, p. 16). Loop closure is enabled for both methods.

Method-specific settings: The methods differ in their registration and optimization pipelines. VGGT-Long keeps its original confidence-weighted Huber IRLS registration and Sim(3) loop optimizer. HiP3R uses bounded point sampling, a progressive Sim(3) layout solver, and fixed-layout shared-camera refinement.

We hope these clarifications and additional experiments address your remaining concerns, and we remain happy to discuss any further questions.

Dear Reviewer,

Thank you for your time and thoughtful engagement throughout the discussion.

As we approach the end of the discussion phase, we hope our replies and the additional experiments have addressed the points you raised. If so, we would be sincerely grateful if you could reflect this in your final assessment.

If any question remains, please let us know. We would be glad to clarify right away.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Responses

Dear Reviewers and Area Chair,

We thank all reviewers for their careful reading and constructive feedback. We appreciate the recognition of the effectiveness of our two-stage design and its efficiency on long sequences. We have posted detailed responses to each reviewer and revised the manuscript. Below, we summarize the main updates.

  • Contribution and refinement analysis. We clarified our relationship to prior work and the contribution of our hierarchy and optimization schedule. Additional boundary-level analysis demonstrates how shared-camera refinement improves local trajectory consistency after global layout recovery.

  • Direct reconstruction comparison. We added a point-cloud comparison with VGGT-Long under the same settings, showing improvements in both Chamfer distance and F1. We also expanded the visual comparisons to include VGGT-Long and illustrate the effects of the two stages (https://hip3r-anonymous.github.io/).

  • Failure analysis. We added experiments with incorrect and missing loop closures, along with stage-wise analysis of cases where HiP3R degrades the backend results. These experiments clarify the limitations of our method and suggest directions for future research.

  • Efficiency and presentation. We added dense-versus-keyframe comparisons to examine the accuracy and efficiency trade-offs. End-to-end runtime comparisons show that HiP3R is faster than VGGT-Long and GlueMap under the same settings. We also clarified the experimental descriptions, corrected table formatting issues, and consolidated repeated results.

We hope these additions help clarify the contribution and the experimental evidence. We welcome further questions and would appreciate hearing whether any concerns remain after the reviewers have considered our responses and revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.