acceptodds
Under review as a conference paper at ICLR 2027

RAGR: Region-Adaptive Group Routing for Multi-View 3D Reconstruction

Abstract

Recent visual geometry foundation models recover camera poses, depth, and dense 3D structure from multiple images, yet a single global prediction model struggles to accurately reconstruct every local region. We investigate whether applying multi-choice, region-level corrections on top of a global prediction can further enhance reconstruction quality. We propose RAGR, a region-adaptive group routing framework for post-prediction refinement. RAGR partitions an initial 3D reconstruction into spatial groups supported by multi-view co-visibility, and employs a lightweight router to decide for each group whether to retain the shared global prediction or apply a compact correction learned from diverse teacher models. In zero-shot evaluations, RAGR improves upon the shared global prediction on ETH3D and outperforms full-frame geometric experts on Tanks and Temples (TNT). Furthermore, while RAGR remains competitive with full-frame experts on ETH3D, its improvement over the global baseline does not transfer stably to TNT. These results demonstrate the feasibility and advantages of localized post-prediction refinement, while identifying transferable action selection as the core requirement for effective spatial routing.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multi-view 3D reconstruction, Visual geometry foundation models, Region-adaptive refinement, Regional action routing, Depth refinement, Conditional computation, Zero-shot domain transfer

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ujf8OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RAGR, a post-prediction refinement framework for multi-view 3D reconstruction. Starting from a frozen FastVGGT prediction, it constructs co-visible spatial groups and routes each group to either retain the original depth or apply one of three lightweight, teacher-derived residual corrections. The correction modules and router are trained exclusively on source-domain data and evaluated zero-shot on ETH3D and Tanks and Temples. RAGR improves over the initial prediction on ETH3D but does not improve AbsRel on TNT. Additional analyses demonstrate regional action complementarity while revealing limitations in cross-domain routing and alignment between local supervision and scene-level evaluation.

Critical strengths

1. Effective teacher-derived corrections. The fixed VGGT-derived correction reduces ETH3D AbsRel from 0.43949 to 0.40507 without requiring teacher execution at inference, demonstrating the practical value of lightweight residual proxies (Table 1).

2. Relevant controls and transparent reporting. Fixed-action baselines, a matched scene-level control, scene-level bootstrap comparisons, and extensive router variants help characterize the method’s benefits and limitations. The paper also explicitly acknowledges cross-domain failures and deployment overhead.

3. Informative analysis of correction and selection. The paper distinguishes transferable correction quality from routing reliability. Local action diagnostics and oracle-selected stitching show that some regional selection opportunities survive the reconstruction pipeline, although the deployed router does not consistently exploit them.

Critical weaknesses

1. Dependence on the initial global predictor remains unclear. All experiments use FastVGGT as the global predictor, leaving open whether the residual corrections primarily compensate for FastVGGT-specific limitations or also benefit stronger initial reconstructions. Evaluating the teacher models independently does not establish applicability across backbones. Please evaluate VGGT and stronger models as initial predictors under matched training and evaluation protocols, reporting both reconstruction gains and computational costs. This would clarify how the method’s effectiveness changes as initialization quality improves.

2. Robustness of co-visible grouping to initial geometry errors is insufficiently evaluated. Group construction depends on predicted depth and camera parameters. Errors in these estimates can propagate through 3D clustering, cross-view associations, window projection, routing features, and residual fusion. Since camera parameters remain fixed, depth-only refinement may not adequately compensate for such errors. Please investigate this sensitivity through controlled depth and pose perturbations, varying initialization quality, and comparisons using alternative geometry for grouping. Reporting grouping stability, action-selection changes, and reconstruction accuracy would help establish robustness.

3. Scale handling in residual transfer requires clarification. The proxies predict additive depth residuals capped at 1.5, but a relative-error loss alone does not make this parameterization scale-equivariant. If teacher and baseline depths have different scales, the proxies may conflate scale compensation with local geometric correction; the fixed residual bound may further introduce scale dependence. The median-depth and depth-variation routing features are also scale-sensitive. Please specify how teacher and baseline scales are reconciled, the scale in which the residual bound is defined, and whether normalization is consistent between training and inference. A geometrically equivalent global-rescaling test should examine whether residuals scale accordingly, routing decisions remain stable, and final scale-aligned accuracy is preserved. Per-window evaluation alignment does not establish scale equivariance of the model itself.

4. The advantage of regional routing over simpler strategies remains unestablished, and overall gains are limited. On ETH3D, RAGR improves over Raw but not significantly over the matched scene-level control; on TNT, its AbsRel is worse than both. Fixed Regime- and VGGT-derived corrections also achieve lower mean AbsRel than RAGR on both benchmarks. These results support the correction modules more clearly than adaptive regional selection. Although target-domain action rankings are unavailable at deployment and the source-selected fixed action is Raw, stable cross-domain benefits from regional routing remain unproven. Please compare against additional simple strategies selected exclusively using source data, across more target domains and random seeds, with paired confidence intervals and accuracy–cost comparisons. Given the mismatch between local supervision and scene-level evaluation, it would also be useful to test whether aligning the routing objective with the final evaluation improves its advantage over these controls.

5. Qualitative evidence is insufficient to explain the benefits and failure modes of local refinement. The current visualization covers only two ETH3D scenes, providing limited evidence across geometric structures, texture conditions, and visibility patterns, with no comparable analysis of TNT failures. Please include representative successes and failures from both benchmarks, showing RGB, ground-truth depth, Raw, fixed corrections, RAGR, error maps, and routing decisions under consistent visualization settings. Multi-view regional comparisons and detailed point-cloud views would further reveal whether corrections improve geometry, introduce boundary artifacts, or fail because of grouping or routing errors. Examples should follow a predefined selection protocol to limit cherry-picking.

Questions for authors

All questions are detailed in the weaknesses above.

Reviewer_4tLCOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

RAGR refines the output of a frozen FastVGGT model by routing co-visible spatial groups to different depth-correction actions. Each group either retains its original depth or receives a residual correction from one of three lightweight proxies trained using RegimeVGGT, VGGT, and VGGT-Omega. A small MLP takes five features as input and selects an action for each group. The proxies and router are trained on 7-Scenes and ARKitScenes, with cross-dataset evaluation on ETH3D and Tanks and Temples (TNT).

Critical strengths
  1. In Table 1, fixed-V reduces AbsRel from 0.43949 to 0.40507 and RMSE from 2.66911 to 2.35661. The reported 95% interval for the AbsRel difference is [−0.05168, −0.01891] (Appendix A.4), supporting the benefit of applying a lightweight residual model to cached reconstruction outputs. The evidence is less conclusive on TNT, where the corresponding interval includes zero, so the transfer claim should remain dataset-specific.
  2. Rather than presenting only favorable results, the authors examine the poor action choices on TNT. Table 12 shows that the router selects Omega for 36.0% of groups, although the local oracle selects it for only 3.6%. Appendix E.5 also evaluates oracle choices through the actual stitching procedure, showing that some of the potential benefit from better selection remains after stitching. These analyses help distinguish action-selection errors from limitations of the available corrections.
Critical weaknesses
  1. RAGR improves AbsRel over Raw on ETH3D, from 0.43949 to 0.42681, but slightly worsens it on TNT, from 0.23708 to 0.23794. The comparison with scene-level selection using the same trained router is also mixed: group-wise routing performs better on ETH3D (0.42681 versus 0.43668) but worse on TNT (0.23794 versus 0.23394). Moreover, fixed-V achieves lower error than RAGR on ETH3D. Since fixed-V was not selected by source validation, it should not be presented as an available target-free deployment choice; nevertheless, its performance suggests that the router does not reliably exploit the corrections available to it. The paper acknowledges these limitations, but the current evidence does not establish a consistent advantage for group-wise selection.
  2. Section 3.4 constructs action labels using per-window scale-aligned AbsRel, whereas the final evaluation averages overlapping predictions and applies a single scale estimate to each stitched scene. This is a consequential difference: Section 4.3 reports only 42.6% label agreement on ETH3D and 42.9% on TNT between local-scale and shared-scene-scale scoring. The analysis identifies the mismatch but does not test whether addressing it during source training improves the deployed router. A control using source labels constructed under a shared scene-scale convention, with the proxies and router architecture held fixed, would be important. Without it, the routing failures are difficult to attribute: they may reflect misaligned supervision, a domain shift in action preferences, or both.
  3. Section 3.3 attributes resistance to domain-specific scale overfitting to teacher-derived targets. However, as Section 4.3 notes, the GT-only control changes both the depth target and the selective-imitation mechanism. It therefore cannot establish which change accounts for the performance difference. Either a matched control is needed, or the claim should be limited to the observed transfer behavior of the combined training procedure.
Questions for authors
  1. Have you trained a router using source-data labels constructed under a shared scene-scale convention rather than per-window scale alignment? With the proxies and router architecture fixed, how does this change the final stitched results in Table 1? The label-agreement analysis establishes that the conventions differ, but not whether correcting the mismatch improves routing.
  2. Was the strong preference for Omega already present on the 11 held-out source sequences? Please report its selection rate and the error relative to Raw on the groups where it was selected. This would help determine whether the failure was detectable during source validation or emerged only on the target domain, without using target data to tune the policy.
  3. Can you compare teacher-depth and GT-depth supervision while keeping the selected training groups, selective-imitation mechanism, and training budget fixed? If such a comparison is unavailable, please narrow the explanation in Section 3.3 so that it does not attribute the observed transfer benefit specifically to teacher-derived targets.
Reviewer_B6zEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces RAGR, a framework that refines the output of a geometry foundation model through region-adaptive group routing. The authors first obtain a shared global reconstruction using a frozen FastVGGT model. The global reconstruction is then partitioned into spatial groups supported by multiple views. For each group, the authors use a policy network to decide whether to keep the initial prediction or apply one of three learned depth-residual corrections from frozen RegimeVGGT, VGGT, and VGGT-Ω teachers. The regional predictions are then combined while keeping the original camera estimates fixed. The authors provide experimental results on the ETH3D and Tanks and Temples datasets.

Critical strengths
  1. The paper is easy to follow. The key idea is straightforward and easy to understand.

  2. The visual examples help explain how regional routing changes the initial predictions.

  3. The authors provide some preliminary experiments and ablations on the ETH3D and Tanks and Temples datasets.

Critical weaknesses
  1. The results do not show a clear benefit from the proposed group-wise routing. In Table 1, uniformly applying the VGGT-derived correction gives lower AbsRel and RMSE than RAGR on both datasets. RAGR also underperforms the RegimeVGGT baseline in some settings. Table 2 also shows that the router performs worse than the best fixed action on both datasets. Although the paper acknowledges these failures, the current results make the effectiveness of RAGR questionable.

  2. The evaluation is limited to ETH3D and Tanks and Temples. Current results cannot lead to meaningful conclusions on the overall effectiveness of RAGR.

  3. The qualitative results raise concerns about regional boundaries. In Figure 1, some corrected regions show visible block-like changes in the predicted depth. While the authors clarify in Section 3.5 that overlap averaging reduces boundary artifacts, the degraded qualitative results are concerning.

Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.