acceptodds
Withdrawn from ICLR 2027

Training-Free Looped Vision Geometry Transformers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Can the geometric predictions of pretrained feed-forward models be improved without retraining or fine-tuning? We study training-free looped inference across VGGT, Pi3, Depth Anything 3, MapAnything, and VGGT-. An architecture-aware wrapper repeats a contiguous block window with damped updates, preserving pretrained weights, native attention, and prediction heads. We compare original (raw) and loop-augmented (loop) inference using one fixed configuration per model across static and dynamic benchmarks. Looping raises VGGT's HiRoom reconstruction F1 from 53.79 to 60.43; MapAnything improves every measured reconstruction cell across five benchmarks. Benefits depend on the task and window. On HiRoom, higher raw patch-feature similarity within a window is associated with larger F1 gains for VGGT and Pi3 after accounting for window length. Consistently updating intermediate features also enables useful loops across layers supplying dense prediction heads. These results demonstrate training-free improvements to multi-view geometry and connect their benefits to the structure of pretrained representations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Vision Geometry Transformers, 3DV, Reconstruction, VGGT

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_jxYcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper applies the idea of training-free looping to five frozen geometry models: VGGT, Pi3, DA3, MapAnything and VGGT-Ω. A contiguous block window is replayed K times with a damped Euler update. In terms of experimental evaluations, the authors measure across 10 benchmarks covering pose, reconstruction and depth. The empirical advantages are not stable. For example, VGGT's HiRoom F1 rises from 53.79 to 60.43 while the differences are minor for other backbones and other datasets.

Critical strengths
  • The proposed method is purely training-free. The user can extend it to new base models without adding any parameters.
  • The authors provide careful engineering to handle pair granularity, special tokens and dense-head feature caches.
  • The paper also offers broad evaluations across five backbone networks and 10 benchmarks.
Critical weaknesses
  • The evaluation configurations are selected on the test benchmarks. Each configuration is picked from candidates using aggregates over the same benchmarks reported with a different selection rule per model. No held-out evaluation is given. Moreover, DA3's chosen setting is the only one of 318 that improves all six aggregates and some of its margins are tiny (+0.06% AbsRel), which looks like noise.
  • Gains concentrate on HiRoom but without uncertainty estimates. Excluding HiRoom, VGGT's aggregates fall to +0.35 AUC@3, −0.05 AUC@30 and +0.83 F1. And reconstruction distance gets slightly worse (−0.28%). Most other deltas are very minor (e.g. ≤0.1 pp) on benchmarks with 7–14 scenes.
  • Novelty is limited. The damped sub-step update and its ODE motivation shares the same idea of Chen et al. [1] and Déjà View [2] which already studies looping for multi-view reconstruction and evaluates against the same five baselines. So the contribution is mainly architecture adapters plus an empirical study, which raises the bar on W1 and W2.
  • Controls that would isolate the mechanism are missing. There is no naive-repetition baseline, no zero-cost residual-rescaling baseline (h_0 + c(a−h_0)) and no random-perturbation baseline. Eq. 3 is never compared against simpler cache readouts and K is limited to {2, 3, 4}.
  • Compute cost is not reported. Appendix H gives only evaluation-pipeline throughput, not loop latency. VGGT [5,8]/4 adds roughly 50% backbone FLOPs but there are no latency or memory numbers and no compute-matched comparison. Given the minor experimental gains from this submission, the introduced extra running costs might be problematic to practical usage.
  • The analysis is narrow and partly confounded. It is HiRoom-only and uses overlapping windows. Controlling for window start, Pi3's similarity–gain association drops from 0.675 to 0.320 while DA3 and MapAnything show weak or no association. The abstract's claim about representation structure is therefore overstated.
  • There are no visual comparison figures for camera trajectories. Also the qualitative demonstrations for 3D reconstruction are also limited. Can you show more visual improvements for each chosen base model? Please also report which sequence is used.

[1] Chen et al., Training-Free Looped Transformers, 2026.

[2] Burzio et al., Déjà View: Looping Transformers for Multi-View 3D Reconstruction, 2026.

Questions for authors

Please see the weakness part.

Reviewer_oCd4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies training-free looped inference for five pretrained visual geometry models: VGGT, Pi3, Depth Anything 3, MapAnything, and VGGT-Ω. It reuses contiguous transformer windows with damped updates while preserving pretrained weights, native attention operations, and prediction heads. An architecture-aware wrapper handles differences in attention structure, special tokens, and intermediate features consumed by dense prediction heads. The evaluation covers camera pose, 3D reconstruction, and depth across nine static and dynamic benchmarks, using one selected configuration per model in the main tables. Further analyses examine task-specific configurations, feature similarity, downstream representation changes, and loops containing intermediate feature-cache layers.

Critical strengths
  1. Broad evaluation across architectures and tasks. Testing five backbones across multiple geometric tasks provides useful evidence that inference-time computation reuse can extend beyond a single architecture. The per-benchmark reporting includes regressions and task trade-offs, which helps readers assess where the intervention is useful.

  2. Some meaningful improvements without retraining. VGGT's HiRoom reconstruction F1 increases from 53.79 to 60.43, and MapAnything improves all reported reconstruction F1 and distance entries in Table 4. These results demonstrate practical potential for improving existing checkpoints without collecting training data or updating model parameters, although additional inference computation is required.

Critical weaknesses
  1. Limited methodological novelty relative to the emphasis on the architecture-aware wrapper. The manuscript explicitly attributes the damped looping update and raw-endpoint anchoring in Equations 1-2 to prior work. The main extension is to adapt this procedure to geometric backbones by preserving attention pairs, special tokens, and dense-head feature inputs. These adaptations are useful, but the paper does not sufficiently establish why they constitute a substantial methodological contribution beyond careful engineering. The paired cache updates in Equation 3 are the most concrete additional component, yet the window scan in Section 4.3 does not isolate their contribution through a comparison of alternative cache-update policies for the same windows. The presentation also makes the contribution difficult to assess: Figure 2 illustrates generic looping but does not show the architectural differences or cache-update paths central to the claimed wrapper, while the brief related-work paragraph provides little technical comparison with the closest approaches. Overall, the novelty claim appears stronger than the demonstrated methodological advance.

  2. The breadth of results is not matched by consistently substantial gains or sufficiently general explanatory insight. Although some reconstruction improvements are meaningful, many changes are marginal and some are negative. Table 1 reports average F1 gains of only 0.334 and 0.263 percentage points for Pi3 and DA3, respectively; DA3's average depth-accuracy gain is 0.059 points. Pi3's ETH3D F1 falls from 73.41 to 71.36 in Table 4. Moreover, configurations are selected using aggregate metrics across the reported benchmarks (Section 3.1 and Appendix D), so keeping a configuration fixed within the main tables does not by itself establish generalization beyond the selection suite.

    The analysis offers useful observations, but it remains largely descriptive and does not adequately explain the differences between models. For example, the similarity-gain association is positive for VGGT and Pi3, weaker for DA3, and absent for MapAnything; Pi3's F1 association also drops from 0.675 to 0.320 after the additional window-position adjustment (Appendix G). The paper does not turn these model-dependent observations into a convincing explanation or a validated rule for choosing effective loops. The qualitative evidence is similarly limited: Figures 1-2 show selected improvements, but do not explain representative regressions, and several Figure 1 examples use configurations different from the main tables (Section 3.4). The very brief conclusion largely restates the observations without synthesizing these differences into deeper takeaways. Together, these limitations leave the empirical contribution insufficient to compensate for the limited methodological novelty.

Questions for authors
  1. Contribution of the wrapper. Beyond transferring the existing looping update to these backbones, which component provides the main methodological advance? For fixed VGGT windows containing feature-cache layers, what comparison isolates the benefit of Equation 3 against simpler cache-readout policies?

  2. Configuration selection and robustness. What evidence shows that the selected configurations retain their gains on scenes or benchmarks excluded from configuration selection? For the small improvements in Pi3 and DA3, can scene-level uncertainty estimates establish how consistently the gains occur?

  3. Scope of the proposed insights. What explains the weaker or absent similarity-gain relationship in DA3 and MapAnything, and the reduction in Pi3's association after controlling for window position? Is there evidence that similarity can guide window selection on data outside the HiRoom analysis?

  4. Qualitative support for the main results. Can the authors provide representative comparisons using the shared main-table configurations, including failure cases such as Pi3 on ETH3D? These would help determine which geometric errors looping corrects or introduces and whether the selected visual improvements reflect the configurations used for the quantitative claims.

Reviewer_CYU2OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates whether pretrained vision geometry transformers can be improved through training-free looped inference. The authors applied the damped looping formulation developed by previous work, which repeatedly execute a selected interval of transformer blocks, to five geometric foundation models: VGGT, Pi3, Depth Anything 3, MapAnything, and VGGT-Ω. To accommodate geometry-specific architectures, where dense prediction heads consume not only the final backbone output but also intermediate features, the authors introduce a cache-aware update mechanism that maintains separate damped states for intermediate frame-wise and global features during looping.

This paper evaluates the approach on a variety of geometric benchmarks and achieves improvements on several benchmarks. It additionally analyzes the relationship between intermediate feature similarity, loop-window placement, and downstream performance, and investigates whether looping can be applied to windows containing intermediate feature-cache layers.

Critical strengths
  • The paper evaluates a training-free looping mechanism across five models and diverse benchmarks covering camera pose estimation, 3D reconstruction, and depth estimation in static and dynamic scenes.
  • The approach achieves notable gains on several benchmarks without additional training.
  • The proposed cache-aware mechanism provides a way of handling intermediate features consumed by the prediction head.
Critical weaknesses
  • Limited methodological novelty: The core looping mechanism was introduced by Chen et al. [1]. Although the authors extend it to vision geometry models and introduce modifications to handle intermediate features, I find the overall methodological novelty and contribution limited.
  • Insufficient justification for the intermediate feature treatment: The main geometry-specific methodological contribution is the treatment of intermediate features during looping. Yet, the paper provides neither a strong theoretical justification nor ablation studies to justify their specific design choice. For instance, simply retaining the intermediate features from the last loop iteration looks like a reasonable alternative to me, yet this setting is not evaluated or ablated.
  • Lack of baselines: The experiments compare the proposed method only against the original models. There are no comparisons with alternative inference-time refinement strategies. To this end, I believe the related work section could also be expanded to better position the proposed approach.

Reference: [1] Lizhang Chen, Jonathan Li, Chen Liang, Ni Lao, and Qiang Liu. Training-free looped transformers

Questions for authors

Could the authors clarify the key contribution beyond applying the looping mechanism developed by previous work, as well as justify their design choices?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.