acceptodds
Under review as a conference paper at ICLR 2027

Leveraging Geometric Evidence across Views for Scene Reconstruction

Abstract

Cross-view correspondence provides geometric constraints for multi-view reconstruction by linking observations of the same physical points across views. Recent feed-forward models learn these associations through multi-view feature interactions, often using frame-wise and global attention. However, when matching supervision is confined to a dedicated prediction head, it shapes internal representations only indirectly. Cross-view interactions may therefore remain weakly constrained, limiting how effectively matching information supports reconstruction. We introduce LEVER, which injects cross-view geometric evidence at three levels: output supervision, intermediate feature interactions, and global scene representation. A dense Match Head learns pixel-level cross-view displacement fields, making fine-grained alignment an explicit learning objective. Within the backbone, selected cross-view attention responses are constrained by patch correspondences to encourage information exchange between geometrically related regions. Sequence-level scene tokens aggregate cross-view information under a correspondence-consistency constraint and provide global context to the prediction heads. Together, these components make correspondence a structural signal for reconstruction rather than an auxiliary output. Across five benchmarks, LEVER improves camera-pose estimation, depth estimation, and point-cloud reconstruction over its underlying baseline. Component ablations show that all three levels contribute to reconstruction quality.

Primary area:
foundation or frontier models, including LLMs
Keywords:
3D Foundation Model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2x3iOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

Summary: This paper aims to improve visual correspondence learning in feed-forward 3D reconstruction foundation models. The authors argue that supervising correspondence solely through auxiliary output heads provides only indirect constraints to the backbone. To address this, they propose LEVER, which explicitly injects geometric evidence across three complementary levels: (1) a training-only dense matching head for displacement regression, (2) attention supervision with a 3D distance penalty within intermediate transformer layers, and (3) sequence-level scene tokens optimized via cross-view assignment agreement.

Key contributions are (1) A three-level geometric supervision framework spanning dense prediction, internal attention logits, and global scene tokens. (2) Carefully designed training objectives and strategies. Validation: The authors evaluate LEVER across five benchmarks (CO3D, TartanAir, 7-Scenes, DTU, and TUM-Dynamic) on camera pose, depth, and point-cloud metrics, showing consistent gains over baselines like VGGT and Flow3R. Component ablations are provided on a reduced training setup.

Critical strengths
  1. Enforcing cross-view pixel correspondence directly at the intermediate attention and frame/sequence-level token stages is an interesting exploration. This paradigm could provide meaningful takeaways and inspire future work in geometric foundation models.
  2. The proposed three-level geometric supervision framework is not strictly tied to a specific network architecture, offering potential compatibility with other transformer-based geometry foundation models.
  3. Based on the presented benchmark results, the method yields noticeable and consistent performance gains across multiple evaluation tasks (pose estimation, depth, and point-cloud reconstruction) compared to existing baselines.
Critical weaknesses
  1. Level 2: Forcibly steering cross-view attention weights toward geometrically corresponding patches is not always beneficial. In textureless or uninformative regions, self- and cross-attention mechanisms inherently need to attend to distant, context-rich patches (long-range global context) rather than purely local counterpart patches to resolve visual ambiguity. Imposing a sharp correspondence constraint may paradoxically impair the model's ability to aggregate non-local contextual cues. Level 3: The underlying intuition appears to be using global tokens to encode object-level consistency. However, in scenes with repetitive textures or multiple visually similar objects, pulling corresponding patches into the same slot without explicit discriminative constraints may introduce severe assignment ambiguity. These ambiguous scenarios are very common in real-world 3D reconstruction, yet an in-depth discussion and theoretical analysis of such failure modes are notably missing.

  2. The proposed three-level supervision is conceptualized as a general, model-agnostic framework for geometric foundation models. However, the empirical validation is strictly confined to a VGGT-style architecture. To substantiate the claimed generalizability and demonstrate that LEVER is truly a plug-and-play paradigm rather than an architecture-specific engineering trick, the authors should validate its effectiveness across other representative multi-view architectures (e.g., DUSt3R, π3\pi^3 ).

  3. While the paper compares against several recent methods, it noticeably omits MASt3R, which is arguably the most essential and directly relevant baseline. MASt3R was explicitly developed to tackle pixel-wise dense correspondence and local descriptor learning for 3D reconstruction. More importantly, given that Level 1 (dense matching head) is already conceptually aligned with MASt3R's matching philosophy, a much more compelling validation would be to apply the proposed Level 2 and Level 3 mechanisms onto MASt3R's open-source codebase/backbone.

Please note that the weaknesses mentioned above reflect my conceptual and structural concerns with the current manuscript, rather than a mandatory request for new large-scale experiments during the rebuttal, which is infeasible within the limited timeframe.

Questions for authors
  1. How effective is explicit pixel correspondence supervision in ill-posed visual regions, such as textureless surfaces or areas with repetitive textures and multiple identical objects? The authors are encouraged to either discuss the underlying mechanism or provide qualitative attention visualizations (similar to Figure 5 in the paper) on such challenging cases to substantiate the behavior of Level 2 and Level 3.

  2. According to the ablation results in Table 4, introducing explicit correspondence supervision brings the most prominent gain to camera pose estimation, whereas the relative gains on depth and point-cloud metrics appear comparatively moderate. Have the authors investigated how do the three individual levels contribute specifically to pose versus depth/dense geometry prediction?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 2x3i: Mechanism Clarification and Additional Experimental Evidence

We thank the reviewer for the constructive feedback. We address the concerns with additional analyses.

A1. Ill-posed regions

We agree that, if Level 2 were understood as imposing hard correspondence constraints at every attention layer and head, it could impair global-context modeling; Level 3 may also face assignment ambiguity with repetitive textures or similar objects.

Our motivation is that learning reliable cross-view correspondences can itself reduce ambiguity in weakly textured regions, where local appearance is insufficient for matching. We use correspondence supervision to learn associations between observations of the same 3D points across views, encouraging the model to exploit global cross-view context rather than relying solely on local texture.

Level 2. Correspondence learning and global-context modeling are complementary. We apply soft supervision only in one global-attention block (layer 24), selected through layer-position ablations. This encourages attention toward geometric correspondences without forcing exclusive attention, while other layers retain global information exchange. Layer-position and component ablations support this selective design.

Level 3. Scene tokens encode shared scene-level representations rather than object-specific identities. The objective encourages corresponding patches to have compatible soft assignments without requiring non-corresponding patches to occupy distinct slots. Thus, assignment ambiguity may remain but does not necessarily imply reconstruction errors; its impact should be assessed through downstream reconstruction.

We compare the reduced ablation reference (Full) with a variant without both losses. Figure in the updated supplementary material shows stronger attention around geometric correspondences in three challenging scenes.

Table R4. Reconstruction on 20 selected samples with weak or repetitive texture.

ModelPose AUC@30 ↑Depth AbsRel ↓Point-cloud [email protected] ↑
Full0.5780.0790.089
w/o both losses0.4660.0930.070

Full improves all three metrics on these challenging samples.

A2. How do the three levels affect reconstruction metrics?

We analyze pose errors and cross-view depth-scale consistency on the same five benchmarks using reduced-model variants. Failure rate is the fraction of image pairs with max(rotation, translation-direction error) >30°. Shared-sequence AbsRel uses one scale per sequence; log-scale MAD measures cross-view scale variation, complementing standard per-view median alignment.

VariantMedian rotation error (°) ↓Median translation-direction error (°) ↓Failure rate (>30°) ↓Per-view AbsRel ↓Shared-sequence AbsRel ↓Log-scale MAD ↓
Ablation reference4.65.80.1180.0720.0810.042
w/o Dense matching7.710.60.2120.0870.1120.089
w/o Attention supervision5.78.00.1460.0790.0950.067
w/o Scene representation5.57.10.1350.0800.1090.081

Level 1. Explicit correspondence learning helps the model capture reliable cross-view geometric relationships, supporting better reconstruction and pose estimation. Removing dense matching increases median rotation/translation-direction errors from 4.6°/5.8° to 7.7°/10.6° and the failure rate from 11.8% to 21.2%, highlighting its contribution to pose accuracy and robustness.

Level 2. Attention supervision encourages geometry-aware cross-view interactions for pose and dense reconstruction. Without it, translation-direction error rises from 5.8° to 8.0°, and shared-sequence AbsRel/log-scale MAD from 0.081/0.042 to 0.095/0.067.

Level 3. Scene tokens aggregate information across views to promote globally consistent scene representations. Their contribution is particularly evident in depth-scale consistency: removing the scene representation increases shared-sequence AbsRel from 0.081 to 0.109 and log-scale MAD from 0.042 to 0.081, despite a relatively small change in per-view AbsRel (0.072 to 0.080). This suggests that scene tokens improve global geometric consistency.

A3. Architecture generality

We progressively add three supervision levels to VGGT under matched initialization, training data, and evaluation protocols, and transfer LEVER to MASt3R.

Backbone / variantPose AUC@30 ↑Depth AbsRel ↓Point-cloud [email protected] ↑
VGGT0.8230.0540.355
VGGT + Matching0.8370.0440.359
VGGT + Matching + Attention0.8540.0390.360
VGGT + Matching + Attention + Scene0.8660.0350.372
MASt3R0.7620.1090.220
MASt3R + LEVER0.8340.0840.318

Each level improves VGGT; LEVER also improves MASt3R across all three metrics, supporting transferability.

I appreciate the additional experiments and results the authors presented in their rebuttal. The new results address most of my concerns and make the paper more complete. Taken together, the experiments, especially those added in the rebuttal, make a coherent case for the paper’s claims. My main remaining concern is whether the contribution is sufficiently novel for ICLR.

AuthorsOpenReview ↗

Clarification of Methodological Novelty

Thank you for the thoughtful follow-up and for recognizing that our additional experiments have addressed most of your earlier concerns and established a coherent case for the paper's claims. We appreciate the opportunity to clarify the remaining novelty concern.

1. Central methodological novelty

LEVER's central novelty is to make geometric correspondence a structural supervision signal for feed-forward 3D reconstruction, beyond auxiliary matching prediction. Using reliability-filtered correspondences derived from existing depth and camera annotations without an external flow teacher, LEVER jointly supervises three complementary levels:

  • Pixel-level associations through dense displacement prediction.
  • Intermediate cross-view interactions through correspondence supervision and 3D spatial-distance constraints.
  • Global scene representations through correspondence-consistent assignments to sequence-shared scene tokens.

Thus, the same geometric evidence guides what correspondences are learned, how information is exchanged across views, and how global scene consistency is represented.

2. Fundamental distinctions from prior work

This goes beyond simply combining existing components.

  • Unlike MASt3R/Flow3R, which primarily supervise correspondence prediction, LEVER establishes geometric correspondence as a fundamental supervisory signal that directly shapes the backbone's internal geometric learning mechanisms.
  • Unlike CAMEO's attention alignment for multi-view generation, LEVER introduces 3D geometry-aware attention supervision that combines correspondence targets with 3D spatial-distance constraints.
  • Unlike VGGT-Ω's frame-specific registers and NOVA3R's non-pixel-aligned global representation, LEVER introduces the first correspondence-supervised, sequence-shared scene tokens that explicitly enforce cross-view assignment consistency while preserving frame-level geometry prediction.

We hope this clarification addresses your remaining novelty concern and would appreciate your consideration of an updated assessment.

AuthorsOpenReview ↗

Looking Forward to Continued Discussion

Dear Reviewer 2x3i,

We greatly appreciate your constructive comments and the time you have devoted to evaluating our work. We hope our additional experiments and recent clarifications have helped address your concerns.

We would be glad to discuss any remaining questions and sincerely welcome further feedback during the remaining discussion period. We highly value your expertise and the opportunity to further improve our work.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_y8D1OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LEVER, a VGGT-style feed-forward 3D reconstruction model (frozen DINOv3 encoder, alternating frame-wise and global attention). Besides the standard camera pose and depth supervision, it adds explicit cross-view correspondence supervision during training. The correspondence targets come from the ground-truth depth and camera poses (for static datasets), and from native optical-flow and stereo-disparity labels for some flow/stereo datasets (used only for the Match Head). These targets are used at three levels: (1) a training-only Match Head that predicts dense displacement between views; (2) a loss on one global-attention layer that pushes each query patch to attend to its matching patch in the other view; and (3) learnable scene tokens, where corresponding patches are encouraged to have similar slot assignments. LEVER achieves the best results on most pose, depth and multi-view 3d reconstruction metrics across CO3D, TartanAir, 7-Scenes, DTU and TUM-Dynamic. Ablations with a smaller model show that each of the three parts helps.

Critical strengths
  1. Simple approach with clear design for correspondence supervision The targets are built from ground-truth depth and poses with well-designed filters, without an external teacher or extra unlabeled data. The Match Head and attention loss are used only during training, so the inference pipeline stays the same as a standard VGGT-style model.
  2. Strong results across five benchmarks. LEVER is best on most pose, depth and 3d reconstruction metrics (Tab. 1–3), including three out-of-domain benchmarks, and the reduced-scale ablation shows that each of the three parts contributes (Tab. 4).
Critical weaknesses
  1. It is unclear whether the gains come from the method or from better engineering. The main results (Tab. 1–3) compare the full model with public checkpoints that differ in almost everything: encoder (frozen DINOv3 vs. DINOv2 in VGGT), training data, and training protocol. The Match Head is also trained on native optical-flow and stereo datasets (TartanAir, KITTI, Sintel, Spring, and about 15 disparity sets), a training corpus where the baseline models do not use. Meanwhile, all ablations (Tab. 4) use a different, smaller model on a reduced data mixture. No published baseline is trained in the same setting, and there is no full-scale model without the three components. So the claim of improving "over its underlying baseline" (inabstract) is not supported, and the reduced "ablation reference" cannot be compared with any number in Tables 1–3. Especially, some ablation numbers are also hard to interpret: removing the training-only Match Head drops pose AUC@30 from 0.644 to 0.378, which is very large for an auxiliary loss (we already know VGGT, DepthAnything3, or Pi3 can work well without explicit correspondence supervision) and suggests the small model is in a different regime from the main one. As a result, I cannot tell whether the state-of-the-art results come from the proposed correspondence supervision or from better training, data and optimization.

  2. The novelty is incremental. Each of the three parts has been used before:

    • Dense matching head with GT correspondences: used in Flow3R, and MASt3R.
    • Correspondence-supervised attention: "Correspondence-Attention Alignment for Multi-View Diffusion Models", which supervises one attention layer with geometric correspondence.
    • Scene tokens: VGGT-Ω, and NOVA3R.
  3. The evaluation is partly in-domain, limited in scope, and missing cost numbers.

    • Two of the five benchmarks (CO3D and TartanAir) are in the training data. The three out-of-domain sets are small and fairly simple: one 10-frame clip per sequence for 7-Scenes and TUM-Dynamic, and 22 object scans for DTU. Harder out-of-domain benchmarks that are not in the training mixture, such as DyCheck, Monkaa and NRGBD (or ETH3D, Bonn, RealEstate10K), would better show whether the gains generalize.
    • The paper does not report runtime, memory or model size compared with the baselines, and all benchmarks use about 10 input frames. Fig. 1 claims LEVER "avoids inference collapse" on an extreme long sequence, but no quantitative result supports this.
Questions for authors
  1. Please show where the gains come from in a controlled setting. For example, fine-tune a public model (e.g., VGGT or Pi3) with and without the LEVER losses on the same data.
  2. For Table 4, why does removing the training-only Match Head cost 27 AUC points? Can you report the reduced models on the same benchmarks as Tables 1–3?
  3. Please report pose and depth on held-out benchmarks not in the training mixture, e.g., DyCheck, Monkaa, NRGBD.
  4. Please report runtime, peak memory and parameter count for LEVER and the baselines, and results as the number of input frames grows (e.g., 10 / 50 / 100+), to support the long-sequence claim in Fig. 1.

Overall: The paper proposes a well-designed feed-forward 3D reconstruction method that achieves state-of-the-art results, although I am not yet fully convinced where these gains come from. However, the novelty is limited, and many experiments are missing to verify the contribution of each module and the practical usage (runtime, memory, long sequences). Thus, I lean toward rejecting this paper.

4 comments
AuthorsOpenReview ↗

Response to Reviewer y8D1: Controlled Validation and Methodological Distinctions

We thank the reviewer for the constructive feedback. We address the concerns with additional experiments and analyses below.

Table R1. Controlled comparisons and generalization to held-out benchmarks.

ModelMain AUC@30 ↑Main AbsRel ↓Main [email protected] ↑DyCheck AUC@30 ↑DyCheck AbsRel ↓DyCheck [email protected] ↑NRGBD AUC@30 ↑NRGBD AbsRel ↓NRGBD [email protected] ↑
G0: Original VGGT0.8170.0540.3560.7740.0640.5500.9710.0250.549
G1: VGGT (fine-tuned)0.8230.0540.3550.7690.0620.5520.9770.0220.548
G2: VGGT + Matching0.8370.0440.3590.7910.0580.5900.9810.0190.550
G3: VGGT + Matching + Attention0.8540.0390.3600.7900.0570.6050.9820.0190.553
G4: VGGT + LEVER0.8660.0350.3720.8110.0520.6110.9820.0170.553

Table R2. Model size and inference efficiency with 10, 50, and 100 input frames.

ModelTotal params (M)Inference params (M)Runtime 10f (s) ↓Runtime 50f (s) ↓Runtime 100f (s) ↓Memory 10f (GiB) ↓Memory 50f (GiB) ↓Memory 100f (GiB) ↓
G1: VGGT baseline architecture1157.91157.91.49113.79626.0996.9849.93914.438
G2: VGGT + Matching1174.51157.9------
G3: VGGT + Matching + Attention1174.51157.9------
G4: VGGT + LEVER1179.41161.31.49814.02226.8067.14911.02716.955

G0 is the original VGGT checkpoint. G1–G4 share the same initialization, training data, and optimization protocol, progressively adding matching, attention, and scene representation to the original VGGT objectives. Main results are averaged over the five benchmarks in the main paper.

Table R2 reports inference costs on a single NVIDIA H20 (518×518, batch size 1, 10–100 frames). G2/G3 have costs comparable to G1 and are omitted for brevity.

A1. Controlled comparisons

We conduct controlled experiments on full-size VGGT with matched encoder, initialization, data, and training protocol (Table R1). Fine-tuning alone yields minimal gains (G0→G1), while progressively adding LEVER's three supervision levels consistently improves reconstruction (G1→G4). These results demonstrate that the gains are not simply due to encoder replacement or fine-tuning.

A2. Reduced-scale ablation

Table 4 uses the same five benchmarks as Tables 1–3, but a smaller model and approximately 20% of the training mixture. This reduced setting may limit generalization and increase reliance on matching supervision, potentially contributing to the 26.6-point AUC@30 drop (0.644→0.378). In contrast, our full-size controlled experiments (Table R1) show a more moderate gain from Match Head supervision (0.823→0.837, G1→G2).

A3. Additional Benchmarks

We additionally evaluate on DyCheck and NRGBD, neither of which is included in our training mixture (Table R1). The results show consistent improvements across both benchmarks.

A4. Computational Cost

As shown in Table R2, LEVER adds only 0.3% inference parameters, with minor runtime overhead at 10/50/100 frames. Peak memory increases by 2.4%/10.9%/17.4%, respectively. G2/G3 have inference costs comparable to G1.

Clarification of Figure 1. By "avoids inference collapse," we refer to avoiding severe reconstruction-quality degradation, not OOM or computational scalability. We have revised the caption to emphasize more reliable correspondences and coherent reconstructions in a challenging outdoor scene.

A5. Methodological Novelty

While Flow3R and MASt3R leverage correspondence learning for geometric reconstruction, LEVER adopts a unified supervision strategy that explicitly guides both local correspondence learning and the internal representations responsible for global scene understanding.

Compared with CAMEO, which focuses on attention alignment for multi-view diffusion, LEVER develops geometry-aware attention supervision tailored to feed-forward 3D reconstruction. Unlike correspondence classification alone, our attention supervision jointly exploits geometric correspondences and continuous 3D spatial relationships, explicitly guiding how multi-view geometric information is integrated within the reconstruction backbone.

Compared with VGGT-Ω and NOVA3R, similarly named scene tokens serve substantially different architectural roles. VGGT-Ω uses frame-specific registers for cross-view information exchange, while NOVA3R employs a global scene latent for generative, non-pixel-aligned reconstruction. LEVER instead preserves frame-based geometry prediction while introducing a separate, sequence-shared scene representation that explicitly aggregates geometrically consistent information across views, rather than replacing frame-level features or merely routing cross-view communication.

I appreciate the authors' efforts in conducting the additional experiments and providing clarifications during the rebuttal. These new results address most of my initial concerns about model evaluation, control experiments, and runtime cost. That said, I agree with Reviewer 2x3i about the technical novelty of the paper for ICLR. The motivation is clear, and the mechanism is reasonable, leading to moderate improvement. Personally, this paper appears to be more of a well-engineered improvement on top of feed-forward models like VGGT, rather than offering an interesting lesson or a new contribution. Thus, I still give this paper a borderline rating, with a slight leaning toward acceptance.

AuthorsOpenReview ↗

Methodological Novelty

Thank you for the thoughtful follow-up. We sincerely appreciate your recognition that our additional experiments have addressed most of your initial concerns regarding evaluation, controlled comparisons, and computational cost. We are particularly encouraged by your stated inclination toward acceptance.

Regarding novelty, the broader contribution of LEVER is not merely to improve a particular reconstruction architecture, but to reveal the importance of where and how reliability-filtered geometric correspondences are used as supervision within geometry foundation models. Our findings suggest that extending such supervision beyond auxiliary prediction provides complementary benefits to geometric learning at different representational levels. The successful transfer to MASt3R further supports the broader relevance of this insight beyond VGGT-specific engineering.

We hope this clarifies the broader methodological contribution of LEVER and would greatly appreciate an updated rating reflecting your current assessment.

AuthorsOpenReview ↗

Looking Forward to Continued Discussion

Dear Reviewer y8D1,

We greatly appreciate your constructive comments and the time you have devoted to evaluating our work. We hope our additional experiments and recent clarifications have helped address your concerns.

We would be glad to discuss any remaining questions and sincerely welcome further feedback during the remaining discussion period. We highly value your expertise and the opportunity to further improve our work.

Thank you again for your time and consideration.

Best regards, The Authors

Reviewer_c86kOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes LEVER, which uses geometric correspondences to supervise dense matching, cross-view attention, and shared scene tokens in a feed-forward reconstruction model. It evaluates camera pose, depth, and point-cloud reconstruction on five benchmarks.

Critical strengths

The three supervision levels form a coherent design. Results are strong across several datasets and reconstruction tasks. The ablations separately examine matching supervision, attention supervision, and scene tokens.

Critical weaknesses

The main comparisons use existing checkpoints, while LEVER uses its own encoder and training configuration. A baseline trained with the same encoder, data, and compute is needed to isolate the proposed method’s contribution.

The ablations use a smaller model and reduced training mixture. These results are useful, but do not establish whether the same component gains hold for the full model. In particular, the additional benefit of attention and scene supervision over matching supervision alone is not directly reported at full scale.

Questions for authors

Could the authors provide a baseline using the same encoder and training configuration without the proposed correspondence components?

Could the authors compare the full model against a variant using matching supervision alone?

1 comment
AuthorsOpenReview ↗

Response to Reviewer c86k: Controlled Comparisons and Full-Scale Ablations

A1. Controlled comparisons and full-model gains

We thank the reviewer for the constructive feedback. To isolate LEVER's contribution, we train full-size VGGT variants from the same initialization, using matched encoder, training data, fine-tuning protocol, and evaluation settings. G0 is the original VGGT checkpoint; G1 retains VGGT's original reconstruction supervision, G2 additionally introduces Match Head supervision, and G3 incorporates all three LEVER levels. To further examine the effectiveness of our supervision strategy beyond VGGT, we also evaluate LEVER on MASt3R.

Backbone / variantPose AUC@30 ↑Depth AbsRel ↓Point-cloud [email protected] ↑
G0: Original VGGT0.8170.0540.356
G1: VGGT (fine-tuned)0.8230.0540.355
G2: VGGT + Matching0.8370.0440.359
G3: VGGT + LEVER0.8660.0350.372
G4: MASt3R0.7620.1090.220
G5: MASt3R + LEVER0.8340.0840.318

Controlled baseline. Compared with G1, full LEVER (G3) improves pose AUC@30 from 0.823 to 0.866, reduces depth AbsRel from 0.054 to 0.035, and improves point-cloud F-score from 0.355 to 0.372. The similar performance of G0 and G1 suggests that these gains are not simply due to fine-tuning. LEVER also improves all three metrics over the original MASt3R, providing additional evidence of its effectiveness beyond VGGT.

Additional gains from Levels 2–3. Compared with G2, G3 further improves pose AUC@30 by 0.029, reduces depth AbsRel by 20.5%, and increases point-cloud F-score by 0.013. These consistent gains confirm that Levels 2–3 provide additional reconstruction benefits beyond matching supervision at full model scale.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Consolidated Rebuttal Summary and Request for Further Discussion

Dear Area Chair,

We thank you and the reviewers for the constructive feedback. We have provided detailed responses with substantial additional experiments to address the main concerns. We briefly summarize the key evidence below.

1. Controlled full-scale validation. We conducted controlled experiments on full-size VGGT with matched initialization, encoder, training data, and optimization protocol. Compared with the fine-tuned baseline, LEVER improves Pose AUC@30 from 0.823 to 0.866, reduces Depth AbsRel from 0.054 to 0.035, and increases Point-cloud [email protected] from 0.355 to 0.372. Importantly, Levels 2–3 provide additional gains beyond matching supervision alone. We also evaluated LEVER on MASt3R, achieving improvements across all three metrics and supporting its transferability beyond VGGT.

2. Cross-dataset generalization and efficiency. We additionally evaluated LEVER on two unseen benchmarks (DyCheck and NRGBD) excluded from our training mixture, observing consistent improvements. We also reported parameter counts, runtime, and memory consumption for 10/50/100-frame inputs, demonstrating limited inference overhead.

3. Mechanism and challenging scenarios. We clarified how soft attention supervision preserves global-context modeling and how scene tokens promote cross-view geometric consistency. Additional attention visualizations, experiments on 20 challenging samples, and detailed pose/depth-scale analyses further support the contributions of the three supervision levels.

4. Methodological distinctions. We clarified how LEVER differs from prior correspondence learning, attention alignment, and scene-token approaches, emphasizing its unified geometric supervision across dense prediction, intermediate attention, and global scene representations.

We hope these additional results address the reviewers' main concerns and provide stronger evidence for LEVER's effectiveness.

We would greatly appreciate your help in encouraging the reviewers to revisit our responses and share any remaining concerns. We welcome further discussion and remain willing to provide any additional clarification.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.