- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Summary: This paper aims to improve visual correspondence learning in feed-forward 3D reconstruction foundation models. The authors argue that supervising correspondence solely through auxiliary output heads provides only indirect constraints to the backbone. To address this, they propose LEVER, which explicitly injects geometric evidence across three complementary levels: (1) a training-only dense matching head for displacement regression, (2) attention supervision with a 3D distance penalty within intermediate transformer layers, and (3) sequence-level scene tokens optimized via cross-view assignment agreement.
Key contributions are (1) A three-level geometric supervision framework spanning dense prediction, internal attention logits, and global scene tokens. (2) Carefully designed training objectives and strategies. Validation: The authors evaluate LEVER across five benchmarks (CO3D, TartanAir, 7-Scenes, DTU, and TUM-Dynamic) on camera pose, depth, and point-cloud metrics, showing consistent gains over baselines like VGGT and Flow3R. Component ablations are provided on a reduced training setup.
Critical strengths
- Enforcing cross-view pixel correspondence directly at the intermediate attention and frame/sequence-level token stages is an interesting exploration. This paradigm could provide meaningful takeaways and inspire future work in geometric foundation models.
- The proposed three-level geometric supervision framework is not strictly tied to a specific network architecture, offering potential compatibility with other transformer-based geometry foundation models.
- Based on the presented benchmark results, the method yields noticeable and consistent performance gains across multiple evaluation tasks (pose estimation, depth, and point-cloud reconstruction) compared to existing baselines.
Critical weaknesses
-
Level 2: Forcibly steering cross-view attention weights toward geometrically corresponding patches is not always beneficial. In textureless or uninformative regions, self- and cross-attention mechanisms inherently need to attend to distant, context-rich patches (long-range global context) rather than purely local counterpart patches to resolve visual ambiguity. Imposing a sharp correspondence constraint may paradoxically impair the model's ability to aggregate non-local contextual cues. Level 3: The underlying intuition appears to be using global tokens to encode object-level consistency. However, in scenes with repetitive textures or multiple visually similar objects, pulling corresponding patches into the same slot without explicit discriminative constraints may introduce severe assignment ambiguity. These ambiguous scenarios are very common in real-world 3D reconstruction, yet an in-depth discussion and theoretical analysis of such failure modes are notably missing.
-
The proposed three-level supervision is conceptualized as a general, model-agnostic framework for geometric foundation models. However, the empirical validation is strictly confined to a VGGT-style architecture. To substantiate the claimed generalizability and demonstrate that LEVER is truly a plug-and-play paradigm rather than an architecture-specific engineering trick, the authors should validate its effectiveness across other representative multi-view architectures (e.g., DUSt3R, ).
-
While the paper compares against several recent methods, it noticeably omits MASt3R, which is arguably the most essential and directly relevant baseline. MASt3R was explicitly developed to tackle pixel-wise dense correspondence and local descriptor learning for 3D reconstruction. More importantly, given that Level 1 (dense matching head) is already conceptually aligned with MASt3R's matching philosophy, a much more compelling validation would be to apply the proposed Level 2 and Level 3 mechanisms onto MASt3R's open-source codebase/backbone.
Please note that the weaknesses mentioned above reflect my conceptual and structural concerns with the current manuscript, rather than a mandatory request for new large-scale experiments during the rebuttal, which is infeasible within the limited timeframe.
Questions for authors
-
How effective is explicit pixel correspondence supervision in ill-posed visual regions, such as textureless surfaces or areas with repetitive textures and multiple identical objects? The authors are encouraged to either discuss the underlying mechanism or provide qualitative attention visualizations (similar to Figure 5 in the paper) on such challenging cases to substantiate the behavior of Level 2 and Level 3.
-
According to the ablation results in Table 4, introducing explicit correspondence supervision brings the most prominent gain to camera pose estimation, whereas the relative gains on depth and point-cloud metrics appear comparatively moderate. Have the authors investigated how do the three individual levels contribute specifically to pose versus depth/dense geometry prediction?
4 comments
I appreciate the additional experiments and results the authors presented in their rebuttal. The new results address most of my concerns and make the paper more complete. Taken together, the experiments, especially those added in the rebuttal, make a coherent case for the paper’s claims. My main remaining concern is whether the contribution is sufficiently novel for ICLR.
Clarification of Methodological Novelty
Thank you for the thoughtful follow-up and for recognizing that our additional experiments have addressed most of your earlier concerns and established a coherent case for the paper's claims. We appreciate the opportunity to clarify the remaining novelty concern.
1. Central methodological novelty
LEVER's central novelty is to make geometric correspondence a structural supervision signal for feed-forward 3D reconstruction, beyond auxiliary matching prediction. Using reliability-filtered correspondences derived from existing depth and camera annotations without an external flow teacher, LEVER jointly supervises three complementary levels:
- Pixel-level associations through dense displacement prediction.
- Intermediate cross-view interactions through correspondence supervision and 3D spatial-distance constraints.
- Global scene representations through correspondence-consistent assignments to sequence-shared scene tokens.
Thus, the same geometric evidence guides what correspondences are learned, how information is exchanged across views, and how global scene consistency is represented.
2. Fundamental distinctions from prior work
This goes beyond simply combining existing components.
- Unlike MASt3R/Flow3R, which primarily supervise correspondence prediction, LEVER establishes geometric correspondence as a fundamental supervisory signal that directly shapes the backbone's internal geometric learning mechanisms.
- Unlike CAMEO's attention alignment for multi-view generation, LEVER introduces 3D geometry-aware attention supervision that combines correspondence targets with 3D spatial-distance constraints.
- Unlike VGGT-Ω's frame-specific registers and NOVA3R's non-pixel-aligned global representation, LEVER introduces the first correspondence-supervised, sequence-shared scene tokens that explicitly enforce cross-view assignment consistency while preserving frame-level geometry prediction.
We hope this clarification addresses your remaining novelty concern and would appreciate your consideration of an updated assessment.
Looking Forward to Continued Discussion
Dear Reviewer 2x3i,
We greatly appreciate your constructive comments and the time you have devoted to evaluating our work. We hope our additional experiments and recent clarifications have helped address your concerns.
We would be glad to discuss any remaining questions and sincerely welcome further feedback during the remaining discussion period. We highly value your expertise and the opportunity to further improve our work.
Thank you again for your time and consideration.
Best regards, The Authors
Response to Reviewer 2x3i: Mechanism Clarification and Additional Experimental Evidence
We thank the reviewer for the constructive feedback. We address the concerns with additional analyses.
A1. Ill-posed regions
We agree that, if Level 2 were understood as imposing hard correspondence constraints at every attention layer and head, it could impair global-context modeling; Level 3 may also face assignment ambiguity with repetitive textures or similar objects.
Our motivation is that learning reliable cross-view correspondences can itself reduce ambiguity in weakly textured regions, where local appearance is insufficient for matching. We use correspondence supervision to learn associations between observations of the same 3D points across views, encouraging the model to exploit global cross-view context rather than relying solely on local texture.
Level 2. Correspondence learning and global-context modeling are complementary. We apply soft supervision only in one global-attention block (layer 24), selected through layer-position ablations. This encourages attention toward geometric correspondences without forcing exclusive attention, while other layers retain global information exchange. Layer-position and component ablations support this selective design.
Level 3. Scene tokens encode shared scene-level representations rather than object-specific identities. The objective encourages corresponding patches to have compatible soft assignments without requiring non-corresponding patches to occupy distinct slots. Thus, assignment ambiguity may remain but does not necessarily imply reconstruction errors; its impact should be assessed through downstream reconstruction.
We compare the reduced ablation reference (Full) with a variant without both losses. Figure in the updated supplementary material shows stronger attention around geometric correspondences in three challenging scenes.
Table R4. Reconstruction on 20 selected samples with weak or repetitive texture.
Full improves all three metrics on these challenging samples.
A2. How do the three levels affect reconstruction metrics?
We analyze pose errors and cross-view depth-scale consistency on the same five benchmarks using reduced-model variants. Failure rate is the fraction of image pairs with max(rotation, translation-direction error) >30°. Shared-sequence AbsRel uses one scale per sequence; log-scale MAD measures cross-view scale variation, complementing standard per-view median alignment.
Level 1. Explicit correspondence learning helps the model capture reliable cross-view geometric relationships, supporting better reconstruction and pose estimation. Removing dense matching increases median rotation/translation-direction errors from 4.6°/5.8° to 7.7°/10.6° and the failure rate from 11.8% to 21.2%, highlighting its contribution to pose accuracy and robustness.
Level 2. Attention supervision encourages geometry-aware cross-view interactions for pose and dense reconstruction. Without it, translation-direction error rises from 5.8° to 8.0°, and shared-sequence AbsRel/log-scale MAD from 0.081/0.042 to 0.095/0.067.
Level 3. Scene tokens aggregate information across views to promote globally consistent scene representations. Their contribution is particularly evident in depth-scale consistency: removing the scene representation increases shared-sequence AbsRel from 0.081 to 0.109 and log-scale MAD from 0.042 to 0.081, despite a relatively small change in per-view AbsRel (0.072 to 0.080). This suggests that scene tokens improve global geometric consistency.
A3. Architecture generality
We progressively add three supervision levels to VGGT under matched initialization, training data, and evaluation protocols, and transfer LEVER to MASt3R.
Each level improves VGGT; LEVER also improves MASt3R across all three metrics, supporting transferability.