acceptodds
Under review as a conference paper at ICLR 2027

BridgeView: Enhancing Cross-View Consistency of Vision Foundation Models via Frozen Opposite-View Targets

Abstract

Vision foundation models provide powerful frozen features, yet image-based pretraining does not require two observations of the same surface to share a descriptor. Existing approaches enforce this relation with an intermediate scene representation and a rendered teacher. Reconstruction is costly, and the rendered teacher is a visibility-weighted mixture in the reconstructed field, so matching it moves student features away from pretrained coordinates. Leaving that coordinate system raises scene-level agreement while frozen semantic transfer falls. Instead, we establish cross-view consistency directly inside the pretrained coordinate system. To this end, we propose BridgeView, which uses the frozen opposite-view descriptor as the teacher on a same-surface pair formed by calibrated geometry. When the encoder cell at that pixel mixes neighboring instances, a bounded residual refines this readout with instance context while remaining inside a cone of the frozen vector. When many pairs observe one instance relation, their backward updates are coordinated inside that relation. We validate BridgeView on indoor RGB-D benchmarks. Across DINOv3 backbones, it raises cross-view correspondence, semantic segmentation, visual localization, and TSDF map rendering, and trains over faster than SnD. Scaling the pair corpus to 3M further improves correspondence. These results suggest a scalable route for large vision foundation models to obtain cross-view consistency inside the pretrained coordinate system. Code is provided in the supplementary material.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Vision Foundation Models, Cross-View Consistency, Dense Correspondence

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_p9spOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes BridgeView, a cross-view representation enhancement framework for vision foundation models that aims to improve cross-view consistency without relying on intermediate 3D scene reconstruction or rendered feature teachers. Given calibrated RGB-D image pairs, BridgeView first constructs reliable same-surface correspondences and directly uses the frozen descriptor from the opposite view as the supervision target. To improve the reliability of this pair-based supervision, the method further introduces a bounded instance residual to mitigate mixed-instance token contamination and relation gradient coordination to reduce inconsistent updates among samples sharing the same instance relation. Experiments on indoor RGB-D benchmarks and DINO-family backbones show that BridgeView improves cross-view correspondence, semantic segmentation, visual localization, and TSDF map rendering, while achieving substantially lower training cost than reconstruction-based methods such as SnD; scaling the training corpus to 3M pairs further improves correspondence performance.

Critical strengths

BridgeView replaces the commonly used reconstruction-and-rendering pipeline with direct supervision from frozen opposite-view descriptors on geometrically certified same-surface pairs. This design is conceptually simple yet effective, and avoids the additional complexity introduced by intermediate 3D scene representations and rendered feature teachers.

The method consistently improves cross-view correspondence and semantic transfer across different DINO backbone sizes, including DINOv3-7B, and also demonstrates gains on visual localization and TSDF map rendering. The evaluation therefore provides relatively broad evidence that the learned cross-view consistency transfers beyond the direct correspondence objective.

By removing scene reconstruction and feature rendering, BridgeView substantially reduces training cost compared with prior methods such as SnD. The reported experiments show over 10×10\times training speedup, lower memory consumption, and continued performance improvements when scaling the training corpus to 3M view pairs.

Critical weaknesses

The paper claims that BridgeView makes two observations of the same surface share a descriptor in the pretrained coordinate system. However, the current evaluation mainly demonstrates improved correspondence accuracy rather than directly measuring the similarity between the learned student descriptors across views. Since the training objective supervises each student feature with the frozen descriptor from the opposite view, it remains unclear whether the learned representation is genuinely view-invariant or whether the model could learn view-dependent transformations between paired observations. A direct analysis of student-to-student feature consistency would make this central claim more convincing.

The paper attributes BIR to correcting mixed-instance token contamination and RGC to reducing gradient disagreement or variance within an instance relation. However, there is no subgroup analysis based on token occupancy or boundary mixing, nor any direct measurement of gradient variance/alignment before and after RGC.

The experiments are restricted to the DINO family of vision foundation models. Although BridgeView performs consistently across different DINO backbone scales, other foundation-model families are equally important and may possess different feature geometries. It would therefore be valuable to verify whether BridgeView remains robust and effective on other representative foundation models, rather than being specific to DINO-based representations.

Questions for authors

For a certified pair (uia,uib)(u_i^a,u_i^b), BridgeView supervises each student readout using the frozen descriptor from the opposite view, i.e., $ z_i^a = R(Z^a,u_i^a) \rightarrow p_i^a = \mathrm{sg}!\left(R(\bar{Z}^b,u_i^b)\right), $ and symmetrically for view bb, with the final supervision applied to the BIR-refined target tivt_i^v. I would appreciate some clarification on why this symmetric frozen-target formulation leads to a genuinely shared cross-view representation, rather than learning view-dependent mappings between paired observations. For an individual pair, the two directional objectives do not explicitly enforce zia≈zibz_i^a \approx z_i^b. Is the desired cross-view invariance expected to emerge primarily from the shared encoder parameters together with the large number of overlapping same-surface pairs?

The paper motivates BIR as addressing mixed-instance token contamination and RGC as reducing gradient disagreement or variance within an instance relation. However, the current ablation mainly shows that adding these components improves downstream performance, without directly verifying the proposed mechanisms. Could the authors provide more direct evidence that BIR indeed benefits samples with stronger token mixing or boundary ambiguity, and that RGC actually reduces gradient variance or improves gradient alignment within the same relation group? Such analyses would help clarify whether the observed gains arise from the intended mechanisms rather than from more general regularization effects.

1 comment
Reviewer_p9spOpenReview ↗

Response to the No.7130 author

Thank you for your response; my confusion has been resolved, and I will maintain my positive rating.

Reviewer_L927OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents BridgeView, which finetunes DINO encoders with LoRA for cross-view consistency, without scene reconstruction. Using ground truths depth and poses on ScanNet++ RGB-D pairs, the student descriptor at a pixel is trained to match the frozen teacher descriptor at the corresponding pixel of the other view. The method has three components:

  1. CFPR: keeps only pairs that pass bidirectional 3D, reprojection and depth checks,
  2. BIR: refines the target with SAM instance prototype and boundary residuals, capped within a 12° cone of the frozen descriptor,
  3. RGC: mixes each pixel's gradient with the mean gradient of its instance pair.

On DINOv2 and v3 at different sizes, the method improves ScanNet correspondence by large margins, gives small linear probe segmentation gains, and improves localization and TSDF rendering. Also, it is compute efficient compared with SnD, training 10x faster.

Critical strengths
  1. Simple and cheap with respect to other compared methods. It drops reconstruction and rendering entirely, uses 0.114 vs 1.170 GPU s/view compared with SnD (Table 3), trains DINOv3-7B where SnD runs OOM (Table 8). This is interesting because it makes 3D-aware finetuning practical for the largest VFMs.
  2. Consistent correspondence gains that transfer to geometric tasks. On DINOv3-B, R@10 rises from 35.32 to 46.34 and median error falls from 33.75 to 12.88 pixels. On DINOv3-7B, R@10 rises from 45.43 to 58.21. Localization recall and TSDF metrics also improve (Table 6), and semantic probes are not degraded.
Critical weaknesses
  1. The central claim which is consistency inside the pretrained coordinate system is not directly supported.
  • CFPR alone, the component that supports the claim degrades frozen probes below the original model at 3M pairs. On DINOv3-B, ScanNet++ mIoU drops from 35.90 to 35.69, and ScanNet from 51.20 to 50.36 (Table 5). Recovery comes from BIR, which injects SAM instance information. The semantic gains may therefore come from SAM supervision rather than from staying in pretrained coordinates. The claim that the method scales better seems thus not true for data scaling.
  • RGC is ablated only through small probe changes, and without correspondence metrics. Its stated mechanism is never measured: Figure 3 only plots the analytic σ2/∣Gi∣\sigma^2/|G_i| under an iid noise model the paper itself says may not hold (l.252). The bias from pulling per pixel updates toward an instance mean, even though pixels on different surface points have different targets is not discussed.
  1. Evaluation reliability and positioning.
  • RoMa v2 reaching only 4.60% R@10 and 55.71 pixels median error on ScanNet (Table 2) is implausible for a state-of-the-art dense matcher, and suggests an evaluation protocol mismatch. The conclusion that "repairing the encoder (...) is what raises correspondence" l.345 is not supported.
  • The baselines often underperform the original model. SnD worsens DINOv3-S correspondence (R@10 drops from 37.50 to 31.75), and FiT3D and SnD both lower mean mIoU (Table 9). This raises questions about how faithful and well-tuned the reimplementations or evaluations from official checkpoints are. BridgeView also uses more supervision (metric depth, poses, SAM masks).
  • The probe gains are small, about +0.3-0.8 mIoI for DINOv3-B, variance is reported only for BridgeView and not across training seeds, and depth RMSE is worse than the original model on DINOv3-S ScanNet and NYUv2.
  • The closest prior work from You et al. (Multiview Equivariance Improves 3D Correspondence Understanding with Minimal Feature Finetuning, ICLR 2025) which also finetunes DINO features using LoRA adapters with 3D correspondences, is missing. A plain correspondence supervised finetuning baseline on the same pairs is needed to isolate what the different components (CFPR, BIR, RGC) add.
Questions for authors
  1. Please give the exact protocol for RoMa v2 evaluation (resolution, query sampling, ...), and a sanity check against its ScanNet-1500 numbers.
  2. Were FiT3D and SnD reimplemented or taken from official code? How were they tuned for DINOv3, and why does SnD hurt DINOv3-S correspondence?
  3. How is the 640K subset selected, and why does it differ so much from 3M at an equal number of samples seen?
  4. How does BridgeView compare with plain correspondence finetuning (You et al., 2025)?
Reviewer_scg1OpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper leverages multi-view correspondences to improve feature consistency across observations of the same physical surface. By directly using frozen features from the corresponding location in another view as supervision, the proposed method avoids reconstruction and rendering, substantially reducing training time. The approach is evaluated on multiple indoor scene datasets.

Critical strengths
  1. The paper addresses an important problem and proposes an interesting and intuitive approach to improving cross-view feature consistency.
  2. The experimental evaluation is comprehensive, with comparisons across multiple datasets, backbone architectures, and downstream tasks.
Critical weaknesses
  1. The readability of the figures could be improved. Some text is difficult to read, and the inconsistent use of horizontal and vertical labels makes the diagrams harder to follow.
  2. Increasing the training data leads to performance degradation across several evaluation datasets and settings. This behavior warrants further analysis, particularly given the paper’s emphasis on scalability. The authors should provide a clearer explanation of when and why additional training data hurts performance.
Questions for authors

see weakness

Reviewer_GMYmOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper improves the cross-view consistency of frozen vision foundation models such as DINOv2/DINOv3 without building an intermediate scene representation. Using RGB-D depth and poses, it finds geometrically verified same-surface pixel pairs and trains a LoRA student so that its feature in one view matches the frozen teacher's feature at the corresponding pixel in the other view. Two additional components refine this: a bounded instance residual based on SAM instances (BIR), and relation-level gradient coordination (RGC). Experiments on indoor RGB-D benchmarks report gains in cross-view correspondence, linear-probe segmentation, visual localization, and TSDF rendering. Training is over 10× faster than SnD and scales to DINOv3-7B.

Critical strengths
  • The method is simple and cheap, needing no reconstruction or rendering (Tab. 3), and it scales to a 7B backbone (Tab. 8).
  • Correspondence improves substantially and consistently across backbones, e.g. DINOv3-B R@10 rises from 34.03 to 46.34 (Tab. 1). Localization and TSDF results improve in the same direction (Tab. 6).
  • Frozen semantic transfer is preserved or slightly improved (Tab. 9).
Critical weaknesses
  • RoMa v2 obtains only 4.60% R@10 (Tab. 2), far below the frozen DINOv3-B nearest-neighbor result (35.32%). This is surprising for a state-of-the-art dense matcher, and App. A.1 does not describe how correspondences are extracted from RoMa v2. The conclusion drawn from this comparison, that repairing the encoder rather than adding a matcher is what raises correspondence, therefore needs support.
    • The reproduced SnD performs below the original backbone on correspondence for DINOv3, e.g. DINOv3-S R@10 is 31.75 vs. 37.50 (Tab. 1). This suggests the baseline may not be well tuned in this setting.
    • Prior work that also fine-tunes 2D features with multi-view correspondences, such as Multiview Equivariance (You et al., ICLR 2025), is not discussed or compared.
Questions for authors
  1. How were RoMa v2 correspondences obtained? Was its dense warp sampled at the query pixels, and at what resolution? Can you confirm the implementation, for example with RoMa v2's standard ScanNet-1500 result?
  2. How were FiT3D and SnD retrained on DINOv3 (hyperparameters, tuning)? Does your reproduction match the original papers in their original DINOv2 setting?
  3. How does BridgeView compare with Multiview Equivariance (You et al., ICLR 2025)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.