- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Poincar3, a self-supervised multi-view masked modeling framework for learning transferable 3D-aware features from an image sequence of a 3D scene. Poincar3 adopts a multi-view iBOT objective that encourages a student network to mimic the features from a teacher network despite patch-wise masking. Moreover, Poincar3 leverages an additional image-level objective and suggests that the teacher network should see more views than the student, which bring performance improvements.
Putting all together, Poincar3 learn strong multi-view features compared to previous single-view (e.g., DINO) and multi-view baselines, on zero-shot cross-view matching, feedforward reconstruction, as a (much) smaller computational cost. The ablation study clearly shows the contribution of each component in the proposed approach.
Critical strengths
(1) The proposed method, Poincar3, is conceptually simple and clean, making it easy for potential follow-ups and further developments.
(2) Empirical results show that Poincar3 learns transferable 3D-aware multi-view features, demonstrating better performance on multi-view matching and feedforward 3D reconstruction than previous single-view (e.g., DINO) and multi-view self-supervised methods (e.g., MuM). Importantly, this is achieved with much smaller computational costs than state-of-the-art models, e.g., DINO.
(3) The ablation study cleanly attributes the contribution of each proposed component.
Critical weaknesses
(1) Insufficient comparison. Poincar3 primarily demonstrates "3D" downstream tasks, such as cross-view matching and feedforward reconstructions, and it achieves inferior performance in terms of semantic understanding (e.g., ImageNet results) compared to single-view backbone. Given the focus of 3D-related tasks, the proposed method should be compared with the family of method that conducts multi-view self-supervised learning with novel-view synthesis (e.g., RayZer, E-RayZer, and their recent follow-ups), which also focus on 3D-related tasks.
(2) The scope of claim and analysis. The title of "Emergent Multi-View Geometry Through Self-Distillation" feels a bit too grand to me. The strongest evidence for the "emergent multi-view geometry" is from Section 4.3, where the difference of the features from different frames are trained to predict pose changes. The results from Tab. 3 show that previous methods, e.g., MuM, also work reasonably well, and even DINO does a reasonable job on this specific task.
First, I'd like to see if the performance on 3D-related task correlates well with this task (i.e., is this a useful task to indicate performance). Concretely, can we have an updated version of Table 4 that adds this R^2 metric? I'd also like to know more (intuitive or non-intuitive) explanations about why this specific task is a good measure for "emergent multi-view geometry".
Second, while empirically results, including comparison on 3D-related task and ablation study, provide evidence for a strong multi-view encoder, the analysis of why this happened is missing. For example, why is iBOT objective more preferred compared to DINOv2 in terms of 3D-related task? Why semantic understanding is much worse compared with DINO? I believe the answer to these questions may make this submission more complete and comprehensive.
(3) In terms of single-view backbone evaluation (e.g., for DINO), I wonder have the authors tried Depth Anything 3 style DINO adaptation, directly rearranging tokens so that DINO can itself reason over multi-view tokens. I think that would also be a reasonable baseline for "what if we directly adopt a single-view backbone into multi-view scenario".
Questions for authors
I primarily expect the authors explanation about "emergent multi-View geometry" and any insights about how the objective of learning affects performance.
4 comments
Second Response to Reviewer B1py
Comment [2/2]:
I'd also like to know more (intuitive or non-intuitive) explanations about why this specific task is a good measure for "emergent multi-view geometry".
We thank the reviewer for the opportunity to clarify how we measure emergent multi-view geometry. We view correspondence estimation as a key measure of 3D awareness, following Banani et al. [1] and the broader multi-view geometry literature, where point correspondences are a fundamental building block [2]. This is also consistent with recent findings that correspondence structure emerges in feed-forward reconstruction models [4, 5].
SeeSE3 [3] provides a complementary measure through motion prediction. As discussed in our response to reviewer 8xU6, we interpret this as evidence of locally consistent geometric structure rather than globally linear SE(3) representations. Together, the explicit correspondence structure in Fig. 2 and Tab. 2 and the implicit motion structure in Tab. 3 provide evidence of emergent multi-view geometry. We agree, however, that related structure is present to some extent in previous models as well, making the new title suggestion a better fit.
We refer the reviewer to the cross-scene SeeSE3 results in our response to reviewer 8xU6.
Understanding 3D and semantic performance
while empirically results, including comparison on 3D-related task and ablation study, provide evidence for a strong multi-view encoder, the analysis of why this happened is missing. For example, why is iBOT objective more preferred compared to DINOv2 in terms of 3D-related task? Why semantic understanding is much worse compared with DINO?
It has been observed in DINOv3 [6] that global and local crops can reduce patch locality, degrading dense performance. Similarly, [7] argues that aligning local and global crops can interfere with the quality of dense feature maps. In VGGT, patch-level correspondences are critical to pose estimation and geometric understanding [4]. We therefore interpret patch-level distillation as providing a better inductive bias for correspondence and geometry than objectives that place greater emphasis on global semantic invariance.
We find that this trade-off is substantially data-dependent. When trained on the same data, the DINOv2 objective achieves semantic performance comparable to our method. Since our architecture and objective do not require multi-view images, single-view data, e.g. ImageNet, could also be incorporated during training to improve semantic representations while retaining the multi-view signal. We leave this to future work.
We additionally experimented with DINOv3 initialization while keeping the rest of the training procedure unchanged. This improves semantic performance while retaining strong geometric performance. Nevertheless, we use the randomly initialized model as our main Poincar3 model to demonstrate that these geometric representations emerge from training from scratch.
Depth Anything 3 style DINO adaptation
I wonder have the authors tried Depth Anything 3 style DINO adaptation. [...] I think that would also be a reasonable baseline for "what if we directly adopt a single-view backbone into multi-view scenario".
We agree with the reviewer that this serves as a reasonable baseline and present the addtional results for Tab. 1 below:
| Method | RE10K @3° | RE10K @30° | ScanNet++ @3° | ScanNet++ @30° | MegaDepth @3° | MegaDepth @30° | ETH3D Acc. | ETH3D NC | DTU Acc. | DTU NC |
|---|---|---|---|---|---|---|---|---|---|---|
| Random init. | 0.0 | 17.4 | 0.0 | 11.3 | 0.0 | 35.9 | 1.24 | 0.60 | 12.34 | 0.56 |
| DINOv3 init. | 1.3 | 31.2 | 0.0 | 30.9 | 6.9 | 73.8 | 0.98 | 0.68 | 10.87 | 0.59 |
| DA3-style init. | 1.1 | 30.5 | 0.7 | 33.2 | 6.4 | 71.2 | 1.11 | 0.63 | 12.51 | 0.59 |
| Poincar3 | 8.3 | 68.2 | 4.5 | 67.0 | 7.4 | 74.6 | 0.81 | 0.80 | 6.73 | 0.63 |
As can be seen, Poincar3 also outperforms this baseline as an initialization for feed-forward reconstruction.
[1] El Banani et al., Probing the 3d awareness of visual foundation models, in CVPR 2024.
[2] Hartley and Zisserman, Multiple View Geometry in Computer Vision, Cambridge University Press, 2004.
[3] Chen et al., SeeSE3: Emergence of 3D Space in Vision Features, in NeurIPS 2026.
[4] Bratulić et al., On Geometric Understanding and Learned Priors in Feed-forward 3D. Reconstruction Models, in ECCV 2026.
[5] Starý et al., Understanding Multi-View Transformers, in ICCVW 2025.
[6] Siméoni et al., DINOv3, in TMLR 2026.
[7] Seitzer et al., Scalable Patch-Level Self-Supervised Learning, pre-print 2026.
Dear authors,
Sorry for the late reply and I appreciate your detailed response and efforts. My concerns have been addressed and I've decided to raise my score to 3.
Best, Reviewer B1py
Thank you Reviewer B1py
We again would like to thank Reviewer B1py for the time and effort put into the review and ultimately improving the paper through their suggestions.
Response to Reviewer B1py
Comment [1/2]: We thank reviewer B1py for the careful reading of our paper and for helping improve it. The rebuttal is split into two comments due to space limitations. We are also happy to address any further concerns of the reviewer during the discussion period.
Insufficient comparisons
We agree that these methods provide valuable additional baselines. We therefore added RayZer, E-RayZer, and xFactor (already cited in the manuscript) to the multi-view correspondence (Tab. 2) and feed-forward reconstruction (Tab.1) evaluations.
First, the results of multi-view correspondence estimation using feature nearest-neighbor matching:
Secondly, feed-forward reconstruction (Tab. 1):
For these experiments, we tried multiple configurations to see what part of the models (pose encoder or geometry decoder) that features best are extracted from. For the feed-forward reconstruction experiment, we use the camera tokens from the pose encoders and the dense features from the geometry decoders while for the correspondence estimation we simply use the best perfoming of the two. RayZer was excluded from the feed-forward reconstruction comparison due to computational constraints (it was also the weakest of the compared methods in the correspondence evaluation).
Overall, Poincar3 provides substantial improvements on both correspondence estimation and feed-forward reconstruction. xFactor's camera tokens perform particularly well on RE10K pose estimation.
Title of the paper
We agree with the reviewer that an alternative title might fit the central claims of the paper better and appreciate the feedback about the initial title ("Emergent Multi-View Geometry Through Self-Distillation"). We propose instead to name the paper “Poincar3: Multi-View Self-Distillation for 3D Representation Learning”, but we are open to further changes if requested by the reviewers.