- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper follows recent approaches that learn 3D geometry from unposed images by jointly predicting camera parameters and scene structure, followed by rendering held-out views for self-supervision. Unlike these methods, the proposed approach introduces self-supervised asymmetry during 3D scene construction rather than feature extraction. It further exploits the asymmetry between sparse and dense representations, and incorporates feature alignment with a frozen vision foundation model, leading to additional performance gains.
Critical strengths
- The paper is clearly written and easy to follow. The figures are well designed and effectively illustrate the key ideas and experimental results.
- The main strength of the paper is the idea of introducing asymmetry in the 3D representation space rather than during feature extraction. This is both novel and well motivated. When the held-out views are interpolated between the context views, they primarily provide additional correspondences to improve geometric accuracy instead of introducing new scene content. Consequently, it is only necessary to prevent a held-out view from contributing to its own 3D representation during rendering. Based on this insight, the authors employ full-view attention during feature computation while excluding the held-out views from the 3D scene construction, enabling richer feature interactions without introducing information leakage.
Critical weaknesses
- The proposed full-view strategy relies on the assumption that held-out views do not introduce new scene content. While this is reasonable for simple camera trajectories, it may not hold for datasets with complex viewpoint changes, where even interpolated views can reveal previously unseen regions. It would be helpful for the authors to discuss whether the proposed strategy remains valid under such settings and what limitations it may have.
- The comparison with prior methods raises some fairness concerns. The proposed model is initialized with the Pixio (Base) visual encoder and further benefits from semantic guidance provided by both DINOv3 and Pixio, all of which are strong 2D pretrained models. In contrast, the compared baselines, including NAS3R and Erayzer, are trained from random initialization. Although Erayzer leverages pretrained priors for view sampling, these priors are not directly incorporated into the reconstruction model itself. As a result, it is difficult to disentangle the gains brought by the proposed method from those attributable to the powerful pretrained priors. This concern is further supported by Table 7, which shows that the pretrained components contribute substantially to the overall performance. It would therefore strengthen the paper if the authors could provide comparisons under a setting without pretrained initialization or semantic guidance.
- Although the paper primarily focuses on geometric evaluation, it would be valuable to include a more comprehensive evaluation of NVS performance. Self-supervised 3D Gaussian representations are also valuable for rendering applications. In addition to comparisons with self-supervised 3DGS methods, it would be helpful to include comparisons against conventional GS methods, including both pose-required and pose-free approaches, to better position the proposed method with respect to the broader literature.
Questions for authors
- The proposed full-view strategy appears to be particularly well suited to pixel-aligned Gaussian representations. It would be interesting to discuss whether this idea could be extended to non-pixel-aligned paradigms, such as C3G [1] or TokenGS [2], even though these methods are not designed for self-supervised learning. A discussion of the generality of the proposed strategy would help clarify its broader applicability.
- According to the supplement, the training views are sampled solely based on frame intervals, I am curious about how stable is the optimization for difficult cases?
[1] C3G: Learning Compact 3D Representations with 2K Gaussians, CVPR 2026 [2] TokenGS: Decoupling 3D Gaussian Prediction from Pixels with Learnable Tokens, CVPR2026
8 comments
(W3) NVS evaluation and comparison with conventional GS methods
Our work treats NVS as a proxy task for self-supervised geometry learning, not as the end goal. Rendering provides a training signal without labels, and we evaluate the geometry it teaches, namely camera pose and depth. This is also what makes our full-view design possible. Since rendering quality is not the target, we let target images inform the context features to learn richer cross-view correspondences, and only prevent each target from being rendered from its own prediction. As a result, the rendering metrics of the full-view model do not measure novel view synthesis, so we gray them out rather than compare them with GS methods.
To still assess rendering without leakage, Appendix A.3 trains with full-view attention for the first 80% of iterations and switches to masked attention for the rest (Full-view → Masked-view). We compare it with the leakage-free designs trained for the same total schedule (DL3DV, averaged over 5/7/9/11 views):
| Method | PSNR↑ | SSIM↑ | LPIPS↓ |
|---|---|---|---|
| Separated | 19.88 | 0.563 | 0.367 |
| Masked-view | 20.35 | 0.599 | 0.342 |
| Full-view → Masked-view | 20.64 | 0.620 | 0.322 |
The switched model renders best on all metrics and at every view count, suggesting that the geometry learned under full-view attention also benefits rendering. We will make this scope clearer in the main text and add a discussion of conventional pose-required and pose-free GS methods to the related work.
(Q1) Extension to non-pixel-aligned representations
Our full-view strategy relies on pixel-aligned Gaussians. Full-view attention is safe only because a target view is never rendered from its own prediction, which requires knowing which view each Gaussian comes from. With pixel-aligned Gaussians, we simply drop those predicted from target pixels and render targets from context-view Gaussians. In non-pixel-aligned methods such as C3G and TokenGS, Gaussians are decoded from queries or tokens that aggregate all views, so target information cannot be separated from the scene. Applying full-view attention directly would then let targets be reconstructed from their own content, a shortcut the rendering loss cannot detect.
We also tried predicting Gaussians freely in 3D space from latent tokens, as in non-pixel-aligned methods, while restricting these tokens to attend only to context views. This did not train well in our self-supervised setting. We believe this is because pixel-aligned Gaussians are tied to the predicted camera rays. Their positions are defined by depth along each ray, so the rendering loss also supervises the cameras through the geometry. Free-space Gaussians lose this coupling, and extending our strategy to them remains an open direction that we will discuss in the revision.
Following up on our response to (Q1), we ran an additional experiment that relaxes per-pixel alignment.
Following ReSplat and iLRM, the model predicts one Gaussian per 2×2 pixel cell instead of one per pixel (4× fewer Gaussians), with a learned offset that moves its ray within the cell. Everything else is unchanged. Since each Gaussian is still predicted from a single view, target-view Gaussians can be dropped exactly as before, and full-view attention remains safe. Results on DL3DV, averaged over 5/7/9/11 views:
| Gaussians | R3 | R5 | R10 | A30 |
|---|---|---|---|---|
| Per pixel (ours) | 66.6 | 81.7 | 91.7 | 88.0 |
| Per 2×2 cell + offset | 64.9 | 82.3 | 92.1 | 88.3 |
R3/R5/R10: RPA@3/@5/@10; A30: AUC@30.
Pose accuracy remains comparable. This refines the requirement stated in our response. The strategy does not need strict per-pixel alignment, but each Gaussian must (i) come from a known source view and (ii) be placed along a ray of that view. Free-space Gaussians decoded from tokens that aggregate all views satisfy neither condition, which is why they remain an open direction. We will include this experiment in the revision.
We thank the reviewer for the careful reading and constructive questions. We address each point below; all new results will be added to the revision.
(W1 & Q2) Validity of the full-view assumption under complex viewpoint changes, and optimization stability
What the assumption requires. A target view may attend to the context views, but it is never rendered from its own prediction: Gaussians come from the context views only. Target content not covered by the context therefore cannot be reconstructed, and its photometric error cannot be reduced. Our assumption is thus not that targets contain no unseen content, but that such content is a small fraction of each target. As it grows, these pixels act as noise in the rendering loss.
Stress test. We widen the frame interval between sampled views during training and at test time; wider intervals mean larger viewpoint changes and more unseen content in each target. We also test a simple remedy, a per-pixel confidence-weighted photometric loss c⋅∥I^−I∥2−αlogc with α=0.02. A separate DPT head with the same architecture as the Gaussian head predicts a confidence c∈(0,1) (sigmoid) for each target pixel, so the model can down-weight pixels the context cannot explain. Results on DL3DV, averaged over 5/7/9/11 views (evaluation span in frames):
R3: RPA@3; A30: AUC@30.
The predicted confidence is low on content absent from the context views, such as objects entering the frame, and on thin, high-frequency structures that are hard to reconstruct. We will add this analysis, visualizations of the confidence maps, and a discussion of this limitation to the revision.
(W2) Fairness of comparison with respect to pretrained priors
We first clarify that our main model uses a single pretrained model, Pixio, both for initialization and as the semantic guidance teacher; DINOv3 appears only as an alternative in the ablation (Tab. 7).
We agree that the pretrained prior contributes substantially (Tab. 7) and that the comparisons with NAS3R and E-RayZer in Tab. 1–2 are not prior-matched. Our controlled comparison (Sec. 3.2, Fig. 3) isolates the design choice from the prior. All variants use the same pretrained encoder, backbone, data, and training schedule, and none uses semantic guidance or context alignment. The separated variant follows E-RayZer and even has more capacity (an additional 12-layer transformer). Under these matched priors, full-view outperforms both alternatives in Fig. 3, and the ranking holds after supervised fine-tuning of each variant (50K iterations, following Tab. 4):
D/S/N: DL3DV/ScanNetv2/NRGBD; R3/R5/R15: RPA@3/@5/@15; A30: AUC@30.
The advantage of our design therefore does not come from the prior.
We acknowledge that this comparison is prior-matched rather than prior-free. Tab. 7 reports our model without pretrained initialization, but this analysis-scale model is not directly comparable to the baselines in Tab. 1–2, which use full-scale training. In the revision, we will state this explicitly for Tab. 1–2 and refer to Fig. 3 and Tab. 7, which separate the contribution of our design from that of the prior.