- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes a warping-based training objective for self-supervised 3D reconstruction, where both the camera poses and depth maps of the given input views are predicted without intermediate 3D representations like 3DGS needed. Experiments show that it outperforms other state-of-the-art self-supervised reconstruction paradigms in camera pose estimation and depth prediction.
Critical strengths
- The idea itself is technically sound, simple, and clean.
- The empirical results, particularly the zero-shot camera estimation, are strong.
- The ablation study is broad and informative, in which many designs are investigated.
Critical weaknesses
- Despite the simplicity, the strong geometric inductive bias underlying reprojection also limits the upper ceiling of Warp3R. Specifically, the reprojection assumes that the input views are from a single rigid scene. Other complex effects (e.g., object deformation, illumination change, view-dependent appearance) might not be properly modelled. In other words, while the method is very simple and effective at modelling rigid geometric correspondence, it sacrifices its capability to handle complex and unavoidable effects in real scenes beyond geometric correspondence and thus results in a clear boundary in applicability.
- E-RayZer and NAS3R learn 3DGS to support novel-view synthesis rather than directly optimizing geometric reconstruction like camera poses and depths. In other words, the ultimate goal of E-RayZer and NAS3R is to improve rendering quality at novel views, which does not necessarily mean explicitly targeting more accurate camera poses and depths like the structure-from-motion methods. Instead, Warp3R uses a reprojection-based objective, directly and explicitly targeting camera pose/depth estimation accuracy. While the advantages are well acknowledged, they do not necessarily imply that warping is a superior reconstruction paradigm overall. It might show that "in the accurate geometric reconstruction case where the reprojection-based objective is favored, the method performs better than other methods that are not primarily designed for the task but prioritize perceptual quality". I do not consider this a critical weakness of the method itself, but rather a concern about the experimental setting.
Questions for authors
- NAS3R enables self-supervised learning from scratch. Can Warp3R realize this without any prior? The failure may indicate the optimization difficulty or an intrinsic limitation of the reprojection-based supervision and provide insights for future improvements.
- Can the authors also provide the 3D-point evaluation results like VGGT? This can greatly strengthen the claim of "3D reconstruction".
- Can the authors provide any discussion or analysis regarding the boundary or failure patterns when the rigid-scene reprojection is violated?|
- Currently, Warp3R explicitly uses temporal adjacency by setting the temporal neighbourhood to adjacent views. Can this be extended to unordered views or randomly sampled frames? This directly answers whether Warp3R is a multi-view reconstruction method or specifically a video reconstruction method.
- Given the same data budget, how is this self-supervised learning objective compared with the supervised counterpart like VGGT?
3 comments
We thank the authors for the brief response. It addresses some of my concerns while leaving W1 and W2 open. The dependency on DINOv2 initialization also reflects limitations. I retain my current score.
Thank you for the follow-up and for acknowledging the concerns addressed by our response. Regarding the remaining concerns, we would like to elaborate on W1 and W2 and clarify the role of initialization.
W1. “illumination change, view-dependent appearance”
We agree that strong illumination changes and view-dependent appearance can undermine photometric agreement even when the geometry is correct, and can be a limitation.
This actually gets to the heart of why we find the result interesting: these classical losses nevertheless train modern VGGT-like systems effectively on real-world DL3DV video containing these appearance changes, yielding strong zero-shot pose accuracy (Table 1), with qualitative examples of accurate poses in reflective evaluation scenes (Figure 10). Figure 6 also shows that purely self-supervised training achieves higher mean pose accuracy (RPA@5) on those same real-world benchmarks than the supervised counterpart, with datasets, frame budgets, and architecture matched.
We believe this is useful for the community: it establishes an effective training recipe that others can build on, and a concrete basis for investigating how far classical warping objectives can be taken in modern reconstruction systems.
W1 (continued). “object deformation” and dynamic objects
Independent object motion and deformation are not modeled by our static-scene formulation, and we agree that this is a current limitation. We will clarify this in the paper.
We view the demonstrated effectiveness of this training recipe as a useful milestone that can inform future efforts to address dynamic scenes.
W2. “E-RayZer and NAS3R learn 3DGS to support novel-view synthesis”
Thank you for highlighting the distinction between novel-view image quality and geometric accuracy. We agree that the additional rendering goal can motivate different design choices.
Both E-RayZer and NAS3R evaluate camera estimation and, like Warp3R, learn cameras and explicit geometry through image reconstruction without camera or depth annotations. E-RayZer says it “operates directly in 3D space, performing self-supervised 3D reconstruction with Explicit geometry.”
Tables 1–2 compare this shared task using matched reconstruction training datasets and their pose evaluation protocols, with all methods evaluated on the same frames.
I do not consider this a critical weakness of the method itself, but rather a concern about the experimental setting.
We appreciate your distinguishing the method from the interpretation of these comparisons. Under these shared conditions, the complete Warp3R system achieves the strong camera accuracy.
Initialization. “dependency on DINOv2 initialization”
Table 3(k–m) shows strong results with MoCo v3, MAE, and Pixio, demonstrating that successful optimization is not specific to DINOv2. In each case, we repurpose blocks pretrained on individual images for attention across frames from initialization (lines 152–156), extending their use beyond the single-image attention setting of pretraining.
Though both the initialization and the subsequent reconstruction training are fully self-supervised, we agree that requiring pretrained self-supervised initialization remains a limitation.
In light of these results, we would appreciate your reconsidering the weight placed on this initialization limitation when assessing the overall contribution of this work.
Thank you for highlighting the "strong" zero-shot camera results. E-RayZer and NAS3R address both geometric reconstruction and novel-view synthesis, which we agree are distinct evaluations. Tables 1–2 compare camera accuracy using their established evaluation protocols and matched reconstruction-training datasets: DL3DV for E-RayZer and RE10K for NAS3R. These results demonstrate the effectiveness of a simple warping-based system for geometric reconstruction.
1. “NAS3R enables self-supervised learning from scratch. Can Warp3R realize this without any prior?”
Warping alone has not reliably converged from random initialization in our runs. Strong results with alternative self-supervised encoders (Table 3k–m) show that successful optimization is not specific to DINOv2 initialization.
2. “Can the authors also provide the 3D-point evaluation results like VGGT?”
Thank you for suggesting this experiment. We have not performed a quantitative point-cloud evaluation of the submitted model. Tables 1–2 and Figure 5 assess pose and depth; Figures 3 and 7 show qualitative point-cloud reconstructions.
3. Discussion of “object deformation, illumination change, view-dependent appearance”
Like E-RayZer (their §5), Warp3R assumes a static scene. Object motion can create image changes a rigid-scene model cannot explain; strong illumination changes and view-dependent appearance can cause photometric errors even with correct geometry. We will discuss these limitations and directions for future work in the final version.
Our results show that training on real-world DL3DV videos with natural lighting and appearance variation yields strong zero-shot pose results (Table 1); Figure 10 illustrates accurate poses in reflective scenes present in our evaluations.
4. “Warp3R explicitly uses temporal adjacency … Can this be extended to unordered views or randomly sampled frames?”
Our training recipe uses temporal continuity in static-scene video (our paper, lines 253–258). Training without frame-index embeddings gives a test-time permutation-equivariant variant with 84.4 DL3DV RPA@5 (Table 3n). Expanding the temporal warping neighborhood from ±1 to ±2 or ±3 retains comparable or improved DL3DV pose accuracy (Table 3f–g). We have not tested random-frame training with the default configuration.
5. Comparison with VGGT-style supervision: “Given the same data budget”
Figure 6’s fully supervised and fully self-supervised endpoints use the same number of frames from the same datasets, with architecture, initialization, batch size and updates fixed. Self-supervision yields higher mean RPA@5; the VGGT-style supervised counterpart yields slightly lower depth error.