- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents NBS, a stereo matching model built on a DINOv2-initialized ViT-Large with alternating within-view and cross-view attention and a DPT disparity prediction head. The method combines high-resolution learnable positional embeddings, standard disparity and gradient-matching losses, and large-scale synthetic training. It reports strong accuracy on ETH3D, Middlebury, and two additional evaluation datasets, together with substantial inference speedups over selected stereo baselines. The main claim is that explicit stereo-specific matching and refinement modules are unnecessary for achieving state-of-the-art performance.
Critical strengths
-
NBS achieves excellent results on ETH3D, including a substantial reduction in strict disparity errors relative to recent methods. The results demonstrate that the proposed pipeline can achieve high reconstruction accuracy without explicit cost volumes or recurrent refinement. The reported combination of accuracy and efficiency is practically relevant.
-
NBS-Base, trained exclusively on public stereo data, achieves the best EPE on SimpleProc-S (0.25 versus 0.29 for S2M2). This is useful evidence that the approach's performance cannot be attributed solely to the additional proprietary high-resolution training data.
Critical weaknesses
-
The central conceptual claim is substantially anticipated by prior work, and the incremental scientific contribution remains insufficiently established. CroCo v2 already achieved state-of-the-art stereo performance without cost volumes or iterative refinement, while DA3/VGGT established the single-pretrained-ViT paradigm with within-view/cross-view attention closely followed by NBS. The key question is therefore what NBS establishes beyond improving the performance of this existing paradigm. While the evaluation is extensive, they not entirely resolve this. In addition, comparing stereo-pair counts does not disentangle architecture from training exposure, initialization, optimization, or resolution, despite the claim in Appendix A.5. The positional-embedding ablation establishes an improvement on FSD, but does not establish its contribution to the main cross-domain results. I acknowledge the strong system performance; however, the manuscript does not sufficiently establish a distinct, transferable finding.
-
The main ETH3D and Middlebury results rely on benchmark-specific fine-tuning, which does not directly establish zero-shot stereo generalization. Although Figure 1 provides a qualitative zero-shot example, quantitative zero-shot disparity evaluations on these standard real-world benchmarks are not clearly reported. The high-resolution results are also obtained using separately adapted models with an expanded training mixture, making it difficult to distinguish the effects of increased resolution, additional training data, and the proposed design choices. Furthermore, the reported fourfold inference speedup is measured at 966×546 resolution, while the corresponding accuracy-latency trade-off at higher resolutions is not established. The SimpleProc and XYZ-IBD evaluations provide useful additional evidence, but the former remains synthetic and the latter contains only 55 selected stereo pairs, limiting conclusions about broader real-world transfer. They leave uncertainty about how broadly the advantages extend beyond the particular evaluation settings. Given the limited methodological novelty, I consider a clearer characterization of these empirical advantages important for assessing the paper's contribution.
Questions for authors
- What is the main contribution or findings beyond CroCo v2, VGGT, and DA3, particularly given the similar attention architecture used in DA3?
- Can the authors report zero-shot NBS-Base results on ETH3D and Middlebury and clarify the fine-tuning conditions of the baselines?
- Can the authors provide training exposure, compute, and dataset sampling ratios? What evidence supports the broader architecture-versus-scaling conclusion?
- Does the high-resolution PE improve performance on independent real-world data? How do NBS-HighRes latency and memory compare with baselines, including Fast-FoundationStereo?
- How are depth scale and camera calibration handled for DA3/VGGT in Table 4, particularly given the unusually large AbsRel values for DA3Large?
- How were the 55 XYZ-IBD pairs selected, and how similar are these scenes to the proprietary training data?
4 comments
Q6, XYZ-IBD.
- Selection: The released raw data contains 60 scenes; 55 have valid ground-truth depth, and the other five were excluded due to ground-truth artifacts. In each scene, we pick the two frames closest to a rectified stereo pair that both contain the full bin, and fuse five structured-light scans into the ground truth.
- Overlap with training data: Part of our internal synthetic dataset contains top-down views and industrial parts, but none of the XYZ-IBD objects, and the XYZ-IBD camera intrinsics and viewpoints were not replicated.
Thank you for the detailed response and additional experiments. The added results address several of my concerns by strengthening the evidence for NBS’s accuracy, efficiency, and zero-shot generalization. I also appreciate the real-world PE ablations and the clearer characterization of the system’s practical advantages.
My remaining reservation concerns the significance of the advance over prior work. The architecture, losses, and acceleration techniques largely build on established methods. The PE-related findings provide useful design insights, but I view the main contribution as a strong empirical realization of established design principles, with a more limited methodological or explanatory advance. Limited methodological novelty does not by itself preclude acceptance, and I consider the strong empirical results an important contribution, so I would like to decide my final evaluation after dicussion with other reviewers.
Thank you for the thoughtful follow-up and constructive feedback.
We intentionally kept our model simple and general. By avoiding new custom modules, we show that high-quality stereo reconstruction does not require stereo-specific components.
Thank you for the thorough review. The shared Table G1 and Table 5 are in the general response and the revised main paper, respectively; the public-only high-resolution result is in our reply to Reviewer EHsk (W2/Q1).
Q1, contribution. We show that stereo-specific modules are not a prerequisite for top-tier stereo matching: a single strong counter-example suffices—a standard ViT+DPT with no cost volume, 3D convolution, or iterative refinement can lead in both accuracy and speed on ETH3D and at 966×546. Specifically, it:
Apart from ours, every non-anonymous top-30 method on ETH3D and Middlebury relies on stereo-specific modules. CroCo-Stereo showed that cost-volume-free stereo is feasible, but it trails specialized methods substantially (ETH3D bad@1: 1.14 vs. 0.16). Without rectified disparity formulation, DA3 and VGGT do not reach stereo-level metric accuracy (Table 4; scale-aligned geometry is compared in Table 3). Because the architecture is generic, NBS directly leverages standard ViT optimizations such as FlashAttention and F16 precision, driving its runtime advantage.
Q2, zero-shot and baseline conditions. Table G1 in the general response reports quantitative zero-shot evaluations: neither NBS-Base nor NBS-HighRes ever saw ETH3D or Middlebury during training, whereas S2M2 and FS were trained on both training splits (per their authors). Even so, zero-shot NBS-Base outperforms both on ETH3D across all four metrics (EPE 0.138 vs. 0.190 and 0.171). Regarding our variants: NBS-ETH3D fine-tunes NBS-Base on FSD + ETH3D + Middlebury; NBS-HighRes adapts NBS-Base to higher resolution on the 14-dataset mixture in Section 3.4.2; and NBS-Middlebury fine-tunes NBS-HighRes with Middlebury added.
Q3, exposure and controlled factor ablations.
Q4a, does high-resolution PE help on real data? Yes—we added a zero-shot real-world PE ablation on ETH3D and Middlebury in Appendix A.5 (Table 10). The results confirm that high-resolution PEs consistently lower EPE and [email protected] across scales on real-world scenes.
Q4b, latency and memory. Additional efficiency analysis is provided in the revised Section 4.1.3, Table 2, and Figure 4, with 1932×1092 results in Appendix A.7 (Table 12). At 966×546, NBS is both the fastest and the most accurate method. At 1932×1092, NBS matches the latency of lightweight Fast-FoundationStereo (0.51 s vs. 0.50 s) with 3.8× lower peak memory (6.56 GB vs. 24.60 GB), while running 1.6× and 4.7× faster than S2M2 and FS and achieving the best accuracy.
Q5, Table 4. We thank the reviewer for catching this. DA3 and VGGT predict depth only up to an unknown global scale. For Table 4, we recover metric scale from the predicted two-view relative pose: we compute the predicted camera baseline ∥C0−C1∥ and rescale depth by Bgt/∥C0−C1∥ using the known rig baseline Bgt. Neither DA3-Large nor VGGT is given GT intrinsics: VGGT does not accept them, and DA3 uses intrinsics only together with input extrinsics (pose-conditioned mode), which we avoided because DA3's built-in Umeyama scale alignment is degenerate for two views (DA3 GitHub issue #241). DA3Metric, a separate metric-depth model, is converted via depth=f⋅d^/300 using the GT focal length f, as specified by its authors. While verifying this, we found a unit-conversion bug in our DA3-Large script (a spurious ×1000 applied after baseline rescaling), which caused the anomalous AbsRel values. The corrected AbsRel values are 0.886 / 2.609 / 1.342 (ETH3D / SimpleProc-S / XYZ-IBD), now updated in Table 4. The VGGT entries are correct; its SimpleProc-S AbsRel of 79.9 stems from near-zero predicted baselines. The remaining gap reflects the ill-conditioned scale estimate when inferring a narrow, nearly pure-translation stereo baseline from two close views, motivating direct disparity prediction under known rectification as in NBS.