- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes D3SS, a depth recovery framework designed to improve the generalization of depth refinement across diverse 3D scene scales and degradation patterns.
The method consists of three main components. First, a diffusion-based raw-depth simulation pipeline generates distorted depth maps by varying diffusion sampling steps, initial noise, and interpolation with ground-truth depth. Second, the framework incorporates predictions from a frozen Depth Anything V2 model as additional depth priors. Third, the authors introduce novel-view consistency losses, including a direct constraint against ground-truth geometry and a reliability-aware supervision strategy that encourages agreement with trustworthy foundation-model predictions while discouraging agreement in unreliable regions.
Experiments on Middlebury 2014, NYUv2, KITTI, and RGBDD demonstrate improvements over several depth recovery and super-resolution baselines. Additional experiments investigate different data simulation strategies, foundation-model integration, and geometric supervision.
Critical strengths
- The paper addresses a practically relevant problem: improving depth recovery under diverse scales, local distortions, and structural errors.
- The method substantially improves depth maps from MiDaS, LeReS, and Marigold across multiple benchmarks, with smaller additional gains for stronger models such as MoGe and MoGe-2.
- Combining diffusion-based degradation synthesis, a frozen depth foundation model, and alternative-view geometric supervision is a promising strategy for reducing dependence on paired training data.
- Ablations cover simulation components, foundation-model integration, geometric losses, and architecture choices, while RGBDD experiments extend evaluation to real-world depth inputs.
Critical weaknesses
-
The theoretical justification for diffusion-based scale diversification contains substantial gaps. Solver convergence does not imply monotonic convergence to metric depth, and the Lyapunov/Jacobian argument does not establish amplification of arbitrary local perturbations. Moreover, the ground-truth scale-and-shift alignment in Equation (2) may remove the intended global variations; empirical measurements of the final distortion distributions are needed.
-
The novel-view geometric loss lacks a precise mathematical and implementation description. Algorithm 1 involves depth sorting, discrete indexing, and pixel assignment without clearly explaining gradient propagation, while the virtual-camera construction is ambiguous. The authors should also justify why Equation (7) maximizes disagreement with unreliable DAV2 predictions instead of masking or downweighting them.
-
The contributions of diffusion simulation and the frozen DAV2 prior are insufficiently disentangled. In Table 4, interpolation alone matches the full simulation strategy on Middlebury AbsRel (0.038), while Table 5 suggests that test-time DAV2 conditioning already approaches the full method. Comparisons with simple interpolation, affine perturbations, and spatially varying noise under matched settings are needed.
-
Existing comparisons do not isolate the proposed method’s benefits because it jointly changes the inputs, architecture, and training procedure. A matched ConvNeXt baseline using the same RGB, raw depth, and frozen DAV2 inputs with standard regression and simple augmentation would better assess the value of the proposed simulation and geometric supervision. Scale-and-shift alignment protocols should also be clarified.
-
Real-world generalization is less convincing than synthetic performance. On RGBDD, the method matches SGN in AbsRel (0.021) and has slightly worse RMSE than the best baseline (0.060 versus 0.059), while the heavy-misalignment results are inconsistent across metrics. Evaluation on additional sensors and controlled real-world degradation types is needed.
-
Statistical analysis and related-work positioning are incomplete. A fixed seed does not establish robustness across training initializations or generated degradations, so multi-seed results and uncertainty estimates are needed. The discussion should include DORNet (CVPR 2025), DEPTHOR (ICCV 2025), and Distilling Monocular Foundation Model for Fine-grained Depth Completion (CVPR 2025).
Questions for authors
-
Could the authors empirically verify that varying diffusion steps systematically changes global depth scale, particularly after the ground-truth scale-and-shift alignment in Equation (2)? The monotonic scale-convergence and noise-amplification claims in Appendix A.1 should also be reconsidered.
-
What differentiable rasterization or point-splatting implementation is used for novel-view consistency, and how do gradients propagate through pixel assignment, depth sorting, and occlusion in Algorithm 1? Please also clarify the construction of camera translation and the orthonormal rotation basis.
-
Why does Equation (7) maximize disagreement with unreliable foundation-model predictions instead of masking or downweighting them? An ablation against confidence-weighted and masked supervision should assess geometry quality and training stability.
-
How does diffusion-based simulation compare with affine depth transformations, spatially correlated noise, and interpolation under identical training budgets? Please also quantify how closely the simulated degradation distributions match real raw-depth inputs.
-
Could the authors provide a controlled ConvNeXt baseline using identical RGB, raw-depth, and frozen DAV2 inputs with standard depth regression? This would isolate the gains from diffusion augmentation and novel-view consistency.
-
Could the method be evaluated on additional real-world sensors and degradation types? Multi-seed results and confidence intervals are also needed, particularly for RGBDD and comparisons with strong depth foundation models.
-
How are scale and shift ambiguities handled for MiDaS, LeReS, Marigold, and DAV2, and is ground-truth-dependent alignment used at test time? Please also report end-to-end latency and memory, including both frozen DAV2 inference and the recovery network.