- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
AnchorDepth is a training-free, model-agnostic post-processing framework that calibrates the output of any frozen monocular depth estimator to metric scale using ultra-sparse metric measurements, typically no more than about 30 points per image. It extracts approximately fronto-parallel planes from the predicted depth map, fits an independent affine correction for each retained plane, and then propagates log-domain residuals to the remaining non-planar regions using either bounded inverse-squared-distance interpolation (AnchorDepth-Interp) or an edge-aware global refinement with hard measurement constraints (AnchorDepth-Refine). The method requires no retraining, no camera intrinsics, and no changes to the base model. The authors also introduce STREET and FAR, two long-range depth benchmarks reaching 2,000 m and about 730 m respectively. Under the same USMM setting, AnchorDepth improves more than ten existing benchmarks and reduces AbsRel by up to roughly 40% relative to state-of-the-art sparse metric-guided methods; its main limitation is that it depends on the base depth predictions and cannot correct substantial geometric errors.
Critical strengths
-
AnchorDepth effectively addresses a practical and underexplored scenario: calibrating monocular depth models using ultra-sparse metric measurements (USMM) from low-cost rangefinders.
-
The framework is training-free, model-agnostic, and requires no camera intrinsics, making it highly adaptable and plug-and-play for arbitrary frozen MDE models.
-
The experimental validation is rigorous and comprehensive, covering diverse MDE backbones, multiple public benchmarks, ablation studies, and real-world on-device deployments
Critical weaknesses
-
Although AnchorDepth addresses a practical problem, the paper suffers from excessive over-packaging and poor readability. While the underlying methodology relies on relatively straightforward engineering operations, the authors employ heavy jargon and verbose formulations that obscure the core intuition. Furthermore, critical implementation details are largely relegated to the appendix, rendering the main text disjointed and exhausting to follow.
-
While AnchorDepth addresses a practical problem, its applicability is highly constrained. Its heavy reliance on fronto-parallel planes limits its effectiveness in unstructured or natural scenes. Furthermore, as a post-processing method, its performance is fundamentally capped by the base model and cannot correct severe geometric errors.
-
Although this paper targets ultra-sparse scenarios, could the authors evaluate AnchorDepth on standard benchmarks (following protocols like LDCM or PriorDA) to reveal its performance ceiling?
-
Since LDCM proposes a Poisson-based approach, how does AnchorDepth compare with it on both standard and ultra-sparse benchmarks, and what are their respective advantages?
-
Is it feasible to integrate the Poisson-based formulation of LDCM with the proposed plane-wise calibration strategy?
Questions for authors
please see the weakness
6 comments
W3(Evaluating AnchorDepth’s Performance Ceiling on Standard Protocols):
A: Following LDCM’s official settings, we evaluated AnchorDepth using five sparse ground-truth sampling schemes, SIFT, ORB, LiDAR16, LiDAR32, and LiDAR64, to investigate its performance ceiling. The detailed results are reported in Table R8. As expected, AnchorDepth achieves further improvements as more sparse depth measurements become available. For example, on ScanNet, AnchorDepth-Refine achieved an AbsRel of 0.010 with LiDAR64 input, compared with 0.035 using ultra-sparse priors, corresponding to a reduction of approximately 71%.
Table R8. Performance comparison between AnchorDepth and LDCM on LDCM’s standard protocols.
| Street-day | STREET-dark | KITTI | ETH3D | NYUv2 | ScanNet | DIODE | |
|---|---|---|---|---|---|---|---|
| sift-LDCM | 0.016/99.041 | 0.017/98.888 | 0.052/94.705 | 0.080/89.672 | 0.015/99.260 | 0.021/98.632 | 0.045/95.997 |
| sift-Interp | 0.017/99.197 | 0.019/98.969 | 0.039/97.951 | 0.037/98.176 | 0.026/98.740 | 0.034/98.082 | 0.083/93.303 |
| sift-Refine | 0.015/99.162 | 0.016/99.096 | 0.046/97.152 | 0.035/97.693 | 0.021/98.729 | 0.028/98.045 | 0.063/95.142 |
| orb-LDCM | 0.017/98.997 | 0.018/98.895 | 0.032/97.899 | 0.040/97.028 | 0.018/99.077 | 0.028/98.129 | 0.059/93.636 |
| orb-Interp | 0.021/99.012 | 0.023/98.867 | 0.033/98.612 | 0.031/98.839 | 0.029/98.585 | 0.040/97.617 | 0.087/93.063 |
| orb-Refine | 0.022/98.973 | 0.023/98.928 | 0.038/98.130 | 0.025/98.739 | 0.026/98.492 | 0.036/97.238 | 0.056/93.645 |
| LiDAR16-LDCM | 0.019/98.687 | 0.020/98.556 | 0.050/95.802 | 0.041/95.924 | 0.011/99.481 | 0.010/99.376 | 0.019/97.865 |
| LiDAR16-Interp | 0.017/98.950 | 0.018/98.808 | 0.039/97.723 | 0.023/99.003 | 0.017/99.221 | 0.018/98.976 | 0.031/97.387 |
| LiDAR16-Refine | 0.016/98.964 | 0.017/98.980 | 0.053/96.696 | 0.022/99.136 | 0.015/99.249 | 0.016/98.916 | 0.023/97.502 |
| LiDAR32-LDCM | 0.017/98.914 | 0.018/98.845 | 0.079/90.933 | 0.027/97.764 | 0.008/99.625 | 0.007/99.556 | 0.018/97.974 |
| LiDAR32-Interp | 0.017/98.917 | 0.017/98.995 | 0.033/98.049 | 0.020/99.046 | 0.015/99.352 | 0.016/99.142 | 0.028/97.673 |
| LiDAR32-Refine | 0.015/99.075 | 0.014/99.132 | 0.042/97.464 | 0.020/99.060 | 0.012/99.383 | 0.013/99.133 | 0.020/97.826 |
| LiDAR64-LDCM | 0.015/99.081 | 0.015/99.077 | 0.038/96.815 | 0.015/99.160 | 0.007/99.718 | 0.006/99.678 | 0.018/97.906 |
| LiDAR64-Interp | 0.015/99.144 | 0.016/99.117 | 0.028/98.494 | 0.017/99.339 | 0.014/99.451 | 0.015/99.269 | 0.027/97.902 |
| LiDAR64-Refine | 0.012/99.209 | 0.012/99.219 | 0.035/98.036 | 0.015/99.425 | 0.010/99.522 | 0.010/99.321 | 0.018/98.085 |
W4(Comparing AnchorDepth and LDCM under Standard and Ultra-Sparse Settings: Performance and Respective Strengths):
A: The main difference between LDCM and AnchorDepth lies in how they perform depth correction. LDCM reconstructs depth globally and uses a neural network to produce refined predictions. In contrast, AnchorDepth treats fronto-parallel planes and the remaining regions separately. It independently applies an affine correction to each plane, then corrects the remaining regions using the remaining sparse ground-truth measurements, either directly or together with planar priors.
Specifically, LDCM uses sparse anchors to generate an initial coarse depth map through Poisson reconstruction. This coarse depth map then serves as a prompt and is fed alongside the RGB image into the depth completion network for depth (point cloud) prediction. This design allows LDCM to better exploit denser, uniformly distributed LiDAR priors. AnchorDepth, by contrast, extracts approximately fronto-parallel planar structures from the initial depth predictions and independently corrects them through affine transformations. It then corrects depth residuals in the remaining regions using the remaining ground-truth measurements or incorporating the corrected planes as priors. AnchorDepth therefore makes more effective use of spatial structure without requiring more complex sparse depth priors.
In W3, we evaluated AnchorDepth in standard scenarios and compared it with LDCM. With relatively dense sparse ground-truth inputs, each method outperformed the other on different datasets. LDCM performed better on indoor datasets, particularly NYUv2 and ScanNet, where it achieved AbsRel values below 0.01. However, AnchorDepth retained an advantage in outdoor evaluations, especially at long ranges. We conducted long-range evaluations on Street-day and Street-dark. On Street-day, AnchorDepth-Refine reduced AbsRel by approximately 20% relative to LDCM.
For evaluations under ultra-sparse conditions, please refer to Table 1 in the original manuscript. In this setting, LDCM’s performance is limited by insufficient sparse depth guidance, and AnchorDepth consistently outperforms LDCM.
W1(Poorly Written Method Section):
A: We sincerely apologize for the confusion. We have resolved the issues above in the newly uploaded PDF and revised the Method section for better readability, including the relevant implementation details. We invite you to check the updated version.
W2(Reliance on Fronto-Parallel Planes May Limit Real-World Applicability, and Severe Geometric Errors in the Base Model’s Predictions Cannot Be Corrected):
A: AnchorDepth’s over-reliance on planar structures: AnchorDepth does rely on fronto-parallel planes. However, the detected planes are not limited to prominent fronto-parallel surfaces such as building façades. Small or slightly tilted planar regions can also provide useful references for correcting depth in neighboring regions, and such structures are considerably more common in real-world scenes than prominent fronto-parallel surfaces.
We evaluated AnchorDepth on more than ten widely used depth estimation datasets covering diverse practical settings, including both indoor and outdoor scenes. The results show clear improvements on every dataset. We additionally quantified correction performance in non-planar regions and compared AnchorDepth with the state-of-the-art methods InfiniDepth and LDCM. The results, reported below, show that AnchorDepth also offers a performance advantage over these methods in regions outside the fronto-parallel planes.
Can AnchorDepth correct severe geometric errors? We constructed a dataset containing visual illusions of 3D structure to evaluate AnchorDepth when depth predictions exhibit substantial geometric errors. The results in Table R7 and Figure A3 in the revision show that AnchorDepth can partially correct these errors. However, it cannot recover near-correct geometry using only ultra-sparse ground-truth measurements. We acknowledge this limitation and have already discussed it in the Conclusion & Limitation section.