acceptodds
Under review as a conference paper at ICLR 2027

Metric Calibration for Arbitrary Monocular Depth Estimation Models Using Ultra-Sparse Measurements

Abstract

Existing monocular depth estimation methods recovering dense 3D structure from a single RGB image has achieved remarkable success while still suffers from inherent absolute metric scale ambiguity. Sparse metric-guided methods address this by incorporating metric depth measurements. In practice, however, available measurements are often far sparser than required, especially at long range. We refer to this setting as depth estimation with ultra-sparse metric measurements (USMM). To address this problem, we propose AnchorDepth, a training-free post-processing framework that calibrates monocular depth predictions using USMM. It anchors the predicted depth map to the real-world metric scale without any learnable parameters. AnchorDepth identifies approximately fronto-parallel planes from the output of any pretrained monocular depth model and fits local metric corrections within them. It then propagates the remaining calibration residuals using either bounded inverse-squared-distance interpolation (AnchorDepth-Interp) or edge-aware global optimization (AnchorDepth-Refine). Extensive evaluations on public benchmarks showed that AnchorDepth reduced AbsRel by up to 76% compared with the corresponding uncalibrated monocular depth models. Furthermore, considering that existing benchmarks mainly focus on short-range scenes and provide limited evaluation of long-range depth, we built two benchmarks of kilometer-scale: synthetic STREET, and real-world FAR. On both benchmarks, AnchorDepth reduced AbsRel by approximately 40% compared with state-of-the-art metric-guided methods. These results demonstrated that our method outperformed both sparse metric-guided and RGB-only baselines in the USMM setting.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Monocular Metric Depth Estimation, Sparse Depth Guidance, Ultra-Sparse Depth Measurements.

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_gDtpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

AnchorDepth is a training-free, model-agnostic post-processing framework that calibrates the output of any frozen monocular depth estimator to metric scale using ultra-sparse metric measurements, typically no more than about 30 points per image. It extracts approximately fronto-parallel planes from the predicted depth map, fits an independent affine correction for each retained plane, and then propagates log-domain residuals to the remaining non-planar regions using either bounded inverse-squared-distance interpolation (AnchorDepth-Interp) or an edge-aware global refinement with hard measurement constraints (AnchorDepth-Refine). The method requires no retraining, no camera intrinsics, and no changes to the base model. The authors also introduce STREET and FAR, two long-range depth benchmarks reaching 2,000 m and about 730 m respectively. Under the same USMM setting, AnchorDepth improves more than ten existing benchmarks and reduces AbsRel by up to roughly 40% relative to state-of-the-art sparse metric-guided methods; its main limitation is that it depends on the base depth predictions and cannot correct substantial geometric errors.

Critical strengths
  1. AnchorDepth effectively addresses a practical and underexplored scenario: calibrating monocular depth models using ultra-sparse metric measurements (USMM) from low-cost rangefinders.

  2. The framework is training-free, model-agnostic, and requires no camera intrinsics, making it highly adaptable and plug-and-play for arbitrary frozen MDE models.

  3. The experimental validation is rigorous and comprehensive, covering diverse MDE backbones, multiple public benchmarks, ablation studies, and real-world on-device deployments

Critical weaknesses
  1. Although AnchorDepth addresses a practical problem, the paper suffers from excessive over-packaging and poor readability. While the underlying methodology relies on relatively straightforward engineering operations, the authors employ heavy jargon and verbose formulations that obscure the core intuition. Furthermore, critical implementation details are largely relegated to the appendix, rendering the main text disjointed and exhausting to follow.

  2. While AnchorDepth addresses a practical problem, its applicability is highly constrained. Its heavy reliance on fronto-parallel planes limits its effectiveness in unstructured or natural scenes. Furthermore, as a post-processing method, its performance is fundamentally capped by the base model and cannot correct severe geometric errors.

  3. Although this paper targets ultra-sparse scenarios, could the authors evaluate AnchorDepth on standard benchmarks (following protocols like LDCM or PriorDA) to reveal its performance ceiling?

  4. Since LDCM proposes a Poisson-based approach, how does AnchorDepth compare with it on both standard and ultra-sparse benchmarks, and what are their respective advantages?

  5. Is it feasible to integrate the Poisson-based formulation of LDCM with the proposed plane-wise calibration strategy?

Questions for authors

please see the weakness

6 comments

W1(Poorly Written Method Section):

A: We sincerely apologize for the confusion. We have resolved the issues above in the newly uploaded PDF and revised the Method section for better readability, including the relevant implementation details. We invite you to check the updated version.

W2(Reliance on Fronto-Parallel Planes May Limit Real-World Applicability, and Severe Geometric Errors in the Base Model’s Predictions Cannot Be Corrected):

A: AnchorDepth’s over-reliance on planar structures: AnchorDepth does rely on fronto-parallel planes. However, the detected planes are not limited to prominent fronto-parallel surfaces such as building façades. Small or slightly tilted planar regions can also provide useful references for correcting depth in neighboring regions, and such structures are considerably more common in real-world scenes than prominent fronto-parallel surfaces.

We evaluated AnchorDepth on more than ten widely used depth estimation datasets covering diverse practical settings, including both indoor and outdoor scenes. The results show clear improvements on every dataset. We additionally quantified correction performance in non-planar regions and compared AnchorDepth with the state-of-the-art methods InfiniDepth and LDCM. The results, reported below, show that AnchorDepth also offers a performance advantage over these methods in regions outside the fronto-parallel planes.

Table R6. Comparison with state-of-the-art methods on depth estimation performance in regions outside the fronto-parallel planes.

FARSTREET-daySTREET-darkKITTIETH3DNYUv2ScanNetDIODE
LDCM0.188/74.2280.038/97.8850.049/95.8910.158/79.8760.041/96.8850.043/96.7820.043/96.8290.057/93.701
InfiniDepth0.893/70.1740.084/87.6800.076/91.3710.052/96.4510.024/99.2480.035/98.0720.040/97.2750.064/95.517
AnchorDepth-Interp0.171/81.8210.034/98.1650.031/98.0020.049/96.6830.033/98.1440.034/98.0110.036/97.8670.058/94.213
AnchorDepth-Refine0.117/86.7630.026/98.8830.029/98.6490.058/95.9940.030/98.4430.035/97.7650.035/97.7420.047/95.751

Can AnchorDepth correct severe geometric errors? We constructed a dataset containing visual illusions of 3D structure to evaluate AnchorDepth when depth predictions exhibit substantial geometric errors. The results in Table R7 and Figure A3 in the revision show that AnchorDepth can partially correct these errors. However, it cannot recover near-correct geometry using only ultra-sparse ground-truth measurements. We acknowledge this limitation and have already discussed it in the Conclusion & Limitation section.

Table R7. Correction performance on the illusion dataset. Illusion denotes metrics computed only within the illusion regions, while Full denotes metrics computed across the entire image.

Absrelδ1RMSESiLog
MoGe2 Illusion0.51734.1030.9856.327
AnchorDepth-Interp Illusion0.04197.1230.1765.018
AnchorDepth-Refine Illusion0.03098.4100.1424.064
MoGe2 Full0.46633.9781.0379.569
AnchorDepth-Interp Full0.04596.3890.2346.833
AnchorDepth-Refine Full0.03298.2080.1955.411

W3(Evaluating AnchorDepth’s Performance Ceiling on Standard Protocols):

A: Following LDCM’s official settings, we evaluated AnchorDepth using five sparse ground-truth sampling schemes, SIFT, ORB, LiDAR16, LiDAR32, and LiDAR64, to investigate its performance ceiling. The detailed results are reported in Table R8. As expected, AnchorDepth achieves further improvements as more sparse depth measurements become available. For example, on ScanNet, AnchorDepth-Refine achieved an AbsRel of 0.010 with LiDAR64 input, compared with 0.035 using ultra-sparse priors, corresponding to a reduction of approximately 71%.

Table R8. Performance comparison between AnchorDepth and LDCM on LDCM’s standard protocols.

Street-daySTREET-darkKITTIETH3DNYUv2ScanNetDIODE
sift-LDCM0.016/99.0410.017/98.8880.052/94.7050.080/89.6720.015/99.2600.021/98.6320.045/95.997
sift-Interp0.017/99.1970.019/98.9690.039/97.9510.037/98.1760.026/98.7400.034/98.0820.083/93.303
sift-Refine0.015/99.1620.016/99.0960.046/97.1520.035/97.6930.021/98.7290.028/98.0450.063/95.142
orb-LDCM0.017/98.9970.018/98.8950.032/97.8990.040/97.0280.018/99.0770.028/98.1290.059/93.636
orb-Interp0.021/99.0120.023/98.8670.033/98.6120.031/98.8390.029/98.5850.040/97.6170.087/93.063
orb-Refine0.022/98.9730.023/98.9280.038/98.1300.025/98.7390.026/98.4920.036/97.2380.056/93.645
LiDAR16-LDCM0.019/98.6870.020/98.5560.050/95.8020.041/95.9240.011/99.4810.010/99.3760.019/97.865
LiDAR16-Interp0.017/98.9500.018/98.8080.039/97.7230.023/99.0030.017/99.2210.018/98.9760.031/97.387
LiDAR16-Refine0.016/98.9640.017/98.9800.053/96.6960.022/99.1360.015/99.2490.016/98.9160.023/97.502
LiDAR32-LDCM0.017/98.9140.018/98.8450.079/90.9330.027/97.7640.008/99.6250.007/99.5560.018/97.974
LiDAR32-Interp0.017/98.9170.017/98.9950.033/98.0490.020/99.0460.015/99.3520.016/99.1420.028/97.673
LiDAR32-Refine0.015/99.0750.014/99.1320.042/97.4640.020/99.0600.012/99.3830.013/99.1330.020/97.826
LiDAR64-LDCM0.015/99.0810.015/99.0770.038/96.8150.015/99.1600.007/99.7180.006/99.6780.018/97.906
LiDAR64-Interp0.015/99.1440.016/99.1170.028/98.4940.017/99.3390.014/99.4510.015/99.2690.027/97.902
LiDAR64-Refine0.012/99.2090.012/99.2190.035/98.0360.015/99.4250.010/99.5220.010/99.3210.018/98.085

W4(Comparing AnchorDepth and LDCM under Standard and Ultra-Sparse Settings: Performance and Respective Strengths):

A: The main difference between LDCM and AnchorDepth lies in how they perform depth correction. LDCM reconstructs depth globally and uses a neural network to produce refined predictions. In contrast, AnchorDepth treats fronto-parallel planes and the remaining regions separately. It independently applies an affine correction to each plane, then corrects the remaining regions using the remaining sparse ground-truth measurements, either directly or together with planar priors.

Specifically, LDCM uses sparse anchors to generate an initial coarse depth map through Poisson reconstruction. This coarse depth map then serves as a prompt and is fed alongside the RGB image into the depth completion network for depth (point cloud) prediction. This design allows LDCM to better exploit denser, uniformly distributed LiDAR priors. AnchorDepth, by contrast, extracts approximately fronto-parallel planar structures from the initial depth predictions and independently corrects them through affine transformations. It then corrects depth residuals in the remaining regions using the remaining ground-truth measurements or incorporating the corrected planes as priors. AnchorDepth therefore makes more effective use of spatial structure without requiring more complex sparse depth priors.

In W3, we evaluated AnchorDepth in standard scenarios and compared it with LDCM. With relatively dense sparse ground-truth inputs, each method outperformed the other on different datasets. LDCM performed better on indoor datasets, particularly NYUv2 and ScanNet, where it achieved AbsRel values below 0.01. However, AnchorDepth retained an advantage in outdoor evaluations, especially at long ranges. We conducted long-range evaluations on Street-day and Street-dark. On Street-day, AnchorDepth-Refine reduced AbsRel by approximately 20% relative to LDCM.

For evaluations under ultra-sparse conditions, please refer to Table 1 in the original manuscript. In this setting, LDCM’s performance is limited by insufficient sparse depth guidance, and AnchorDepth consistently outperforms LDCM.

Reviewer_pFvsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses metric depth calibration from ultra-sparse measurements with a training-free, model-agnostic framework. The problem is practical, and the proposed approach is simple and potentially useful for real-world deployment. The experimental evaluation is extensive, covering multiple monocular depth models, public benchmarks, long-range scenes, and real-world measurements, with promising improvements over existing methods. The main weakness is that the core algorithm builds on established techniques rather than offering a groundbreaking innovation, which limits its technical novelty but not its practical utility. The authors' questions are well-posed and could strengthen the evaluation: evaluating AnchorDepth under random measurement locations with a matched per-image budget (Table 1), isolating the effect of plane-extraction errors on calibration accuracy (Table 3), explaining the Interp-vs-Refine discrepancy across datasets (Tables 1–2), and separating calibration inputs from evaluation ground truth to enable matched-budget comparisons (Table 4). Overall, the paper offers a simple, well-evaluated, and practically relevant contribution; I lean toward weak acceptance.

Critical strengths
  1. The paper addresses a practical problem of metric depth calibration using ultra-sparse measurements. The proposed training-free and model-agnostic framework is simple and potentially useful for real-world applications.
  2. The experimental evaluation is extensive, covering multiple monocular depth models, public benchmarks, long-range scenes, and real-world measurements. The results demonstrate promising improvements over existing methods.
Critical weaknesses

1.The core algorithm is primarily based on established techniques rather than a groundbreaking innovation.

Questions for authors
  1. In Table 1, Competing methods are evaluated with both random and structured measurement locations, whereas AnchorDepth is reported only with structured locations. Could the authors evaluate AnchorDepth with random locations under the same per-image measurement budget?
  2. Table 3 examines different backbone sizes but does not isolate plane extraction errors. Could the authors quantify how false or missed plane detections affect final calibration accuracy?
  3. In Tables 1 and 2,AnchorDepth-Interp performs better than AnchorDepth-Refine on KITTI, while AnchorDepth-Refine performs better on several other datasets. Could the authors explain which scene or measurement characteristics account for this difference?
  4. Table 4 reports substantially different ground truth counts per frame across methods. Could the authors distinguish measurements used for calibration from ground truth used for evaluation and, if feasible, compare methods under matched calibration-input budgets?
4 comments

W1(Lack of Groundbreaking Innovation):

A: We appreciate the opportunity to clarify this point. We believe that AnchorDepth’s core insight lies in identifying the distinctive behavior of planar structures during calibration and using this insight to develop an effective calibration method. This is the theoretical contribution we wish to emphasize. Furthermore, AnchorDepth is specifically designed for ultra-sparse measurements and can correct depth maps using only a handful of ground-truth depth measurements. To the best of our knowledge, existing depth calibration methods have not specifically addressed how to effectively handle this degree of sparsity.

Q1(Evaluating AnchorDepth with Random Point Sampling):

A: We conducted experiments with random measurement locations, with the results reported in Table R3. However, AnchorDepth’s calibration algorithm is designed around our measurement-selection strategy. Fully random sampling therefore cannot fully reflect AnchorDepth’s performance, as it does not guarantee that each detected plane contains sufficient ground-truth depth measurements for affine calibration.

Table R3. AnchorDepth performance with random measurement sampling. Following the terminology in Table 1 in the manuscript, 'Structured' denotes measurement selection according to AnchorDepth’s intended sampling strategy. 'Random' denotes randomly selecting the same number of measurements as 'Structured' from all valid ground-truth depth locations across the image for calibration.

FARSTREET-daySTREET-darkKITTIETH3DNYUv2ScanNetDIODE
AnchorDepth-Interp0.164/82.5390.034/98.1470.031/98.0480.047/96.7810.025/98.9130.032/98.1920.035/98.0080.051/95.083
AnchorDepth-Interp Random0.182/79.0030.039/97.7830.045/96.5410.093/96.8490.037/99.0250.040/97.9900.045/97.4130.070/93.316
AnchorDepth-Refine0.115/87.0230.027/98.8360.029/98.6010.057/96.0870.026/98.8970.034/97.9480.035/97.8660.043/96.415
AnchorDepth-Refine Random0.225/79.6230.046/97.9630.038/97.0270.062/95.3950.035/98.3020.041/97.7510.046/97.0200.058/94.185

Q2(Quantifying the Impact of Incorrect or Missed Plane Detections on Calibration Accuracy):

A: We used FAR, the real-world long-range dataset introduced in our paper, to simulate imperfect plane detection. Specifically, we simulated missed detections by discarding each successfully detected plane instance with a specified probability. The results in Table R4 show that AnchorDepth experiences some performance degradation when planes are missed, with further degradation as the probability of missed detections increases.

Table R4. Impact of missed plane detections on AnchorDepth’s performance. Missed percentage denotes the probability of discarding each detected plane instance. Discarded instances are subsequently processed as non-fronto-parallel regions.

missed percentageInterp AbsRelInterp δ1Refine AbsRelRefine δ1
0%0.16482.5390.11587.023
25%0.17181.1390.14684.756
50%0.17879.6910.17682.389
75%0.18478.1940.20679.898
100%0.19176.6470.23477.261

Q3(Which Scene or Measurement Characteristics Explain the Performance Differences Between the Two AnchorDepth Variants):

A: The main method's difference between AnchorDepth-Interp and AnchorDepth-Refine lies in how they handle regions outside the fronto-parallel planes. AnchorDepth-Interp uses sparse depth priors within these regions to perform distance-based residual interpolation. AnchorDepth-Refine instead treats the corrected planes as known metric priors and performs boundary-aware Poisson propagation into the remaining regions. Refine therefore makes better use of the corrected planes as references: detecting more planar structures or achieving greater planar coverage provides Refine with more reference information, increasing its advantage over Interp. Conversely, fewer detected planes or lower planar coverage reduce this advantage.

We analyzed the fraction of pixels covered by detected fronto-parallel planes on KITTI and FAR, where AnchorDepth-Interp and AnchorDepth-Refine exhibit their largest respective advantages. Detected fronto-parallel planes accounted for only 2.532% of all pixels on KITTI, compared with 11.297% on FAR.

Q4(Distinguishing Calibration and Evaluation Ground Truth and Comparing Methods under the Same Calibration Input Budget):

A: We would like to emphasize that our evaluation directly measures relative ranging error using MAPE, rather than conventional depth estimation metrics such as AbsRel. For example, a relative error of 0.2 corresponds to estimating the distance between two objects as 8 m or 12 m when the true distance is 10 m. All ground-truth distances or dimensions were obtained manually in 3D space, rather than computed from LiDAR outputs. The measurements used for calibration and the ground truth used for evaluation are therefore entirely disjoint.

We evaluated two baseline methods using exactly the same sparse measurement budget as AnchorDepth. We evaluated three types of protocols: object-to-camera distance, inter-object distance, and object dimensions. The results are reported below. AnchorDepth substantially outperforms the baseline methods under the same sparse measurement budget.

Table R5. Real-world evaluation under the same measurement budget. All baseline methods used the same ultra-sparse calibration input budget as AnchorDepth.

SceneMethodRelative Error(MAPE)
OutdoorPriorDA0.150
OutdoorInfiniDepth-DepthSensor0.112
OutdoorAnchorDepth0.012
IndoorPriorDA0.103
IndoorInfiniDepth-DepthSensor0.114
IndoorAnchorDepth0.030
Reviewer_pFvsOpenReview ↗

Reply

Thanks for the author's reply. I have checked the response and maintain the weak acceptance.

We sincerely appreciate your thoughtful feedback and are pleased that our responses have fully addressed your concerns.

Reviewer_Y2xsOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AnchorDepth, which uses a small number of metric depth measurements to correct the predictions of a pretrained monocular depth model. It first identifies roughly fronto-parallel regions and fits local affine corrections. The remaining areas are then adjusted using either residual interpolation or edge-aware global optimization. The method requires no retraining and can be applied to different depth models. The authors evaluate it on several public benchmarks, along with their new STREET and FAR datasets. They also include experiments with real sensors and UAV images. The reported results show substantial reductions in AbsRel, especially when the original depth predictions have large scale errors.

Critical strengths
  1. The method works with different metric and relative depth models without retraining. This makes it relatively simple to apply and potentially useful in practice. It also avoids the cost of fine-tuning a separate model for each dataset or scene.

  2. The experiments cover a wide range of datasets, including indoor, outdoor, and long-range scenes. The authors also test different model sizes and simulated measurement noise, which helps assess the method's effectiveness and robustness. The results show consistent improvements in AbsRel across many of the evaluated settings.

Critical weaknesses
  1. The ablation study is not convincing enough. It is unclear how much of the improvement comes from the measurement selection strategy and how much comes from the calibration method itself. The paper does not directly evaluate AnchorDepth with randomly selected measurements under the same budget. This makes it difficult to judge the contribution of each part of the method.

  2. The method relies heavily on the quality of the initial depth predictions. When the depth boundaries or local geometry are incorrect, it is unclear whether AnchorDepth can still improve the results reliably. The paper lacks sufficient experiments on these failure cases, which makes it difficult to assess the method's robustness.

Questions for authors
  1. Can the authors evaluate AnchorDepth with randomly selected measurement points under the same budget? This would help clarify how much of the improvement comes from point selection and how much comes from the calibration itself.

  2. How does AnchorDepth perform when the initial depth predictions have incorrect boundaries or local geometry? Could the authors provide more experiments or failure cases to show how robust the method is in these situations?

4 comments

W1&Q1(Disentangling the Contributions of Measurement Selection Strategy and Calibration Method to Performance Gains):

A: Thank you for your question. We would like to clarify that AnchorDepth comprises a measurement selection strategy and a calibration method designed around that strategy. These two components therefore cannot simply be treated as independent. AnchorDepth performs affine correction independently for each plane, requiring at least two ground-truth depth measurements on that plane. Random sampling does not guarantee that this requirement is met. Exsting callbration methods primarily use sparse depth priors to estimate a global affine transformation. In particular in Table 1 in the manuscript, we compared AnchorDepth with a baseline that fits an affine transformation using RANSAC, and the results show that AnchorDepth achieved superior performance.

We conducted additional calibration experiments with random measurement as you suggested, with the results reported in Table R1. However, these results cannot fully isolate the contribution of the measurement selection strategy to the performance gains. Random sampling fails to provide valid calibration points for individual planes, and AnchorDepth cannot independently calibrate planes with insufficient ground-truth measurements. The resulting evaluation therefore does not faithfully reflect AnchorDepth’s performance.

Table R1. AnchorDepth performance with random measurement sampling. Following the terminology in Table 1 in the manuscript, 'Structured' denotes measurement selection according to AnchorDepth’s intended sampling strategy. 'Random' denotes randomly selecting the same number of measurements as 'Structured' from all valid ground-truth depth locations across the image for calibration.

FARSTREET-daySTREET-darkKITTIETH3DNYUv2ScanNetDIODE
AnchorDepth-Interp Structured0.164/82.5390.034/98.1470.031/98.0480.047/96.7810.025/98.9130.032/98.1920.035/98.0080.051/95.083
AnchorDepth-Interp Random0.182/79.0030.039/97.7830.045/96.5410.093/96.8490.037/99.0250.040/97.9900.045/97.4130.070/93.316
AnchorDepth-Refine Structured0.115/87.0230.027/98.8360.029/98.6010.057/96.0870.026/98.8970.034/97.9480.035/97.8660.043/96.415
AnchorDepth-Refine Random0.225/79.6230.046/97.9630.038/97.0270.062/95.3950.035/98.3020.041/97.7510.046/97.0200.058/94.185

W2&Q2(Addressing Potential Errors in Depth Boundaries or Local Geometry in Initial Depth Predictions):

A: We would first like to clarify that AnchorDepth can improve the geometric accuracy of depth predictions. Tables A6–A13 in the manuscript report SiLog, which measures the accuracy of the relative geometry in depth predictions [1]. AnchorDepth reduces the SiLog error of depth estimation models on almost all datasets, indicating improvements in the predicted geometry. We believe these gains arise from anchor-guided residual corrections in regions with geometric errors.

To specifically investigate this behavior, we used several open-source 3D scenes and applied photographs with complex textures or images depicting 3D illusions to planar surfaces, causing depth estimation models to incorrectly predict nonplanar structures in these regions. This subset contains 276 images in total. We used these data to simulate geometric errors in depth predictions and evaluated performance both within the illusion regions (Illusion) and across the full image (Full), as reported in Table R2. The performance gains within the illusion regions are greater than those across the full image.

Table R2. Correction performance on the illusion dataset. Illusion denotes metrics computed only within the illusion regions, while Full denotes metrics computed across the entire image.

Absrelδ1RMSESiLog
MoGe2 Illusion0.51734.1030.9856.327
AnchorDepth-Interp Illusion0.04197.1230.1765.018
AnchorDepth-Refine Illusion0.03098.4100.1424.064
MoGe2 Full0.46633.9781.0379.569
AnchorDepth-Interp Full0.04596.3890.2346.833
AnchorDepth-Refine Full0.03298.2080.1955.411

We hypothesize that the larger performance gains within the illusion regions stem from larger geometric errors in the original predictions. These errors produce larger residuals at anchors within the illusion regions, leading to stronger corrections in nearby regions during residual propagation.

We further visualized the errors in the point clouds obtained by back-projecting the depth maps, as shown in Figure A3 in the revision. The leftmost column shows the RGB images provided to the depth estimation model. We marked the illusion regions with yellow boxes for visualization; these boxes were not included in the model inputs. The remaining three columns show the point cloud errors for MoGe-2, AnchorDepth-Interp, and AnchorDepth-Refine, respectively. Both AnchorDepth-Interp and AnchorDepth-Refine clearly reduce geometric errors. In particular, AnchorDepth substantially reduces extreme errors. These results further support AnchorDepth’s ability to improve geometric accuracy to some extent.

Nevertheless, the Figure A3 also shows that the residual errors are not spatially smooth: using only a few anchors, AnchorDepth cannot fully restore an incorrectly predicted nonplanar structure to the correct planar geometry. We acknowledge this limitation and have identified it as a direction for future research. We have already discussed it in detail in the Conclusion & Limitation section.

[1] Eigen D, Puhrsch C, Fergus R. Depth map prediction from a single image using a multi-scale deep network[J]. Advances in neural information processing systems, 2014, 27.

Reviewer_Y2xsOpenReview ↗

Comments from Reviewer Y2xs

Thanks for the detailed responses. The authors have addressed my concerns.

Thank you for confirming that all your concerns have been addressed and for your careful evaluation of our work.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear Area Chair,

We sincerely thank Reviewers Y2xs, pFvs, and gDtp for their constructive feedback and for recognizing the practical value and SOTA performance of our work. We are encouraged by their recognition that AnchorDepth addresses a practical yet underexplored problem: calibrating monocular depth estimation models using ultra-sparse metric measurements (Reviewer gDtp). The reviewers also acknowledged its practical utility and adaptability (Reviewers Y2xs, pFvs, and gDtp), as well as the extensive experimental evidence demonstrating its advantages over existing methods (Reviewers Y2xs, pFvs, and gDtp).

During the rebuttal period, we conducted extensive additional experiments to address the reviewers' concerns, focusing on performance with randomly selected measurements (Reviewers Y2xs and pFvs), robustness to substantial geometric errors in the initial depth predictions (Reviewers Y2xs, pFvs, and gDtp), dependence on planar regions (Reviewers pFvs and gDtp), evaluation under standard testing protocols (Reviewer gDtp), and concerns regarding groundbreaking novelty (Reviewer pFvs). We summarize the main updates below.

1. Performance with Randomly Selected Measurements (Reviewers Y2xs and pFvs)

The reviewers asked about AnchorDepth's performance with randomly selected measurement locations. Reviewer Y2xs also requested an analysis of the respective contributions of the measurement-selection strategy and the calibration algorithm. We have provided additional analyses and experiments to address these concerns.

  • Contributions of the two components. We would like to clarify that AnchorDepth comprises a measurement-selection strategy and a calibration algorithm designed around it. Under a limited measurement budget, measurement locations are crucial, making the selection strategy integral to AnchorDepth. These two components are therefore closely coupled and cannot simply be treated as independent.
  • Evaluation with random measurements. With fully random measurement locations, AbsRel for both AnchorDepth variants typically increased by approximately 0.01 (up to 0.05) on public datasets, highlighting the benefits of our jointly designed measurement selection strategy and calibration algorithm.

2. Robustness to Substantial Geometric Errors in the Initial Depth Predictions (Reviewers Y2xs, pFvs, and gDtp)

AnchorDepth can improve geometry beyond scale alignment using ultra-sparse depth priors, reducing SiLog, a measure of geometric accuracy, for nearly all evaluated models on their respective test datasets.

To assess robustness to initial geometric errors, we constructed a 276-image dataset by adding visual illusions to open-source scenes, where depth models often misinterpret planar surfaces as 3D structures. AnchorDepth reduces AbsRel within illusion regions by up to 94% relative to initial predictions and substantially lowers SiLog. Qualitative results further show partial correction of these geometric errors, supporting its robustness.

3. Dependence on Planar Regions (Reviewers pFvs and gDtp)

Reviewer gDtp questioned its effectiveness in unstructured scenes, while Reviewer pFvs asked how erroneous or missed plane detections affect AnchorDepth's performance. We conducted additional experiments to investigate these concerns.

  • Effectiveness in unstructured scenes. We evaluated AnchorDepth in regions outside the detected planes and compared its performance with that of SOTA models. AnchorDepth continued to outperform these models in these regions, supporting its effectiveness in handling non-planar regions.
  • Sensitivity to missed plane detections. We independently discarded each detected plane with a specified probability. Performance degraded as the discard probability increased, reaching its lowest level when all detected planes in an image were discarded.

4. Evaluation under Standard Testing Protocols (Reviewer gDtp)

Following LDCM’s standard testing protocols, we show that AnchorDepth achieves further performance gains with additional sparse measurements, demonstrating its applicability beyond the ultra-sparse setting and supporting its practical utility.

5. Clarification of Groundbreaking Novelty (Reviewer pFvs)

AnchorDepth’s core theoretical contribution lies in exploiting the distinctive calibration behavior of planar structures to design both a measurement selection strategy and a calibration algorithm. Designed for ultra-sparse measurements, it calibrates depth maps using only a few ground-truth depth measurements, addressing a setting underexplored by existing methods.

We believe that the additional experimental evidence, particularly the analyses of robustness to geometric errors and performance in non-planar regions, further strengthens the contributions of our work. We hope that the revised manuscript meets the high standards of ICLR.

Sincerely,

The Authors of Paper 27722

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.