acceptodds
Under review as a conference paper at ICLR 2027

Learning to Trust Sparse Metric Evidence for Monocular Depth Foundation Models

Abstract

Monocular depth foundation models provide strong dense geometric priors, while their predictions can still retain errors in metric scale and local depth. We study how little external metric evidence is needed to ground such priors, focusing on the extreme few-point regime where only one to a handful of sparse metric observations are available. We find that a few reliable observations can already yield substantial metric gains, but that this regime is inherently fragile: when evidence is scarce, each observation carries disproportionate influence, so unreliable measurements can erase the benefit of sparse grounding. We introduce Reliable Few-Point Metric Grounding (RFMG), a lightweight, backbone-agnostic module that learns both which observations to trust and how much correction the available evidence should be allowed to induce. RFMG estimates observation-level reliability and uses the amount of reliable evidence to control correction authority: scarce reliable evidence primarily supports global metric calibration, while spatially varying local refinement is progressively enabled as reliable evidence accumulates. The monocular backbone remains frozen throughout. Experiments across three frozen monocular depth backbones, four datasets, observation budgets, and corruption conditions show effective few-point grounding and reduced sensitivity to unreliable observations. On NYUv2 with UniDepthV2, only eight clean observations reduce AbsRel from 0.0766 to 0.0540, a 29.5% relative reduction, while RFMG still achieves 0.0608 when 80% of the sparse observations are corrupted. These results support viewing sparse depth not as incomplete scene geometry that must be densified from scratch, but as compact metric evidence for grounding the dense geometric prior already encoded by a monocular foundation model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
monocular depth estimation, metric depth estimation, sparse depth completion, depth foundation models, sparse metric grounding

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jqiSOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes Reliable Few-Point Metric Grounding (RFMG), a lightweight framework for improving frozen monocular depth foundation models using extremely sparse metric observations. RFMG estimates observation-level reliability, performs reliability-weighted global scale calibration, and progressively enables local refinement according to the effective amount of reliable evidence. Experiments across three backbones and four datasets demonstrate improvements in metric depth accuracy and robustness to synthetic observation corruption.

Critical strengths

S1. Practical and meaningful problem. The paper addresses reliable metric grounding with only 1–16 observations, a relevant setting that complements conventional depth completion and modern monocular depth foundation models.

S2. Simple and modular design. Separating observation reliability from correction authority provides an intuitive framework for combining global calibration with conservative local refinement while keeping the backbone frozen.

S3. Broad experimental coverage. Evaluation across three backbones, four datasets, and multiple observation budgets demonstrates consistent improvements. On NYUv2, eight clean observations reduce UniDepthV2's AbsRel from 0.0766 to 0.0540, while maintaining 0.0608 under 80% corruption.

S4. Useful component analysis. The ablations suggest that reliability estimation and controlled local refinement both contribute to robustness, with only a small sacrifice in clean-observation accuracy compared with direct local grounding.

Critical weaknesses

W1. Incomplete evidence of superiority over existing methods. In Table 1, LDCM achieves better clean-observation accuracy at N=8 and N=16 (0.0488/0.0338 versus 0.0540/0.0401 for UniDepthV2+RFMG). Although RFMG performs better under heavy corruption, the baseline training and adaptation protocols are insufficiently clarified. Comparisons with simple robust scale estimators, such as median or Huber regression, are also missing.

W2. Robustness is evaluated primarily under matched synthetic corruption. Training and evaluation use the same depth-dependent Gaussian corruption model, with observations uniformly sampled from valid ground-truth pixels. This does not establish robustness to unseen noise distributions, systematic sensor bias, structured outliers, or realistic sparse sensing patterns.

W3. Observation reliability is not directly evaluated. Despite explicitly supervising the reliability scores, the paper reports no classification or calibration metrics. It remains unclear whether the reliability module can distinguish genuinely corrupted measurements from accurate measurements that strongly disagree with an erroneous monocular prior.

W4. Insufficient ablation of the correction-authority mechanism. The proposed gate depends on the sum of predicted reliability scores, but its specific design is not isolated experimentally. Comparisons against fixed gating, observation-count-based gating, and ungated reliability-aware refinement are missing. Furthermore, the effective-count formulation does not explicitly account for spatial redundancy among observations.

W5. Limited statistical and protocol transparency. Results lack variability estimates across independent observation-sampling and corruption seeds, which is particularly important in the few-point regime. The train/validation/test partitions for ETH3D, KITTI, and DIODE are also insufficiently specified despite dataset-specific refiner training.

Questions for authors

Q1. Regarding W1: Could the authors clarify whether the baselines use comparable training and adaptation protocols, and provide backbone-matched comparisons with simple robust scale estimators? This would help establish the benefit beyond corruption-aware training.

Q2. Regarding W2: How does RFMG perform under unseen corruption types, such as systematic bias, structured outliers, or real RGB-D/LiDAR noise? An out-of-distribution robustness evaluation would substantially strengthen the main claim.

Q3. Regarding W3: Could the authors report reliability prediction metrics (e.g., AUROC, AUPRC, and calibration error) and analyze clean observations exhibiting large disagreement with the monocular prior?

Q4. Regarding W4: Could the authors compare the proposed authority gate against fixed, observation-count-based, and ungated alternatives, including scenarios with clustered versus spatially distributed observations?

Q5. Regarding W5: Could the authors report mean and standard deviation across multiple observation-sampling and corruption seeds, and clarify the dataset splits used for in-domain training and evaluation?

1 comment

The rebuttal deadline is approaching, and I would like to remind the authors to provide their response before it closes.

I believe this work addresses a worthwhile problem and am open to discussing the concerns raised in my review. The authors' clarifications may help me reconsider my assessment. However, without a response to these concerns, I would not have sufficient grounds to adjust my current rating.

Reviewer_DNJQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how few sparse depth points are needed to correct a frozen monocular depth model, focusing on 1–16 points. It proposes RFMG, a small module that scores each point's reliability, corrects the global scale with the weighted points, and enables local correction only when enough reliable points exist. The backbone is frozen and only RFMG is trained, using synthetic corrupted points to learn reliability. Experiments on three backbones and four datasets show clear gains from a few points and better robustness to corrupted points.

Critical strengths
  1. The problem is practical and well motivated. The paper asks how many sparse depth points are needed to correct a frozen monocular depth model, and focuses on the difficult few-point case.

  2. The method is simple and easy to follow. Each point receives a reliability score, the weighted points set the global scale, and local correction is allowed only when enough reliable points are available. Reducing to global scaling when only one point is available is a clean design choice.

Critical weaknesses
  1. Corruption model. Training and evaluation use the same synthetic Gaussian corruption, so the robustness results may not transfer to real errors. It would be better to test unseen corruption types, such as outliers, flying pixels, and biased points, and real sensor inputs.

  2. Sparse patterns. All sparse points are sampled uniformly from the ground truth. It would be better to also test structured and real patterns, such as SIFT/ORB keypoints, LiDAR lines, and SfM points.

  3. Visual comparisons. It would be better to add visual comparisons with other methods, such as LDCM, PriorDA, and Marigold-DC.

Questions for authors

Please see weaknesses above.

Reviewer_ZNfWOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how to ground a frozen monocular depth foundation model using only a few sparse metric observations (N = 1–16) that may be unreliable. The authors propose RFMG, a lightweight, backbone-agnositic module that estimates observation-level reliability, performs reliability-weighted global scale calibration, and progressively enables local refinement as the amount of reliable evidence increases. Experiments on three backbones and four datasets under clean and synthetically corrupted conditions show that RFMG achieves substantial metric gains from only a few observations while reducing sensitivity to corrupted measurements.

Critical strengths

This paper systematically studies how a strong monocular depth foundation model can be grounded with only a few metric observations (N = 1–16) and how far these observations can be trusted. Unlike prior-guided methods such as PriorDA, Marigold-DC, and LDCM, which mainly use sparse measurements to improve a monocular prior, this work addresses the underexplored problem of observation reliability under extreme scarcity. This problem formulation is the main reason I see potential value in this work.

Critical weaknesses

W1. Unfair comparison : RFMG is trained on the same few-point and corruption distributions used for evaluation, whereas the baselines appear to be evaluated in a zero-shot manner without training on this setting. In particular, the reported CompletionFormer result (AbsRel 0.247 at N = 16) is difficult to reconcile with the retrained results in the original paper, suggesting that the 500-point checkpoint may have been applied without retraining.

W2. Incomplete evaluation metrics : Although Appendix D.2 states that RMSE and δ are additionally computed, these results are not reported in either the main paper or the appendix. As these are standard metrics in depth completion, they would also be important for substantiating the robustness claims.

W3. Missing standard benchmarks and protocols : The method is not evaluated under the standard depth completion settings, namely 500 points on NYUv2 and LiDAR scan-line patterns on KITTI DC. In addition, the evaluation resolution (480 × 640) differs from the standard protocol (304 × 228 center crop), which precludes direct comparison with the existing literature.

W4. Lack of cross-dataset generalization : A separate refiner is trained for each dataset, so it remains unclear how the method performs when applied to unseen datasets without retraining. Such an evaluation would also be necessary for a fair comparison with zero-shot baselines.

W5. Limited technical novelty : The individual components, including null attention and sigmoid/tanh gating, are largely established techniques, and the threshold design of β appears to be hand-crafted. Furthermore, no ablation isolates the contribution of β gating.

W6. Bounded global scale calibration : The global scale is constrained to (0.7, 1.3), which means that scale errors exceeding 30% cannot be fully corrected by global calibration. No justification or sensitivity analysis is provided for this range, and it may have limited performance on datasets with large prior errors, such as DIODE and ETH3D.

Questions for authors

Q1. Were the baselines in Table 1 retrained or fine-tuned under the proposed few-point (N = 1–16) and corruption settings? If not, could the authors provide results for baselines trained under the same conditions?

Q2. Could the authors report the RMSE and δ(Accuracy metric) results for Tables 1–3, which Appendix D.2 states were computed?

Q3. How does RFMG perform under the standard depth completion settings, namely 500 points on NYUv2 and LiDAR scan-line patterns (64/16/4/1 lines) on KITTI DC? It would also be helpful if these results followed the standard evaluation protocol used by existing methods (resizing to 320 × 240 followed by a 304 × 228 center crop).

Q4. How does a refiner trained on NYUv2 perform when applied to ETH3D, KITTI, and DIODE without retraining?

Q5. What is the rationale for the global scale bound Δ = 0.30 and the clipping range [0.2, 5.0] for tᵢ? Could the authors report results with different values of Δ (e.g., 0.5, 1.0, or unbounded)? In particular, it would be informative to show the actual distribution of global scale errors for each backbone on DIODE and ETH3D, where prior errors are large, and whether this bound limits performance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.