acceptodds
Under review as a conference paper at ICLR 2027

Metric Scale Propagation Network for Zero-Shot Depth Completion

Abstract

Zero-shot depth completion aims to recover dense metric depth (i.e., physical distance) from RGB images and sparse depth observations in unseen scenes. We identify that the key challenge of this task is to preserve the critical scene scale from input sparse depth to final dense prediction. Existing end-to-end methods generally normalize scene scales to facilitate generalization and stabilize training, which may distort the scene scale. Recent depth-foundation-model-based methods substantially improve generalization via relative depth priors, which lack the scale information. To mitigate this issue, we propose MSP-Net, a Metric Scale Propagation Network to explicitly identify and propagate scene scale through an end-to-end process. It comprises two main modules: the Scale Identification Module (SIM) and the Scale Propagation Module (SPM). SIM identifies reliable sparse metric observations and estimates scene scale at sparse locations. SPM then propagates these scale cues from sparse locations to the whole image, producing a dense scale field. Optionally, a pretrained depth foundation model is integrated to further boost cross-scene generalization. Experiments are conducted on six unseen benchmarks with diverse sparse depth patterns. Without pretraining, our MSP-Net outperforms all recent end-to-end methods and even most pretrained methods. With pretraining, our MSP-Net outperforms all recent pretrained methods with faster inference speed.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
depth completion, zero-shot generalization, foundation model, 3D computer vision

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XniYOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a Metric Scale Propagation Network (MSP-Net) aimed at addressing the key issue of scene scale distortion and loss during feature extraction and normalization in zero-shot depth completion tasks. The network primarily consists of two core modules: 1) Scale Identification Module (SIM), which predicts the reliability of each sparse depth point by fusing RGB and sparse depth features, and extracts global scale anchors and multi-level scale tokens; 2) Scale Propagation Module (SPM), which utilizes visual affinity and spatial distance to globally iterate and propagate reliable local sparse scale cues, generating a dense scale field. Ultimately, the network predicts residual depth in the normalized space and combines the propagated scale field to recover high-precision dense metric depth.

Critical strengths
  1. The SIM module proposed in this paper predicts pixel-wise reliability masks by fusing RGB and sparse features, enabling adaptive selection of reliable measurement points and solving the noise and unreliability issues in sparse depth.
  2. The SPM module proposed in this paper introduces an affinity matrix guided by RGB and a spatial distance penalty mechanism, achieving precise iterative propagation of scale information within visually homogeneous regions. It not only successfully solves the problem of global densification of sparse scales but also strictly prevents the "leakage" of scale information when it crosses object boundaries.
Critical weaknesses
  1. This article mentions converting multi-level scale tokens ({tl}\{t_l\}) into channel-level modulation parameters and injecting them into the encoder's features through "ScaleFiLM". However, the article does not provide a specific implementation plan, diagram, or mathematical definition.
  2. In the Scale Propagation Module (SPM), the number of iterations KK through the affinity matrix determines the computational complexity and memory usage of the algorithm. However, the article does not mention how the number of iterations KK is specifically set. The experimental details also lack information regarding the value of KK. It is recommended to supplement with ablation experiments on the choice of KK.
  3. In Table 1 and other tables, what do the superscripts K and N in OGNI-DC represent? It is recommended to provide relevant explanations.Regarding the limitations mentioned in the conclusion, what is the core reason why the introduction of Depth Anything V2 prior actually improves the performance when dealing with sparse patterns with extremely uneven physical measurement distributions, such as KITTI 64-line LiDAR?
Questions for authors

Regarding the limitations mentioned in the conclusion, what is the core reason why the introduction of Depth Anything V2 prior actually improves the performance when dealing with sparse patterns with extremely uneven physical measurement distributions, such as KITTI 64-line LiDAR?

Reviewer_1gvcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MSP-Net for zero-shot depth completion. It argues that the key challenge for depth completion is preserving scene scale from sparse observations to the dense prediction. A Scale Identification Module (SIM) predicts a reliability map, a scale anchor, and scale tokens. A Scale Propagation Module (SPM) diffsuses sparse scale cues into a dense scale field. An optional frozen Depth Anything V2 prior can be added to improve the generalization ability of residual depth prediction. Experiments on six unseen datasets with simulated and raw sparse patterns demonstrate the effectiveness of the proposed MSP-Net.

Critical strengths
  1. The scale field with a multiplicative residual is interpretable, and the model is lightweight (36.7M parameters).
  2. The evaluation covers six unseen datasets with five simulated and five raw sparse patterns.
  3. The accuracy–efficiency trade-off is strong relative to much larger foundation-model-based approaches.
Critical weaknesses
  1. Discussion of reliability map. It would be better to inject noise or flying pixels into the sparse input to verify that the reliability map identifies unreliable points, with quantitative results or at least visualizations of the reliability map.
  2. More comparisons. It would be better to compare with more recent depth completion baselines, like LingBot-Depth [1], LDCM [2], and Any2Full [3].
  3. Ablation with DA2. Since DA2 gives the largest gain in Table 3, it would be better to ablate SIM and SPM on top of DA2 and report their marginal gains with the relative depth prior.

References:

[1] LingBot-Depth: Masked Depth Modeling for Spatial Perception. ECCV, 2026.

[2] Large Depth Completion Model from Sparse Observations. ICLR, 2026.

[3] Any to Full: Prompting Depth Anything for Depth Completion in One Stage. ECCV, 2026.

Questions for authors

Please see weaknesses above.

Reviewer_txmHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies zero-shot depth completion for unseen scenes. The Scale Identification Module takes RGB, sparse depth, and a validity mask to predict a reliability map, a weighted global depth anchor, and scale tokens. The Scale Propagation Module initializes a dense scale field from the sparse observations and global anchor, then iteratively propagates values using RGB-derived feature affinity and spatial distance. A residual depth network receives normalized sparse depth and the normalized scale field, with scale tokens injected through feature-wise modulation. The final dense metric depth is obtained by multiplying the propagated scale field by one plus the predicted residual. A frozen relative-depth model can optionally provide an additional structural prior. Evaluation uses six unseen benchmarks with simulated and raw sparse-depth patterns, both with and without Depth Anything V2.

Critical strengths
  1. MSP-Net gives sparse metric observations a dedicated path through scale identification and propagation rather than relying only on the completion network to preserve their magnitude implicitly.
  2. The propagation update repeatedly reinjects reliability-weighted sparse measurements while using local affinity to guide scale values into unobserved pixels.
  3. The optional depth-foundation-model input is integrated alongside the explicit metric path, separating relative structural guidance from the sparse observations that carry physical scale.
Critical weaknesses
  1. The reliability map is learned through the final depth objective, without direct labels or calibration for whether individual sparse measurements are correct. Its values may therefore function as task-specific gating weights rather than calibrated measurement reliabilities.
  2. The global anchor is a reliability-weighted mean of observed depths, so it changes with the depth range and sampling pattern. Its interpretation as a scene-scale quantity is therefore ambiguous when sparse inputs omit near or far surfaces.
  3. The propagation affinity is computed from RGB-derived features and combined with spatial distance. Visually similar regions on different depth layers may therefore exchange scale values, while large gaps depend on repeated local propagation steps.
  4. The final prediction is formed as the propagated field times one plus a learned residual, with no hard data-consistency operation that copies observed sparse depths into the output. Preservation of those measurements depends on the learned loss rather than the output construction.
Questions for authors
  1. Are the predicted reliability values intended to be calibrated probabilities of measurement correctness, or task-specific weights learned for the final completion objective?
  2. Since the global anchor is a reliability-weighted mean of observed depths, how does the model keep this reference stable when sparse samples disproportionately cover near or far surfaces?
  3. How does the RGB-derived affinity avoid propagating scale between visually similar regions that lie on different depth surfaces?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.