Metric Scale Propagation Network for Zero-Shot Depth Completion

Zero-shot depth completion aims to recover dense metric depth (i.e., physical distance) from RGB images and sparse depth observations in unseen scenes. We identify that the key challenge of this task is to preserve the critical scene scale from input sparse depth to final dense prediction. Existing end-to-end methods generally normalize scene scales to facilitate generalization and stabilize training, which may distort the scene scale. Recent depth-foundation-model-based methods substantially improve generalization via relative depth priors, which lack the scale information. To mitigate this issue, we propose MSP-Net, a Metric Scale Propagation Network to explicitly identify and propagate scene scale through an end-to-end process. It comprises two main modules: the Scale Identification Module (SIM) and the Scale Propagation Module (SPM). SIM identifies reliable sparse metric observations and estimates scene scale at sparse locations. SPM then propagates these scale cues from sparse locations to the whole image, producing a dense scale field. Optionally, a pretrained depth foundation model is integrated to further boost cross-scene generalization. Experiments are conducted on six unseen benchmarks with diverse sparse depth patterns. Without pretraining, our MSP-Net outperforms all recent end-to-end methods and even most pretrained methods. With pretraining, our MSP-Net outperforms all recent pretrained methods with faster inference speed.