Geometry-Aware Scale Propagation for Robust Depth Completion

Depth completion aims to recover a dense depth map from an RGB image and sparse depth measurements. Recent depth completion methods leverage relative depth foundation models to substantially improve real-world generalization. However, how to fuse the crucial metric scale information from sparse depth measurements with relative depth remains a challenge. In this paper, we identify that sparse depth patterns in the 2D plane can lead to scale ambiguity for depth completion in the 3D scene. That is, similar sparse depth patterns may correspond to different 3D geometries, resulting in ambiguous scale correspondence between sparse measurements and relative depth. It consequently yields distorted 3D geometry in depth completion. To mitigate, we propose GeoSP, a Geometry-aware Scale Propagation framework for robust depth completion, which propagates metric scale from sparse depth to dense relative depth guided by 3D geometry. More specifically, GeoSP identifies 3D geometric relations between sparse depth measurements and dense relative depth, and then formulates them as propagation cues to compute metric scale in depth completion. It contains three complementary modules. Geometric Neighborhood Selection uses the relative 3D geometry priors to select valid scale constraints and identify geometric relations. Geometric Affinity Prediction then introduces a lightweight head with ground-truth 3D geometry supervision to adaptively estimate the contribution of each constraint from these relations. Finally, Affinity-Weighted Solver integrates the selected valid scale constraints according to their predicted affinities to compute metric scale. Extensive experiments are conducted on ten unseen benchmarks with diverse sparse depth patterns. Our GeoSP achieves state-of-the-art performance with lower complexity, particularly in 3D geometric accuracy, when compared to both traditional models and recent depth-foundation-model-based methods.