acceptodds
Under review as a conference paper at ICLR 2027

Geometry-Aware Scale Propagation for Robust Depth Completion

Abstract

Depth completion aims to recover a dense depth map from an RGB image and sparse depth measurements. Recent depth completion methods leverage relative depth foundation models to substantially improve real-world generalization. However, how to fuse the crucial metric scale information from sparse depth measurements with relative depth remains a challenge. In this paper, we identify that sparse depth patterns in the 2D plane can lead to scale ambiguity for depth completion in the 3D scene. That is, similar sparse depth patterns may correspond to different 3D geometries, resulting in ambiguous scale correspondence between sparse measurements and relative depth. It consequently yields distorted 3D geometry in depth completion. To mitigate, we propose GeoSP, a Geometry-aware Scale Propagation framework for robust depth completion, which propagates metric scale from sparse depth to dense relative depth guided by 3D geometry. More specifically, GeoSP identifies 3D geometric relations between sparse depth measurements and dense relative depth, and then formulates them as propagation cues to compute metric scale in depth completion. It contains three complementary modules. Geometric Neighborhood Selection uses the relative 3D geometry priors to select valid scale constraints and identify geometric relations. Geometric Affinity Prediction then introduces a lightweight head with ground-truth 3D geometry supervision to adaptively estimate the contribution of each constraint from these relations. Finally, Affinity-Weighted Solver integrates the selected valid scale constraints according to their predicted affinities to compute metric scale. Extensive experiments are conducted on ten unseen benchmarks with diverse sparse depth patterns. Our GeoSP achieves state-of-the-art performance with lower complexity, particularly in 3D geometric accuracy, when compared to both traditional models and recent depth-foundation-model-based methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Depth Completion, Depth Foundation Models, Metric Scale Propagation, 3D Geometry, Zero-shot Generalization

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_AhLuOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GeoSP for dense depth completion from an RGB image and sparse metric measurements using a frozen depth foundation model. Following global scale-and-shift alignment, the method selects sparse neighbors by 3D distance within a 2D candidate pool. A lightweight affinity network combines foundation-model features, point properties, and geometric relations to estimate propagation weights. An affinity-weighted least-squares solver then aggregates neighboring depth ratios to recover local scale and correct the dense depth. The model is trained on simulated sparse inputs from NYUv2 and KITTI and evaluated on six simulated-input and four raw-sensor datasets, achieving strong average ranks, particularly on 3D metrics.

Critical strengths

The evaluation spans diverse scenes and sensors and provides practically useful results. Training uses only NYUv2 and KITTI, while Tables 1–2 cover ten target datasets and distinguish simulated sparse patterns from raw sensor inputs. GeoSP achieves overall average ranks of 1.33, 1.56, and 2.04 for simulated indoor, simulated outdoor, and raw-input evaluations, respectively. Although it is not best on every individual metric, the strong aggregate performance across these settings supports the practical value of the completion strategy.

The lightweight affinity network and explicit scale aggregation form a clear and reasonably interpretable design. Freezing the foundation model and learning only constraint weights preserves an explicit connection between the output and sparse measurements, with only 0.905M additional trainable parameters. The geometric-feature, supervision, and depth-space-transfer ablations in Table 3 also provide positive evidence. This is a promising simple design, and the simplicity of the weighted solver alone does not diminish its potential contribution.

Critical weaknesses

The depth representation and alignment underlying the shared 3D space are insufficiently specified, affecting the interpretation of the core geometry. Equation (1) applies depth-domain scale-and-shift alignment to the foundation-model output, whereas the relative Depth Anything V2 model predicts affine-invariant inverse depth. The paper does not specify whether DmD_m is the raw output, a transformed depth map, or a prediction from another checkpoint, nor the fitting objective for γ,ρ\gamma,\rho and validity handling. If alignment occurs in inverse-depth space followed by conversion, this should be stated; if raw output is directly fitted affinely to metric depth, the rationale needs explanation. This definition also determines the meaning of the 3D neighborhoods, normals, and depth ratios in Eq. (5). The full transformation and handling of nonpositive or near-zero depths should be clarified.

The ablations do not sufficiently isolate the central contribution of geometry-guided scale propagation. PSD already uses foundation-model guidance and propagation in 3D space, so the additional benefit over this class of methods requires controlled evidence. When Table 3a removes 3D features, it is unclear whether 3D neighborhood selection remains. Table 3d compares depth-space and point-space transfer, but lacks direct references such as global alignment, uniform scale aggregation, or 2D neighborhoods with the weighting otherwise fixed. The gains therefore remain incompletely attributed between neighborhood selection, local ratio propagation, and learned geometric weights. Moreover, the ablations use only the training domains, NYU/KITTI, while the central claim concerns OOD robustness, and p. 7 explicitly notes a training-resolution difference from PSD. The ten-dataset results are valuable, but do not independently establish the proposed mechanism or its cross-domain advantage. Key controls with matched foundation models and preprocessing would strengthen this attribution and help align the conclusions with the evidence.

Questions for authors

These clarification requests could change my assessment. Implementation details or existing results should take priority; a new large-scale benchmark expansion can be left for future work.

Which Depth Anything V2 checkpoint and representation define DmD_m? Please give the complete sequence and objective for raw prediction, scale/shift fitting, depth conversion, and back-projection, including the handling of nonpositive or near-zero denominators in Eq. (5). Do A0/A1 in Table 3a still select neighbors by 3D distance? Apart from removing input features, which geometric operations remain, and which specific component does this ablation isolate? Are controls using global alignment alone, uniform ratio aggregation, or a 2D/3D neighborhood substitution with other settings fixed already available? If so, can the most informative existing control be reported on a target-domain or raw-sensor setting? Which PSD foundation model and checkpoint are used in Tables 1–2, and do they match GeoSP? How do training/test resolution, intrinsic transformations, and sparse-input handling differ? Are existing results available that separate these factors?

5 comments

We sincerely appreciate the reviewers’ constructive comments. All these comments have been carefully studied and completely addressed in the revised paper. These comments helped us significantly improve the quality of the paper. Most major revisions are marked in red in the revised paper.

Depth Representation and Alignment

Weakness 1:

"The depth representation and alignment underlying the shared 3D space are insufficiently specified, affecting the interpretation of the core geometry. Equation (1) applies depth-domain scale-and-shift alignment to the foundation-model output, whereas the relative Depth Anything V2 model predicts affine-invariant inverse depth. The full transformation and handling of nonpositive or near-zero depths should be clarified."

Question 1:

" Which Depth Anything V2 checkpoint and representation define the relative depth? Please give the complete sequence and objective for raw prediction, scale/shift fitting, depth conversion, and back-projection, including the handling of nonpositive or near-zero denominators in Eq. (5)."

Response

First, we have clarified that GeoSP uses the frozen Depth Anything V2-L checkpoint, which predicts affine-invariant inverse depth rather than metric depth.

Second, we have added Eqs. (1)–(3) to explicitly describe the full transformation from inverse-depth prediction to the shared 3D space, including the corrected alignment formulation. Specifically, Eq. (1) defines the least-squares objective for global affine alignment in the inverse-depth domain, Eq. (2) converts the aligned inverse depth to metric depth, and Eq. (3) back-projects the aligned dense depth and sparse metric depth into the shared 3D space.

Finally, we have clarified the handling of nonpositive or near-zero inverse-depth values by applying a minimum threshold of 10−410^{-4} to ensure numerical stability during depth conversion and subsequent scale propagation.

Please find the revisions in Sec. 3.1 on Page 3, as follows.

(Page 3) 3.1 OVERVIEW

"Specifically, we perform Global Alignment in disparity space by fitting a global scale γ\gamma and shift ρ\rho to sparse depth measurements:

(\gamma^{\ast},\rho^{\ast}) = \arg\min_{\gamma,\rho} \sum_{i\in\Omega_s} \left(\gamma R_m(i)+\rho-\frac{1}{D_s(i)}\right)^2 \tag{1}

where Ωs\Omega_s denotes the set of valid sparse measurements used for alignment. The aligned inverse depth is then converted into depth: \widetilde{D}_m(q) = \frac{1}{\gamma^* R_m(q) + \rho^*} \tag{2} For numerical stability, the aligned disparity is clamped to a minimum of 10−410^{-4} before inversion, while the resulting depth is also clamped to a minimum of 10−410^{-4} for subsequent scale propagation.

We then back-project the aligned depth D~m\widetilde D_m and sparse metric depth DsD_s using the camera intrinsic matrix KK:

\qquad P_s(i)=D_s(i)K^{-1}\bar i. \tag{3}$$ *where $\bar q$ and $\bar i$ are homogeneous pixel coordinates."* ### Depth Representation and Alignment **Weakness 2:** *"The ablations do not sufficiently isolate the central contribution of geometry-guided scale propagation. PSD already uses foundation-model guidance and propagation in 3D space, so the additional benefit over this class of methods requires controlled evidence. When Table 3a removes 3D features, it is unclear whether 3D neighborhood selection remains."* **Question 2:** *"Do A0/A1 in Table 3a still select neighbors by 3D distance? Apart from removing input features, which geometric operations remain, and which specific component does this ablation isolate?"* ### Response We clarify that **A0/A1 in Table 3a still select neighbors by 3D distance**. Specifically, the removed 3D features refer to the 3D geometric relations extracted from the selected neighbors in Sec. 3.2, rather than the 3D neighborhood selection itself. Thus, these ablations isolate the effects of geometric relations on affinity prediction. Please find the clarification in **Sec. 4.3.2** as follows: *“Both variants retain the same 3D neighborhood selection and affinity-weighted solver as the full model, isolating the effects of geometric relations extracted from 3D neighbors.”*
AuthorsOpenReview ↗

Response to Concerns about Controlled Ablations and OOD Generalization

Weakness 3:

“Table 3d compares depth-space and point-space transfer, but lacks direct references such as global alignment, uniform scale aggregation, or 2D neighborhoods with the weighting otherwise fixed. The gains therefore remain incompletely attributed between neighborhood selection, local ratio propagation, and learned geometric weights. Moreover, the ablations use only the training domains, NYU/KITTI, while the central claim concerns OOD robustness.”

Question 3:

“Are controls using global alignment alone, uniform ratio aggregation, or a 2D/3D neighborhood substitution with other settings fixed already available? If so, can the most informative existing control be reported on a target-domain or raw-sensor setting?”

Response

Following the reviewer's suggestion, we have added controlled ablation studies to better attribute GeoSP's gains to neighborhood selection, local ratio propagation, and learned geometric weights, and to validate their contributions under OOD settings.

First, we have added three controlled baselines: global alignment only, uniform affinity aggregation, and 2D neighborhood selection, while keeping the remaining components unchanged where applicable. The results show that local ratio propagation substantially improves over global alignment, while 3D neighborhood selection and learned geometric weights provide complementary benefits.

Second, we have extended these controlled comparisons to four unseen raw-sensor benchmarks: SUN RGB-D, TOFDC, DIML Indoor, and VOID1500. All configurations follow the original NYUv2–KITTI training protocol without target-domain fine-tuning. The results further support the effectiveness of geometry-guided scale propagation under OOD settings.

Please find the revisions on Page 8, Table 3, and Sec. 4.3.1 of the revised manuscript, as follows:

“We evaluate the core mechanisms on four unseen raw-sensor benchmarks: SUN RGB-D, TOFDC, DIML Indoor, and VOID1500. Specifically, we isolate the contributions of neighborhood selection and affinity prediction through controlled comparisons: comparing 2D and 3D neighborhoods while keeping learned affinity prediction fixed, and comparing uniform and learned affinities while keeping neighborhood selection fixed. Global alignment without local scale propagation is also included as a reference baseline. All configurations follow the NYUv2–KITTI training protocol without target-domain tuning or adaptation.

Table 3. Core Mechanism Ablation on Unseen Raw-Sensor Benchmarks.

MethodNeighborhoodAffinitySUN RGB-D Rel ↓SUN RGB-D Ov. ↓TOFDC Rel ↓TOFDC Ov. ↓VOID1500 Rel ↓VOID1500 Ov. ↓DIML Rel ↓DIML Ov. ↓
Global Alignment——12.071008.64.5860.75.1956.819.30483.0
2D + Uniform2DUniform4.16262.12.5333.73.9236.916.58136.8
2D + Learned2DLearned4.02251.82.4832.53.1235.316.28134.9
3D + Uniform3DUniform3.70252.12.4131.63.3134.615.06132.9
Full GeoSP3DLearned3.622502.3231.52.1225.214.60130.0

Both 3D neighborhood selection and learned affinity prediction consistently improve Rel and Ov. across all four benchmarks. Combining them yields the best performance across all eight benchmark–metric combinations, demonstrating their complementary contributions to zero-shot metric scale recovery.”

AuthorsOpenReview ↗

Reponse to concern about fairness of Comparison with PSD

Fairness of Comparison with PSD

Weakness 4:

"and p. 7 explicitly notes a training-resolution difference from PSD. The ten-dataset results are valuable, but do not independently establish the proposed mechanism or its cross-domain advantage. Key controls with matched foundation models and preprocessing would strengthen this attribution and help align the conclusions with the evidence."

Question 4:

" Which PSD foundation model and checkpoint are used in Tables 1–2, and do they match GeoSP? How do training/test resolution, intrinsic transformations, and sparse-input handling differ? Are existing results available that separate these factors?"

Response

First, we clarify that PSD uses the same pretrained Depth Anything V2-Large checkpoint as GeoSP, enabling a backbone-matched comparison in Tables 1–2.

Second, we clarify that all compared methods follow the same test-time evaluation protocol, including identical test images, sparse-depth observations, evaluation resolutions, and valid-pixel masks. For methods requiring camera intrinsics, we use dataset-provided calibration with corresponding transformations.

Finally, regarding training configurations, PSD uses a broader KITTI training set with fixed-size cropping and resizing, whereas GeoSP is trained on NYUv2 and the KITTI Eigen subset at their original resolutions. Although these training differences remain, the shared test-time protocol ensures consistent evaluation conditions.

Please find the clarification in Sec. 4.1 (Implementation Details) of the revised manuscript, as follows:

“ All compared methods follow the same evaluation protocol as PSD, using consistent test images, sparse-depth observations, evaluation resolution, and valid-pixel masks. For methods requiring camera intrinsics, we use the dataset-provided calibration with corresponding transformations.”

We sincerely thank the reviewer for highlighting these clarification requests. We have prioritized clarifying the implementation details and providing controlled ablation results to directly address the concerns. We hope these revisions help resolve the remaining questions and support a reassessment of our work. We welcome any further questions or discussion. We will upload the revised manuscript and supplementary material shortly.

Reviewer_n3fDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

GeoSP addresses depth completion by treating sparse depth patterns in the 2D image plane as a source of 3D scale ambiguity. Given an RGB image and sparse metric depth, it uses a frozen depth foundation model to predict dense relative depth, globally aligns it with the sparse measurements, and lifts both into a shared 3D space using dataset-provided camera intrinsics. For each dense query point, it selects sparse neighbors by 3D distance, extracts geometric relations between the query and its neighbors, and trains a lightweight affinity head to estimate how reliable each sparse constraint is for local metric scale recovery. The predicted affinities are then used to weight these constraints and solve for a local metric scale, which propagates the metric scale from sparse depth to the dense relative depth.

Critical strengths
  1. This paper selects sparse neighbors in a shared 3D space rather than the 2D image plane and estimates each constraint's affinity from geometric relations, giving the method a clear geometric motivation.

  2. This paper keeps the depth foundation model frozen and trains only a lightweight affinity head with about 0.905M parameters, achieving a favorable accuracy-complexity trade-off.

Critical weaknesses
  1. It would strengthen the empirical comparison to include recent sparse depth completion methods such as, OMNI-DC[1], LDCM[3] and Lingbot-Depth[3] as additional baselines.

  2. The authors are encouraged to compare the inference time of GeoSP with that of the baselines to clarify its practical efficiency.

  3. Testing GeoSP with additional depth foundation models, such as MoGe, different Depth Anything V2 variants, and Marigold, would help show that the gains are not specific to Depth Anything V2.

  4. Some depth completion methods that explicitly operate in 3D space or leverage 3D geometric representations [3][4][5][6] should be mentioned

  5. It would be helpful to evaluate GeoSP without ground-truth camera intrinsics, for example using estimated or perturbed intrinsics, since the method relies on intrinsics to lift sparse and relative depth into 3D space and its sensitivity to intrinsic errors is currently unclear.

[1] ICCV 2025, OMNI-DC: Highly Robust Depth Completion with Multiresolution Depth Integration

[2] ICLR 2026, Large Depth Completion Model from Sparse Observations.

[3] ECCV 2026, Masked Depth Modeling for Spatial Perception

[4] ICCV 2019, Learning Joint 2D-3D Representations for Depth Completion.

[5] CVPR 2023, BEV@DC: Bird’s-Eye View Assisted Training for Depth Completion.

[6] ICCV 2023, Aggregating Feature Point Cloud for Depth Completion.

[7] CVPR 2024, Tri-Perspective View Decomposition for Geometry-Aware Depth Completion.

Questions for authors

Please see weakness

4 comments
AuthorsOpenReview ↗

Responses to Weaknesses 1–3

Thank you for your constructive comments. These comments helped us significantly improve the quality of the paper. We have carefully addressed all comments and highlighted the major revisions in red in the revised manuscript.

1. Additional Recent Baselines.

We have added these recent sparse depth completion methods such as, OMNI-DC, LDCM, and LingBot-Depth in Table 1 and Table 2. The results show that GeoSP consistently achieves the best performance under all three evaluation settings, including indoor, outdoor and raw sparse depth.

Table 1 Comparison on simulated sparse depth.

MethodDIODE Indoor (MAE / Ov. / F)Stanford (MAE / Ov. / F)Middlebury (MAE / Ov. / F)All Rank ↓3D Rank ↓
LingBot-Depth1455 / 389 / 34.7412 / 359 / 10.53424 / 763 / 22.413.4413.50
OMNI-DC181 / 111 / 64.891.0 / 163 / 21.0217 / 102 / 54.92.612.75
LDCM364 / 182 / 60.1104 / 184 / 18.91102 / 249 / 42.07.838.17
GeoSP (Ours)192 / 98.8 / 68.089.1 / 161 / 22.0240 / 91.4 / 55.01.441.00
MethodDIML Outdoor (MAE / Ov. / F)DIODE Outdoor (MAE / Ov. / F)DrivingStereo (MAE / Ov. / F)All Rank ↓3D Rank ↓
LingBot-Depth12791 / 2490 / 7.408073 / 1230 / 6.3013467 / 2345 / 9.7011.7811.33
OMNI-DC2342 / 615 / 18.01735 / 458 / 13.6916 / 224 / 21.63.113.17
LDCM3566 / 1141 / 15.11906 / 573 / 11.01401 / 346 / 17.85.835.92
GeoSP (Ours)2714 / 761.3 / 19.01651 / 395.6 / 16.0821.2 / 219.0 / 22.01.671.67

Table 2 Comparison on raw sparse depth

MethodDIML Indoor (MAE / Ov. / F)SUN RGB-D (MAE / Ov. / F)TOFDC (MAE / Ov. / F)VOID1500 (MAE / Ov. / F)All Avg. Rank ↓3D Avg. Rank ↓
OMNI-DC416.4 / 133.4 / 27.4412 / 253 / 9.6340.8 / 31.5 / 89.728.9 / 23.3 / 92.13.212.69
LDCM400.8 / 130.8 / 27.51250 / 332 / 9.6040.1 / 31.8 / 89.649.6 / 33.2 / 84.36.836.44
LingBot-DC650.4 / 164.0 / 25.11671 / 465 / 9.5542.1 / 32.5 / 89.7460 / 175 / 19.311.7511.00
GeoSP (Ours)383 / 130 / 27.9408 / 250 / 9.6740.1 / 31.5 / 89.936.2 / 25.2 / 91.82.381.81

2.Inference Efficiency

We have compared the inference time of GeoSP with that of the baselines to clarify its practical efficiency. GeoSP achieves better performance with faster inference than LingBot-Depth-DC (2.1×), TestPromptDC (46.2×), and Marigold-DC (76.2×). Although slower than other baselines, including PSD, Any2Full, OMNI-DC, and LDCM, GeoSP achieves the best performance across all three evaluation settings.

Detailed evaluation settings and results are provided in Supplementary Sec. A.5 and Table 9 as follows:

“Specifically, we evaluate on 50 images from the NYUv2 test set at the native resolution of 640×480. All methods are run on an RTX 3090 with a batch size of 1. We report the average end-to-end inference time per image after warm-up and the corresponding throughput (FPS). The measured time covers the complete inference pipeline from this native-resolution input, including backbone inference, any internal resizing, and subsequent processing.”

Table 9 Inference Time

ModelInference Time (ms) ↓FPS ↑
PromptDA33.629.75
DepthPrompting65.815.20
OMNI-DC96.210.39
PriorDA116.88.56
PSD118.28.46
LDCM150.56.65
Any2Full177.35.64
GeoSP (Ours)396.82.52
LingBot-Depth-DC832.81.20
TestPromptDC18346.70.05
Marigold-DC30254.10.03

3.Generality Across Depth Foundation Models

We additionally evaluated GeoSP with two alternative depth foundation models, MoGe and Depth Anything V1 (DAv1). The results show that GeoSP maintains comparable performance across different foundation models. These findings support the applicability of GeoSP beyond the default DAv2-L backbone.

Detailed results are provided in Supplementary Sec. A.6 and Table 10.

Table. Generalization of GeoSP across Different Depth Foundation Models on Unseen Raw-Sensor Benchmarks.

Teacher FMDIML MAEDIML Ov.DIML F (%)SUN RGB-D MAESUN RGB-D Ov.SUN RGB-D F (%)TOFDC MAETOFDC Ov.TOFDC F (%)VOID1500 MAEVOID1500 Ov.VOID1500 F (%)
DAv1412.0133.927.4384.1248.89.740.131.789.740.027.589.3
MoGe418.2134.027.4373.4241.59.740.331.589.737.725.790.5
DAv2-L (GeoSP)38313027.94082509.6740.131.589.936.225.291.8

(We follow the same raw sparse depth setting as in Table 2.)

AuthorsOpenReview ↗

Responses to Weaknesses 4–5

4. Related Work on 3D Geometry

Following the reviewer's suggestion, we have cited these depth completion methods in the related works. Specifically, we have discussed BEV@DC, PointDC, and TPVD in Section 2.1 (Traditional Depth Completion) to better cover approaches,and discussed OMNI-DC, LDCM, and LingBot-Depth in Section 2.3 (Foundation-Model-Based Depth Completion) to provide a more comprehensive discussion of recent advances.

Please refer to the revised manuscript (Section 2.1 and 2.3) as follows:

“Beyond 2D image-based processing, several methods explicitly incorporate 3D geometric representations into depth completion. Chen et al. (Joint2D3D) and PointDC leverage 3D point features to enhance depth completion, while BEV@DC and TPVD exploit multi-view representations and geometric information to improve spatial reasoning and propagation.”

“While PromptDA, Any2Full, DepthPrompting, and OMNI-DC incorporate sparse metric information through feature integration, prompting, or depth propagation.”

“LingBot-Depth leverages large-scale RGB-D pretraining through masked depth modeling. Beyond image-domain integration, LDCM predicts metric 3D point maps using foundation-model priors, while PSD exploits 3D Euclidean distance for neighborhood selection.”

5.Sensitivity to Camera Intrinsics

We have conducted additional experiments to evaluate GeoSP's sensitivity to camera intrinsic errors using both estimated and perturbed intrinsics. The results show that GeoSP maintains stable 2D depth accuracy (Rel and MAE) under intrinsic errors, demonstrating its robustness to imperfect camera calibration. Although 3D reconstruction accuracy (Ov. and F-score) is more sensitive to intrinsic errors due to their direct influence on back-projected geometry, GeoSP remains effective in recovering metric depth.

Detailed experimental settings and results are provided in Supplementary Sec. A.7 and Table 11, as follows:

“Experimental Settings. We investigate the sensitivity of GeoSP to camera intrinsic errors using both perturbed and estimated camera intrinsics. The dataset-provided ground-truth intrinsics serve as the reference. For focal-length perturbations, we multiply both fxf_x and fyf_y by a common factor α∈{0.8,0.9,0.95,1.05,1.1,1.2}\alpha \in \{0.8, 0.9, 0.95, 1.05, 1.1, 1.2\}, while keeping the principal point unchanged. We additionally evaluate camera intrinsics estimated by MoGe-2 (ViT-S), which is used exclusively for intrinsic estimation rather than depth prediction. All experiments use the same trained GeoSP model without fine-tuning. Only the camera intrinsics are modified at inference time, while the RGB images, sparse metric depth, and model parameters remain unchanged.

Table 10: Sensitivity of GeoSP to camera intrinsic errors. GT denotes dataset-provided camera intrinsics. Focal-length perturbations scale both fxf_x and fyf_y, while MoGe-2 provides estimated camera intrinsics. Lower values are better.

ConditionNYUv2 RelMAEOv.FKITTI RelMAEOv.F
gt1.8453.438.70.7902.12362.5192.90.196
focal ×0.951.8453.440.00.7792.12362.3198.40.186
focal ×1.051.8453.437.50.8002.12362.6187.80.205
focal ×0.91.8453.441.30.7662.12362.2204.40.175
focal ×1.11.8453.436.50.8102.12362.7183.10.215
focal ×0.81.8453.544.60.7352.12361.8218.20.154
focal ×1.21.8453.534.60.8262.12363.0174.70.233
moge_k1.8453.441.50.7632.12362.4194.30.198
Finally, we sincerely hope that our clarifications and additional experiments have addressed your concerns. We welcome any further questions or discussion and will upload the revised manuscript and supplementary material shortly.

Thanks the authors for their rebuttal. The rebuttal has addressed my concerns, and I would like to maintain my score of weak accept.

Thank you for your positive feedback.

Reviewer_k7xHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a foundation model-based scale propagation method for metric depth completion. Although depth estimates from foundation models generally provide accurate relative depth, their metric accuracy remains limited. Therefore, aligning these estimates with sparse depth measurements to recover metric scale is essential. To address this challenge, the proposed method first performs initial scale alignment, followed by the selection of neighboring pixels based on geometric conditions. It then estimates affinities by leveraging 2D and 3D geometric relationships and applies depth propagation. The proposed method is trained on NYUv2 and KITTI DC and evaluated on various datasets to assess its generalization capability. Experimental results demonstrate that the proposed approach achieves performance comparable to existing methods while using fewer parameters.

Critical strengths
  • The problem addressed in this paper is practical and relevant to real-world applications.
  • The proposed 2D/3D neighbor selection strategy based on geometric relationships is reasonably designed and well-motivated.
  • The proposed method is trained on NYUv2 and KITTI and evaluated on various datasets, providing empirical evidence of its scalability across different datasets.
Critical weaknesses
  • The paper does not provide a sufficiently detailed analysis of the 3D scene scale distortion caused by sparse depth measurements in the 2D image plane, despite emphasizing this issue in the main contributions.
  • Although Fig. 1 provides a graphical illustration, the explanation of the pairwise geometric relationships in Sec. 3.2 should be further elaborated to provide a more intuitive understanding of the proposed approach.
  • Although the proposed method achieves a relatively high average ranking across various datasets, its performance appears to vary across datasets and evaluation metrics, indicating some inconsistencies in its effectiveness.
  • The ablation studies are primarily presented through quantitative results, with limited discussion of the underlying reasons for the observed performance differences. More detailed analyses are needed to better explain the contributions and effectiveness of individual components.
Questions for authors
  • L40–L41: Usage of \citep and \citet is incorrect.
  • The paper should provide a more detailed explanation of the scale ambiguity and distortion arising from different spatial patterns of sparse depth measurements, as highlighted in the main contributions. In particular, additional analyses are needed to investigate whether certain sampling patterns can induce global scale drift, whether their effects are more pronounced under specific spatial distributions, and how these effects vary depending on the underlying scene structure.
  • In Eq. 1, the linear scaling process may be highly sensitive to depth measurements at extremely close or distant ranges, potentially affecting the overall accuracy of metric scale alignment. The authors should investigate how robust the subsequent geometric propagation process is to such errors and whether it can effectively mitigate their influence on the final depth estimates.
  • Sec. 3.2: The . at the end of the section title should be removed.
  • In Sec. 3.2, various 2D and 3D geometric cues used in the proposed method should be formally defined, with more detailed explanations of their respective meanings, motivations, and effects. In particular, the authors should clarify how each geometric cue contributes to neighbor selection and affinity estimation, providing a more intuitive understanding of the proposed geometric propagation mechanism.
  • Although the proposed method achieves a relatively high average ranking across the evaluated datasets, its performance appears to vary considerably depending on the dataset and evaluation metric. A more detailed analysis is needed to identify the underlying causes of these variations. In particular, it would be valuable to investigate how domain gaps between the training datasets (i.e., NYUv2 and KITTI) and the evaluation datasets affect the effectiveness and robustness of the proposed scale propagation approach.
3 comments

First, we sincerely appreciate the reviewers’ constructive comments. All these comments have been carefully studied and completely addressed in the revised paper. These comments helped us significantly improve the quality of the paper. Most major revisions are marked in red in the revised paper.

Analysis of Scale Ambiguity under Different Sparse Depth Patterns

Weakness 1

“ The paper does not provide a sufficiently detailed analysis of the 3D scene scale distortion caused by sparse depth measurements in the 2D image plane...”

Question 2

“The paper should provide a more detailed explanation of the scale ambiguity and distortion arising from different spatial patterns of sparse depth measurements, as highlighted in the main contributions. In particular, additional analyses are needed to investigate whether certain sampling patterns can induce global scale drift, whether their effects are more pronounced under specific spatial distributions, and how these effects vary depending on the underlying scene structure.”

Response:

First, we have provided a more detailed explanation of the scale ambiguity and distortion arising from 2D sparse depth patterns. Specifically, we clarify how perspective projection leads to different 3D geometric relationships for similar 2D sparse patterns, resulting in different scale correspondences with dense relative depth.

Please find the revisions in the Introduction (Page 1) as follows:

(Page 1) INTRODUCTION

“In this paper, we identify that sparse depth patterns in the 2D image plane can lead to ambiguity in recovering the metric scale of 3D scenes during depth completion. This ambiguity arises from perspective projection, where similar 2D sparse depth patterns can correspond to different 3D geometries. Consequently, relying solely on 2D spatial relationships to fuse metric scale information may lead to unreliable associations between sparse metric measurements and dense relative depth predictions, resulting in inaccurate metric scale recovery and distorted 3D geometry.”

Second, we have added experiments evaluating global alignment across five unseen datasets under four sparse sampling patterns: low-density random sampling (0.1%), high-density random sampling (1%), Harris and Hole. The results are summarized as follows:

Global alignment exhibits substantially larger errors under certain sampling patterns, such as Harris. Compared with 1% random sampling, Harris increases Rel from 8.89 to 25.92 on DIODE indoor and from 40.06 to 60.38 on DIODE outdoor. In contrast, varying random sampling density from 1% to 0.1% yields similar alignment errors, while Hole sampling produces errors comparable to random sampling. These results suggest that global alignment is particularly sensitive to specific spatial distributions, rather than being determined solely by the number of sparse measurements.

The impact of Harris sampling on global alignment varies substantially across different scene structures. Compared with 1% random sampling, Harris increases Rel by 17.03 and 20.32 on DIODE indoor and outdoor, respectively, but by only 0.98 and 1.18 on Stanford and Middlebury.

Detailed results and analyses are provided in Supplementary Sec. A.1 and Table 5.

Sensitivity to Extreme Depth Measurements

Question 3

“In Eq. 1, the linear scaling process may be highly sensitive to depth measurements at extremely close or distant ranges, potentially affecting the overall accuracy of metric scale alignment. The authors should investigate how robust the subsequent geometric propagation process is to such errors and whether it can effectively mitigate their influence on the final depth estimates.”

Response:

We conducted additional depth-range sensitivity experiments on NYUv2 and KITTI, with sparse measurements separately sampled from near, middle, and far depth ranges.

The results show that the linear scaling process is highly sensitive to measurements at extreme depth ranges, particularly distant ranges, while subsequent geometric propagation can substantially mitigate their influence on the final depth estimates. Specifically, on NYUv2, compared with linear scaling using the full set of sparse measurements (Rel = 7.82), using only far-range or near-range measurements increases Rel to 35.50 and 24.73, respectively. Compared to use the same depth-range-specific global alignment, GeoSP with the original complete sparse metric measurements maintains stable Rel values of 2.39--2.67 on NYUv2 and 1.55--1.97 on KITTI. These results demonstrate that geometry-aware local scale recovery can substantially mitigate errors introduced by depth-range-biased global alignment when the original sparse metric observations remain available.

Detailed experimental settings and quantitative results are provided in Supplementary Sec. A.3 and Table 7.

AuthorsOpenReview ↗

Clarification of 2D and 3D Geometric Cues and Analysis of Cross-Domain Performance Variations

Clarification of 2D and 3D Geometric Cues

Weakness 2:

“Although Fig. 1 provides a graphical illustration, the explanation of the pairwise geometric relationships in Sec. 3.2 should be further elaborated to provide a more intuitive understanding of the proposed approach.”

Question 5:

“ In Sec. 3.2, various 2D and 3D geometric cues used in the proposed method should be formally defined, with more detailed explanations of their respective meanings, motivations, and effects. In particular, the authors should clarify how each geometric cue contributes to neighbor selection and affinity estimation, providing a more intuitive understanding of the proposed geometric propagation mechanism.”

Response

First, we have provided an overall explanation of the geometric relationships and their roles in the proposed geometric propagation mechanism in Sec. 3.2 (Page 5). Specifically, the pairwise geometric cues do not participate in neighborhood selection, which is performed based on 3D Euclidean distance. Instead, these cues are extracted between the selected neighbors and the query point and used for affinity prediction. The 2D cues characterize structural priors from relative depth and Image-plane positional priors, while the 3D cues incorporate sparse metric measurements to capture 3D geometric priors in 3D scene.

Second, we have provided formal definitions and detailed explanations of each geometric cue in Supplementary Sec. A.2. Specifically, we formally define each 2D and 3D cue and explain its geometric meaning, design motivation, and specific role in affinity prediction.

Please find the corresponding revisions in Sec. 3.2 (Page 5) and Supplementary Sec. A.2, with the revised text from Sec. 3.2 provided as follows:

(Page 5) Sec. 3.2

"The 2D geometric cues include image-plane offsets, relative-depth differences, local depth gradients, and curvature differences. They are derived from the image-plane coordinates of qq and ii, together with their corresponding globally aligned dense depth values D~m(q)\widetilde D_m(q) and D~m(i)\widetilde D_m(i).The dense depth map provides image-plane arrangement and relative depth structure, enabling characterization of 2D spatial relationships.

"However, relying solely on 2D spatial relationships to fuse metric scale information may lead to unreliable associations between sparse metric measurements and dense relative depth predictions. Furthermore, relative depth lacks metric scale information. Therefore, we further extract 3D geometric relations between the query point and sparse metric depth points in the shared 3D space to explicitly characterize their spatial relationships and provide more reliable geometric cues for metric scale recovery."

Analysis of Cross-Domain Performance Variations

Weakness 3:

“Although the proposed method achieves a relatively high average ranking across various datasets, its performance appears to vary across datasets and evaluation metrics, indicating some inconsistencies in its effectiveness.”

Question 6:

“Although the proposed method achieves a relatively high average ranking across the evaluated datasets, its performance appears to vary considerably depending on the dataset and evaluation metric. A more detailed analysis is needed to identify the underlying causes of these variations. In particular, it would be valuable to investigate how domain gaps between the training datasets (i.e., NYUv2 and KITTI) and the evaluation datasets affect the effectiveness and robustness of the proposed scale propagation approach.”

Response:

We have conducted additional experiments to investigate how domain gaps affect the cross-domain performance of GeoSP by comparing global alignment alone with full GeoSP on four unseen raw-sensor benchmarks (DIML, SUN RGB-D, TOFDC, and VOID1500).

GeoSP consistently improves depth accuracy and 3D reconstruction over global alignment reducing MAE by 47.4%--79.8% and Ov. by 48.1%--75.2% over GAO, demonstrating robust cross-domain metric scale recovery. The results suggest that variations in cross-domain performance are associated with the quality of the globally aligned relative-depth prior across unseen domains. Specifically, DIML and SUN RGB-D exhibit larger global alignment errors (MAE =1438.7 and 2022.0 mm), with corresponding GeoSP errors of 383.0 and 408.0 mm. In contrast, TOFDC and VOID1500 show lower global alignment errors (76.2 and 89.3 mm), accompanied by lower GeoSP errors (40.1 and 36.2 mm).

The detailed quantitative comparisons and discussion are provided in Supplementary Sec. A.4 and Table 7.

Formatting Corrections

Question 1:

“ L40–L41: Usage of \citep and \citet is incorrect.”

Question 4:

“ Sec. 3.2: The . at the end of the section title should be removed.”

Response: We have corrected the citation commands at L40–L41 and removed the unnecessary period at the end of the Sec. 3.2 title.

AuthorsOpenReview ↗

Further Analysis of Ablation Studies

Weakness 4:

“The ablation studies are primarily presented through quantitative results, with limited discussion of the underlying reasons for the observed performance differences. More detailed analyses are needed to better explain the contributions and effectiveness of individual components.”

Response:

We have expanded the ablation analysis to better explain the underlying reasons for the observed performance differences and clarify the contributions of individual components from four perspectives: (1) geometric relations, (2) training supervision, (3) geometric affinity prediction, and (4) affinity-weighted solver.

The detailed analyses are provided in Sec. 4.3.2 (Page 10) of the revised manuscript, as follows:

(1) Ablation on geometric relations.

We first examine the geometric relations introduced in Sec. 3.2. A0 removes the explicit 3D features while keeping depth-related cues and 2D position, while A1 further removes the depth cues and keeps only 2D position. Both variants retain the same 3D neighborhood selection and affinity-weighted solver as the full model, isolating the effects of geometric relations extracted from 3D neighbors. The better performance of A0 over A1 highlights the benefit of depth cues beyond 2D position, while the further gain of the full model demonstrates that 3D geometric relations improve affinity prediction by capturing spatial relationships beyond 2D cues.

(2) Ablation on training supervision.

We study the contribution of different supervision terms in Table 3b. B0 removes the 3D loss (L3D\mathcal{L}_{3D}), while B1 removes the low-frequency loss (Llow\mathcal{L}_{low}). Both lead to small performance drops, increasing NYU Rel from 1.27 to 1.32/1.33 and KITTI Rel from 3.14 to 3.24. These results suggest that the two losses provide complementary supervision: L3D\mathcal{L}_{3D} encourages 3D geometric consistency, while Llow\mathcal{L}_{low} promotes spatially coherent metric scale recovery.

(3) Ablation on geometric affinity prediction.

We study the three inputs to the affinity head in Sec. 3.3 and Table 3c. Removing RGB in C0 causes a clear performance drop, while removing either gg or ϕ\phi in C1/C2 changes Rel by at most 0.03. This suggests that RGB provides complementary appearance cues for affinity estimation, whereas the limited impact of removing gg or ϕ\phi may result from their partially redundant geometric information and the 3D geometry already used in neighborhood selection.

(4) Ablation on affinity-weighted solver.

We study metric scale recovery using the affinity-weighted solver in Table 3d. D0 replaces the depth-ratio scale constraint (sis_i) in Eq. (5) with 3D point-space transfer, which adds the estimated relative 3D displacement from ii to qq to the metric point at ii and projects it onto the query ray. This increases NYU Rel from 1.27 to 1.54 and Ov. from 31.5 to 34.5. On KITTI, Rel increases from 3.14 to 5.60 and Ov. from 288.6 to 383.6. These results suggest that depth-space transfer is more effective by directly leveraging relative-depth ratios while avoiding potential errors from 3D displacement estimation.

Finally, we sincerely hope that our clarifications and additional experiments have addressed your concerns. We will upload the revised manuscript and supplementary material shortly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.