- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GeoSP for dense depth completion from an RGB image and sparse metric measurements using a frozen depth foundation model. Following global scale-and-shift alignment, the method selects sparse neighbors by 3D distance within a 2D candidate pool. A lightweight affinity network combines foundation-model features, point properties, and geometric relations to estimate propagation weights. An affinity-weighted least-squares solver then aggregates neighboring depth ratios to recover local scale and correct the dense depth. The model is trained on simulated sparse inputs from NYUv2 and KITTI and evaluated on six simulated-input and four raw-sensor datasets, achieving strong average ranks, particularly on 3D metrics.
Critical strengths
The evaluation spans diverse scenes and sensors and provides practically useful results. Training uses only NYUv2 and KITTI, while Tables 1–2 cover ten target datasets and distinguish simulated sparse patterns from raw sensor inputs. GeoSP achieves overall average ranks of 1.33, 1.56, and 2.04 for simulated indoor, simulated outdoor, and raw-input evaluations, respectively. Although it is not best on every individual metric, the strong aggregate performance across these settings supports the practical value of the completion strategy.
The lightweight affinity network and explicit scale aggregation form a clear and reasonably interpretable design. Freezing the foundation model and learning only constraint weights preserves an explicit connection between the output and sparse measurements, with only 0.905M additional trainable parameters. The geometric-feature, supervision, and depth-space-transfer ablations in Table 3 also provide positive evidence. This is a promising simple design, and the simplicity of the weighted solver alone does not diminish its potential contribution.
Critical weaknesses
The depth representation and alignment underlying the shared 3D space are insufficiently specified, affecting the interpretation of the core geometry. Equation (1) applies depth-domain scale-and-shift alignment to the foundation-model output, whereas the relative Depth Anything V2 model predicts affine-invariant inverse depth. The paper does not specify whether is the raw output, a transformed depth map, or a prediction from another checkpoint, nor the fitting objective for and validity handling. If alignment occurs in inverse-depth space followed by conversion, this should be stated; if raw output is directly fitted affinely to metric depth, the rationale needs explanation. This definition also determines the meaning of the 3D neighborhoods, normals, and depth ratios in Eq. (5). The full transformation and handling of nonpositive or near-zero depths should be clarified.
The ablations do not sufficiently isolate the central contribution of geometry-guided scale propagation. PSD already uses foundation-model guidance and propagation in 3D space, so the additional benefit over this class of methods requires controlled evidence. When Table 3a removes 3D features, it is unclear whether 3D neighborhood selection remains. Table 3d compares depth-space and point-space transfer, but lacks direct references such as global alignment, uniform scale aggregation, or 2D neighborhoods with the weighting otherwise fixed. The gains therefore remain incompletely attributed between neighborhood selection, local ratio propagation, and learned geometric weights. Moreover, the ablations use only the training domains, NYU/KITTI, while the central claim concerns OOD robustness, and p. 7 explicitly notes a training-resolution difference from PSD. The ten-dataset results are valuable, but do not independently establish the proposed mechanism or its cross-domain advantage. Key controls with matched foundation models and preprocessing would strengthen this attribution and help align the conclusions with the evidence.
Questions for authors
These clarification requests could change my assessment. Implementation details or existing results should take priority; a new large-scale benchmark expansion can be left for future work.
Which Depth Anything V2 checkpoint and representation define ? Please give the complete sequence and objective for raw prediction, scale/shift fitting, depth conversion, and back-projection, including the handling of nonpositive or near-zero denominators in Eq. (5). Do A0/A1 in Table 3a still select neighbors by 3D distance? Apart from removing input features, which geometric operations remain, and which specific component does this ablation isolate? Are controls using global alignment alone, uniform ratio aggregation, or a 2D/3D neighborhood substitution with other settings fixed already available? If so, can the most informative existing control be reported on a target-domain or raw-sensor setting? Which PSD foundation model and checkpoint are used in Tables 1–2, and do they match GeoSP? How do training/test resolution, intrinsic transformations, and sparse-input handling differ? Are existing results available that separate these factors?
5 comments
Response to Concerns about Controlled Ablations and OOD Generalization
Weakness 3:
“Table 3d compares depth-space and point-space transfer, but lacks direct references such as global alignment, uniform scale aggregation, or 2D neighborhoods with the weighting otherwise fixed. The gains therefore remain incompletely attributed between neighborhood selection, local ratio propagation, and learned geometric weights. Moreover, the ablations use only the training domains, NYU/KITTI, while the central claim concerns OOD robustness.”
Question 3:
“Are controls using global alignment alone, uniform ratio aggregation, or a 2D/3D neighborhood substitution with other settings fixed already available? If so, can the most informative existing control be reported on a target-domain or raw-sensor setting?”
Response
Following the reviewer's suggestion, we have added controlled ablation studies to better attribute GeoSP's gains to neighborhood selection, local ratio propagation, and learned geometric weights, and to validate their contributions under OOD settings.
First, we have added three controlled baselines: global alignment only, uniform affinity aggregation, and 2D neighborhood selection, while keeping the remaining components unchanged where applicable. The results show that local ratio propagation substantially improves over global alignment, while 3D neighborhood selection and learned geometric weights provide complementary benefits.
Second, we have extended these controlled comparisons to four unseen raw-sensor benchmarks: SUN RGB-D, TOFDC, DIML Indoor, and VOID1500. All configurations follow the original NYUv2–KITTI training protocol without target-domain fine-tuning. The results further support the effectiveness of geometry-guided scale propagation under OOD settings.
Please find the revisions on Page 8, Table 3, and Sec. 4.3.1 of the revised manuscript, as follows:
“We evaluate the core mechanisms on four unseen raw-sensor benchmarks: SUN RGB-D, TOFDC, DIML Indoor, and VOID1500. Specifically, we isolate the contributions of neighborhood selection and affinity prediction through controlled comparisons: comparing 2D and 3D neighborhoods while keeping learned affinity prediction fixed, and comparing uniform and learned affinities while keeping neighborhood selection fixed. Global alignment without local scale propagation is also included as a reference baseline. All configurations follow the NYUv2–KITTI training protocol without target-domain tuning or adaptation.
Table 3. Core Mechanism Ablation on Unseen Raw-Sensor Benchmarks.
| Method | Neighborhood | Affinity | SUN RGB-D Rel ↓ | SUN RGB-D Ov. ↓ | TOFDC Rel ↓ | TOFDC Ov. ↓ | VOID1500 Rel ↓ | VOID1500 Ov. ↓ | DIML Rel ↓ | DIML Ov. ↓ |
|---|---|---|---|---|---|---|---|---|---|---|
| Global Alignment | — | — | 12.07 | 1008.6 | 4.58 | 60.7 | 5.19 | 56.8 | 19.30 | 483.0 |
| 2D + Uniform | 2D | Uniform | 4.16 | 262.1 | 2.53 | 33.7 | 3.92 | 36.9 | 16.58 | 136.8 |
| 2D + Learned | 2D | Learned | 4.02 | 251.8 | 2.48 | 32.5 | 3.12 | 35.3 | 16.28 | 134.9 |
| 3D + Uniform | 3D | Uniform | 3.70 | 252.1 | 2.41 | 31.6 | 3.31 | 34.6 | 15.06 | 132.9 |
| Full GeoSP | 3D | Learned | 3.62 | 250 | 2.32 | 31.5 | 2.12 | 25.2 | 14.60 | 130.0 |
Both 3D neighborhood selection and learned affinity prediction consistently improve Rel and Ov. across all four benchmarks. Combining them yields the best performance across all eight benchmark–metric combinations, demonstrating their complementary contributions to zero-shot metric scale recovery.”
Reponse to concern about fairness of Comparison with PSD
Fairness of Comparison with PSD
Weakness 4:
"and p. 7 explicitly notes a training-resolution difference from PSD. The ten-dataset results are valuable, but do not independently establish the proposed mechanism or its cross-domain advantage. Key controls with matched foundation models and preprocessing would strengthen this attribution and help align the conclusions with the evidence."
Question 4:
" Which PSD foundation model and checkpoint are used in Tables 1–2, and do they match GeoSP? How do training/test resolution, intrinsic transformations, and sparse-input handling differ? Are existing results available that separate these factors?"
Response
First, we clarify that PSD uses the same pretrained Depth Anything V2-Large checkpoint as GeoSP, enabling a backbone-matched comparison in Tables 1–2.
Second, we clarify that all compared methods follow the same test-time evaluation protocol, including identical test images, sparse-depth observations, evaluation resolutions, and valid-pixel masks. For methods requiring camera intrinsics, we use dataset-provided calibration with corresponding transformations.
Finally, regarding training configurations, PSD uses a broader KITTI training set with fixed-size cropping and resizing, whereas GeoSP is trained on NYUv2 and the KITTI Eigen subset at their original resolutions. Although these training differences remain, the shared test-time protocol ensures consistent evaluation conditions.
Please find the clarification in Sec. 4.1 (Implementation Details) of the revised manuscript, as follows:
“ All compared methods follow the same evaluation protocol as PSD, using consistent test images, sparse-depth observations, evaluation resolution, and valid-pixel masks. For methods requiring camera intrinsics, we use the dataset-provided calibration with corresponding transformations.”
We sincerely thank the reviewer for highlighting these clarification requests. We have prioritized clarifying the implementation details and providing controlled ablation results to directly address the concerns. We hope these revisions help resolve the remaining questions and support a reassessment of our work. We welcome any further questions or discussion. We will upload the revised manuscript and supplementary material shortly.
We sincerely appreciate the reviewers’ constructive comments. All these comments have been carefully studied and completely addressed in the revised paper. These comments helped us significantly improve the quality of the paper. Most major revisions are marked in red in the revised paper.
Depth Representation and Alignment
Weakness 1:
"The depth representation and alignment underlying the shared 3D space are insufficiently specified, affecting the interpretation of the core geometry. Equation (1) applies depth-domain scale-and-shift alignment to the foundation-model output, whereas the relative Depth Anything V2 model predicts affine-invariant inverse depth. The full transformation and handling of nonpositive or near-zero depths should be clarified."
Question 1:
" Which Depth Anything V2 checkpoint and representation define the relative depth? Please give the complete sequence and objective for raw prediction, scale/shift fitting, depth conversion, and back-projection, including the handling of nonpositive or near-zero denominators in Eq. (5)."
Response
First, we have clarified that GeoSP uses the frozen Depth Anything V2-L checkpoint, which predicts affine-invariant inverse depth rather than metric depth.
Second, we have added Eqs. (1)–(3) to explicitly describe the full transformation from inverse-depth prediction to the shared 3D space, including the corrected alignment formulation. Specifically, Eq. (1) defines the least-squares objective for global affine alignment in the inverse-depth domain, Eq. (2) converts the aligned inverse depth to metric depth, and Eq. (3) back-projects the aligned dense depth and sparse metric depth into the shared 3D space.
Finally, we have clarified the handling of nonpositive or near-zero inverse-depth values by applying a minimum threshold of 10−4 to ensure numerical stability during depth conversion and subsequent scale propagation.
Please find the revisions in Sec. 3.1 on Page 3, as follows.
(Page 3) 3.1 OVERVIEW
"Specifically, we perform Global Alignment in disparity space by fitting a global scale γ and shift ρ to sparse depth measurements:
(\gamma^{\ast},\rho^{\ast}) = \arg\min_{\gamma,\rho} \sum_{i\in\Omega_s} \left(\gamma R_m(i)+\rho-\frac{1}{D_s(i)}\right)^2 \tag{1}
where Ωs denotes the set of valid sparse measurements used for alignment. The aligned inverse depth is then converted into depth: \widetilde{D}_m(q) = \frac{1}{\gamma^* R_m(q) + \rho^*} \tag{2} For numerical stability, the aligned disparity is clamped to a minimum of 10−4 before inversion, while the resulting depth is also clamped to a minimum of 10−4 for subsequent scale propagation.
We then back-project the aligned depth Dm and sparse metric depth Ds using the camera intrinsic matrix K:
\qquad P_s(i)=D_s(i)K^{-1}\bar i. \tag{3}$$ *where $\bar q$ and $\bar i$ are homogeneous pixel coordinates."* ### Depth Representation and Alignment **Weakness 2:** *"The ablations do not sufficiently isolate the central contribution of geometry-guided scale propagation. PSD already uses foundation-model guidance and propagation in 3D space, so the additional benefit over this class of methods requires controlled evidence. When Table 3a removes 3D features, it is unclear whether 3D neighborhood selection remains."* **Question 2:** *"Do A0/A1 in Table 3a still select neighbors by 3D distance? Apart from removing input features, which geometric operations remain, and which specific component does this ablation isolate?"* ### Response We clarify that **A0/A1 in Table 3a still select neighbors by 3D distance**. Specifically, the removed 3D features refer to the 3D geometric relations extracted from the selected neighbors in Sec. 3.2, rather than the 3D neighborhood selection itself. Thus, these ablations isolate the effects of geometric relations on affinity prediction. Please find the clarification in **Sec. 4.3.2** as follows: *“Both variants retain the same 3D neighborhood selection and affinity-weighted solver as the full model, isolating the effects of geometric relations extracted from 3D neighbors.”*