- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ContextDepth. During the inference of one image, ContextDepth first generates 3 outpainted images using OneReward and runs a off-the-shelf depth model on each. ContextDepth then selects a candidate based on the depth model's error on an retrived RGB - depth sample from an external bank (by default, mixing all 6 datasets). Experiments cover 6 models and 6 datasets but mainly on centrally cropped images.
Metrics depth models' performances are commonly much worse on boundaries than central areas. But this paper is trying to hiding this. The outpainting of 1 single case contributes much more than selecting from multiple candidates. The most significant gains are on centrally cropped images but not on full images. And not all depth models yield improved performance when this method is applied. I lean towards clear rejection of this paper.
Critical strengths
-
Experimental results show significant gains. In table 1, DA3 without intrinsics was improved from 51.17% to 68.47% in with one outpainting and 73.82% with selection.
-
In figure 3, outpainting beats all other padding controls, which supports the importance of outpainting meaningful content.
-
Table 2 compares default mixed bank, cross-dataset bank, and single-dataset bank, which clearly tells the effect when retriving from different references.
Critical weaknesses
-
The main results, which supports the "significant gains", is conducted on images that are centered cropped to 2/3 along each dimension. However, in full-image experiments, 3 of 6 models deliver worse results when the intrinsics are unknown. Yes, all 6 improve when intrinsics are known in full-image experiments, but the improvements are little. Whether such improvements are worthwhile under the huge additional computational costs and much slower speed is questionable. 3 outpainting using the full multi-step denoising, the latency of a single denoising step can be enough for some depth mdoels to predict depth for an image, let along the retriving, multiple depth model inferencing, and selecting. These significnatly limits the evidence for the claim of this paper. Though ignoring these limitations, figure 1's "resolving scale ambiguity" should also be narrowed down to "reducing" not "resolving".
-
The selection's contribution also needs more clarification. In table 1's without intrinsics branch, 1 outpainting contributs much more than selecting from 3. However, the full method includes 3 outpaintings, 3 retrivings, and 3 + 3 (3 outpainted images and 3 retrived images) depth model infereces. The latencies of each stage and the entire end-to-end pipeline are missing. The sweep of the number of candidates is also missing. Table 3 rescales "Query-only" predictions, however, the proposed method selects among predictions on outpainted images, and therefore does not cleanly isolate the benefit of selection over rescaling.
Questions for authors
see weaknesses
We thank all three reviewers for their careful evaluation and constructive feedback. We have added experiments, updated results, and clarified the method and protocol. We first address shared concerns.
For concerns about cropped inputs and full-image results, we added real-context and camera-controlled experiments to test how existing context affects gains and whether camera re-estimation after expansion causes degradation. We updated Tables 1 and 10 and examined results from a single outpainting to clarify whether gains depend on cropping and when the method is applicable.
The real-context experiments cover 3,406 images from six datasets, DIODE, ETH3D, HAMMER, KITTI, NYUv2, and iBims-1. Starting from a central crop of approximately two-thirds of the original width and height, we restore 25%, 50%, 75%, and 100% of the removed periphery, producing five nested inputs ending with the full image. All added pixels come from the same original image, with original sampling and no extra resizing, generated content, or selector. We test all six models with and without GT intrinsics, giving 12 model and camera combinations.
For each of the four adjacent input pairs, we infer depth from both inputs and crop the larger prediction back to the smaller input's region. We compare δ1 and AbsRel on exactly the same valid GT pixels, with equal weight per image, to measure the benefit of added surroundings. With GT intrinsics, calibration always accounts for cropping, canvas expansion, and internal resizing. Without GT intrinsics, both Uni models reuse the smaller query's estimated camera, mapped to the larger canvas; the other four retain native camera recovery.
Figures 9 and 10 on pages 26 and 27 of the revised manuscript show that, with GT intrinsics, all five models except Depth Pro improve both metrics on average at every expansion. UniDepth V2, DA3, and UniK3D generally show diminishing gains; the MoGe models maintain more consistent positive gains. Depth Pro's gains fluctuate near zero even with GT intrinsics, indicating little consistent benefit from extra context. Without GT intrinsics, it independently predicts focal length and canonical inverse depth, then converts to metric depth using the focal-length/image-width ratio. Focal-length errors propagate directly into global scale errors. As expansion changes these predictions, camera-induced scale changes can outweigh the limited contextual gains. Limited context response and sensitivity to camera errors together explain its weaker performance under ContextDepth; similar degradation occurs with real context. The Uni models instead feed camera rays into depth decoding; MoGe and DA3 recover scale through shared scene representations and geometric or metric branches. Camera errors thus follow different paths. See Appendix G.2 on page 27, and G.5 on page 30 for Depth Pro controls and implementation checks.
To address z6mU's concern about FOV and camera bias, we keep generated content unchanged and control the cameras of both Uni models without GT intrinsics for a fairer comparison. We map the Query-only camera to the expanded canvas to generate decoding rays, replacing the camera re-estimated from the expanded image. We control the Uni models because their depth decoders explicitly use camera estimates; the other four retain native inference. Changing only camera handling turns UniK3D and UniDepth V2's full-image seed-0 δ1 changes from −9.48 and −3.96 in original Table 10 on page 21 to +0.18 and +1.74 in Table 14, respectively. Crop gains remain +11.88 and +7.85. Updated versions of Tables 1 and 10 appear on pages 28 and 29. These two degradations thus arose from adverse camera re-estimation that obscured contextual gains. Controlling it preserves gains on both crops and full images.
Whether smaller full-image gains justify extra computation and latency should be assessed across input conditions: marginal gains are smaller on full images and much larger on central crops. Tight framing or post-capture cropping produces valid RGB inputs, and users cannot judge context sufficiency in advance. For the five models that use context, a single outpainting already gives substantial crop gains and largely preserves or improves full-image accuracy, with only small declines in a few metrics. Reference-guided selection further improves accuracy and stability. We therefore support ContextDepth for these five models in both camera settings, with or without the selector, without requiring per-image context assessment. Depth Pro without GT intrinsics is excluded from this default recommendation for the two reasons above.