acceptodds
Under review as a conference paper at ICLR 2027

Depth Pro 2: Sharp and Temporally Consistent Metric Monocular Depth

36 views

Abstract

We present Depth Pro 2, an approach to estimating dense metric depth for images and video at full resolution. By analyzing prior approaches to monocular depth estimation, we find that they commonly trade off temporal consistency and boundary sharpness – multi-frame methods favor consistency, single-frame methods favor sharpness. To overcome this trade-off, we train a model that processes one or multiple images and predicts depth maps at native resolution. The network architecture is designed for simplicity and scalability, and is agnostic to whether the input is a single image or a video volume – the same network processes images and video. Depth Pro 2 produces depth maps that are metrically accurate, sharp, and temporally consistent all at once, outperforming prior work in all of these dimensions. We release a family of models that define a new Pareto frontier in terms of metric accuracy, sharpness, temporal consistency, and runtime.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Monocular Depth Estimation

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_QWyLOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents Depth Pro 2, a monocular metric depth model that achieves improvements over the recent affine literature, in particular regarding sharpness and temporal consistency (if a sequence of frames is provided). The authors claim that achieving sharpness and temporal consistency at the same time is an open problem, where prior work focuses only on one of the two problems (Fig. 2).

The core idea behind this paper is an engineering combination of factors that provide a depth that is both sharp and time-consistent under a unified ViT architecture family. In particular, the paper proposes the following methodology:

  • A camera focal length estimator model to convert normalized inverse depth into metric depth;
  • A frame-permutation-invariant DINOv3 ViT backbone with initial self-attention (first 33% of layers) and alternating self- and cross-attention for the remaining layers for multi-frame consistency;
  • A novel aspect-ratio-aware spatial position encoding for image tokens;
  • A modified DPT head with transposed conv for feature upsampling;
  • Training using a large amount of synthetic and real datasets (>50M samples) and a large computing cluster (i.e., 128 NVIDIA B200s);
  • A resolution-preserving augmentation that avoids interpolation to maintain GT sharpness;
  • A multi-view self-consistency loss based on optical flow;
  • Evaluating at full resolution to maintain the details of the original image, thus increasing sharpness.

The authors compared the proposal to several single-image depth estimation works (Tab. 1) and time-consistent depth estimation works (Tab. 2), comparing both accuracy (Tables 1-2) and sharpness (Tables 3.a-3.b). The benchmarks show that Depth Pro 2 achieves the best average rank on all tables.

Critical strengths

Major

  • State-of-the-art results for sharpness and time-consistency at the same time. The tables speak for themselves: the proposed Depth Pro 2 achieves the best average rank on all tables of the main paper.
  • Interesting engineering choices. Given the frame-permutation-invariant and aspect ratio-preserving spatial position encoding, the self- and cross-attention alternating strategy, the native resolution processing, the DPT with transposed convolutions, and the multi-view self-consistency, the authors managed to create a very powerful depth estimation model.

Minor

  • Easy to follow. The paper is easy to read and well structured, with clear explanations of the method and the experiments. The qualitative results are impressive and show the sharpness improvements. The anonymous website is also very interesting and shows the capabilities in video.
Critical weaknesses

Major

  • Novelty is limited. A lot of the engineering strategies used in the paper are already studied in the literature. For example, the alternating self- and cross-attention for the remaining layers is already presented in DepthAnythingV3, the focal-invariant normalization and FoV estimator model are already presented in DepthPro, and the self-consistency loss is adapted from Luo et al.

  • Native Resolution claim needs more clarification. The claim itself sounds. The problems are related to the experimental/pratical part: 1) It is not clear if increasing resolution for other baselines increases sharpness as well as for DepthPro 2 (Tab. 14). The authors should investigate this with a similar table as Tab. 14 but using baselines. If the additional experiments confirm this, other baselines in Tab. 3 should be re-evaluated using native resolution for a fairer comparison. Tab. 17 suggests that this is not always the case: the Bokeh performances degrade for DAv3 if using higher res. 2) Processing at native resolution is expensive (for high-res inputs), thus requiring hardware (e.g., NVIDIA B200) not available to the majority of people. The authors could add an additional table (similar to Tab. 7) with runtime on lower-tier hardware and/or additional tables (similar to Tab.7) but with other baselines.

Questions for authors
  • Please consider adding additional tables as requested in the weaknesses section, in particular regarding the native resolution.
  • Row 128: "Depth Pro predicts focal-length-normalized inverse depth together with focal length,[...]" -> "Depth Pro 2 predicts focal-length-normalized inverse depth together with focal length,[...]" ?
  • Arkitscenes is cited two times.
  • Row 1722: "The camera camera" -> "The camera model"?
3 comments

Thank you for the thoughtful review and the concrete suggestions. We are glad you find Depth Pro 2 "a very powerful depth estimation model" and that its "qualitative results are impressive". We ran each experiment you suggested. The supplementary QWyL.pdf contains the full tables.

W1: novelty. Please see our general response.

W2.1: higher resolution for baselines.

"It is not clear if increasing resolution for other baselines increases sharpness as well as for DepthPro 2 (Tab. 14). The authors should investigate this with a similar table as Tab. 14 but using baselines. If the additional experiments confirm this, other baselines in Tab. 3 should be re-evaluated using native resolution for a fairer comparison. Tab. 17 suggests that this is not always the case: the Bokeh performances degrade for DAv3 if using higher res."

We thank you for the suggestion. Following the suggestion, we directly re-evaluated Tables 1-4 of the main paper with all methods running at native resolution (except Depth Pro and MetricAnything, which only operate at a fixed resolution). Tables Q1-Q4 report results of the baselines evaluated at native resolution and show the performance delta relative to the results at their default resolution. At native resolution, the baselines produce slightly sharper boundaries but less accurate metric depth. For all tables, Depth Pro 2 either retains or improves its average rank. It remains the best-performing method with the lowest average rank on all tables.

Place of Depth Pro 2-H+ among all methods, ranked by average rank (its average rank in parentheses). Full tables: Tables Q1–Q4 in QWyL.pdf.

TableMetricInputDefault resolutionNative resolution
Q1Metric δ1 w/ GT intrinsicsImage1st (2.4)1st (1.8)
Q1Metric δ1 w/ predicted intrinsicsImage1st (2.9)1st (2.6)
Q2Metric δ1 w/ GT intrinsicsVideo1st (1.0)1st (1.0)
Q2Metric δ1 w/ predicted intrinsicsVideo1st (1.5)1st (1.5)
Q3Boundary F1Image1st (1.6)1st (1.2)
Q3Boundary F1Video1st (1.3)1st (1.0)
Q4JitterVideo1st (1.5)1st (1.5)

W2.2: runtime.

"Processing at native resolution is expensive (for high-res inputs), thus requiring hardware (e.g., NVIDIA B200) not available to the majority of people. The authors could add an additional table (similar to Tab. 7) with runtime on lower-tier hardware and/or additional tables (similar to Tab.7) but with other baselines."

We agree that processing long clips at native resolution can exceed the capabilities of lower-tier hardware, which is why we present a family of models. The family allows selecting an optimal model for specific hardware and task requirements.

As suggested, we benchmark the throughput of our model family on three generations of GPUs. For comparison, Table Q5 reports results on A100 (40GB), H100, and B200 GPUs.

Excerpt of Table Q5: throughput and peak memory of our models on three GPUs, with torch.compile enabled (full table in QWyL.pdf):

ModelInputB200 FPS ↑H100 FPS ↑A100 FPS ↑B200 peak GiB ↓
Depth Pro 2-S1080p53.126.011.05.1
4K3.71.70.819.7
Depth Pro 2-B1080p27.313.05.58.3
4K1.90.8OOM31.9
Depth Pro 2-L1080p10.44.92.111.2
4K0.70.3OOM40.8
Depth Pro 2-H+1080p6.12.91.217.8
4K0.4OOMOOM60.8

We also measured the throughput and peak memory of all baselines operating at native resolution on a B200 GPU (Table Q6). Video models, ours included, process a 32-frame clip in one call; single-image models process its frames one at a time. Depth Pro 2-S achieves a higher throughput than all baselines (except VeloDepth at 4K). At native 1080p, Depth Pro 2-H+ also needs less memory than 8 of the 12 video baselines.

Excerpt of Table Q6, native 1080p, not compiled: the five fastest baselines and ours (full table in QWyL.pdf). ˢ: single-image method, memory for one frame.

MethodFPS ↑Peak GiB ↓
VeloDepth16.811.5
MoGe-2-Lˢ12.44.4
UniDepthV2-Lˢ12.45.7
UniK3D-Lˢ12.35.9
DAGE12.133.2
Depth Pro 2-S47.16.6
Depth Pro 2-H+5.426.8

Minor. Thank you for catching the typos. We have fixed them in the revision.

Dear Authors,

Thanks for the additional PDF file with the required experiments. I'm satisfied with the authors' response to the "Native Resolution claim needs more clarification" weakness: even when running competitors at native resolution, the proposal still ranks first. Also, the larger model can run on lower-tier hardware up to 1080p. Thus, I'm going to raise my score.

Best regards,

Reviewer QWyL

Dear QWyL, Thank you very much!

Reviewer_XT1bOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents Depth Pro 2, a scalable framework for dense metric depth estimation from images and videos at native resolution. The authors identify a key trade-off in existing methods: multi-frame approaches tend to prioritize temporal consistency at the expense of sharp depth boundaries, while single-frame methods often produce sharper details but lack temporal consistency. To address this limitation, Depth Pro 2 adopts a unified and scalable architecture that processes either single images or multiple frames using the same network. This design enables the model to jointly achieve metric accuracy, sharp depth boundaries, and temporal consistency without sacrificing inference efficiency. The proposed model family establishes a new Pareto frontier across these dimensions, outperforming previous methods in depth accuracy, spatial detail, temporal consistency, and runtime.

Critical strengths
  1. The performance is strong and promising, demonstrating the effectiveness of the proposed approach across multiple dimensions, including metric accuracy, depth boundary sharpness, and temporal consistency.

  2. The analysis of the trade-off between depth sharpness and temporal consistency is reasonable and insightful.

  3. The paper is clearly written and easy to follow, with a well-organized presentation of the motivation, method, and experimental results.

Critical weaknesses
  1. Despite its strong practical value, the paper offers limited methodological novelty. The work is more of a comprehensive technical integration of existing ideas than a fundamentally new approach.

  2. Camera pose estimation and 3D reconstruction are underexplored. Given the strong metric depth predictions, it would be interesting to investigate whether the model could also predict camera poses in a feed-forward manner or whether its depth estimates could facilitate more accurate pose recovery through bundle adjustment. Comparisons of camera pose estimation against existing methods would strengthen the evaluation. Furthermore, visual comparisons of reconstructed point clouds using the predicted depth maps and camera intrinsics/extrinsics would help demonstrate the geometric consistency and practical value of the proposed approach.

Questions for authors

See weaknesses above.

1 comment

Thank you for the positive and thoughtful review. We are glad you find the performance "strong and promising" and the work of "strong practical value".

W1: novelty. Please see our general response.

W2: camera poses.

"it would be interesting to investigate whether the model could also predict camera poses in a feed-forward manner or whether its depth estimates could facilitate more accurate pose recovery through bundle adjustment."

We thank the reviewer for the suggestion, but consider camera pose estimation and 3D reconstruction beyond the scope of this paper. We will leave an extension to predict additional modalities or an application to 3D reconstruction to future work.

For an evaluation of the geometric consistency of our work, we refer to the quantitative results on metric and relative depth accuracy, and on temporal consistency, and the qualitative video results. The appendices also provide evaluations on two downstream applications, namely synthetic bokeh rendering, and novel view synthesis, which further demonstrate the practical value of Depth Pro 2.

Reviewer_ELzcOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Depth Pro 2, a framework for estimating sharp, temporally consistent metric depth from a single image or video. Built upon a pre-trained DINOv3 backbone, the method employs alternating intra-frame and cross-frame attention mechanisms to facilitate multi-frame inference, and employs native-resolution processing to preserve fine geometric details. Furthermore, the authors incorporate a multi-view consistency loss and a training strategy that accommodates variable resolutions and sequence lengths. Extensive experiments demonstrate strong performance in terms of metric depth accuracy, boundary sharpness, and temporal consistency.

Critical strengths
  1. This paper addresses an important practical challenge: generating accurate metric depth with sharp boundaries and temporal consistency within a unified image-and-video framework.

  2. The experimental evaluation is comprehensive, covering a wide range of datasets and competitive baselines.

  3. The method achieves impressive overall performance, particularly in boundary sharpness and temporal stability.

Critical weaknesses
  1. Limited methodological novelty relative to existing depth and multi-view foundation models. The proposed architecture is primarily composed of existing components. The alternating application of intra- and cross-frame attention has been extensively explored in multi-view transformers, including VGGT and DA3. The focal-length-normalized inverse-depth formulation and DPT-based decoding also closely follow Depth Pro, while geometric cross-view consistency supervision is well established in video depth estimation.

  2. The temporal consistency evaluation does not fully establish long-term geometric consistency. The primary temporal metric, "jitter," measures frame-to-frame variations in depth prediction errors. While appropriate for evaluating temporal stability, it does not directly capture persistent metric bias, slowly accumulating scale drift, or long-range geometric inconsistency. The paper also employs the TAE metric, but the relative performance is less consistent under this alternative evaluation: Depth Pro 2-H+ achieves an average rank of 1.5 in Table 4, compared with 7.1 in Table 11.

Questions for authors
  1. Regarding methodological novelty, could the authors clarify the key technical differences between Depth Pro 2 and existing approaches such as Depth Pro and Depth Anything 3?

  2. Although the authors discuss the limitations of TAE, this discrepancy suggests that the conclusions drawn are highly sensitive to the specific definition of "temporal consistency" employed. Additional experiments examining long-range depth alignment, scale stability, and geometric consistency over increasing temporal intervals would provide stronger support for the central claims.

1 comment

Thank you for the thoughtful review. We are glad you find that the paper "addresses an important practical challenge" and that the method "achieves impressive overall performance, particularly in boundary sharpness and temporal stability".

W1/Q1: Differences to Depth Pro and DA3.

We address this question in the general response regarding technical novelty.

W2/Q2: Evaluation of long-term temporal consistency.

"The temporal consistency evaluation does not fully establish long-term geometric consistency. [...] Additional experiments examining long-range depth alignment, scale stability, and geometric consistency over increasing temporal intervals would provide stronger support for the central claims."

In the paper, we only claim that Depth Pro 2 is locally temporally stable (i.e. the predictions do not "jitter"), not that it exhibits long-term geometric consistency. We will further disambiguate our phrasing.

Following the reviewer's suggestion, we conduct an additional experiment to evaluate consistency over increasing temporal intervals. Specifically, we measure jitter, TAE and scale consistency over increasingly large inter-frame distances, from 1 to 128 frames. The measurements are made on 240-frame videos from Spring, which is the highest-quality dynamic dataset among our evaluation datasets. To measure scale consistency, we align the predicted depth for each frame to its corresponding ground truth depth, and measure the change in the alignment factor between frames (in log space). We perform a per-video affine alignment beforehand to remove any global shift.

We find that our model is the most consistent on small inter-frame distances. At longer time intervals, methods that also predict camera poses benefit. We view the prediction of camera poses as an interesting direction for future work.

Excerpt of Table E1, track jitter (×100) over frame gaps, lower is better: the three best ranked methods and ours, by average rank. Bold: best of the ranked methods:

MethodΔ=1Δ=4Δ=16Δ=64Δ=128Avg. rank
π³1.041.672.042.783.781.5
Depth Pro 2-H+0.881.342.415.689.401.9
VGGT-1B1.101.792.544.507.762.6
VeloDepth1.122.244.5011.5316.574.4

Excerpt of Table E2, TAE (×100) over frame gaps, lower is better: the three best ranked methods and ours, by average rank. Bold: best of the ranked methods:

MethodΔ=1Δ=4Δ=16Δ=64Δ=128Avg. rank
π³0.961.522.414.504.921.5
Depth Pro 2-H+0.841.272.878.1712.782.0
VGGT-1B1.041.702.735.8610.042.6
VeloDepth0.982.085.3214.2424.164.2

Excerpt of Table E3, scale consistency over frame gaps, lower is better: the three best ranked methods and ours, by average rank. Bold: best of the ranked methods:

MethodΔ=1Δ=4Δ=16Δ=64Δ=128Avg. rank
VGGT-1B0.00030.00060.00140.00370.00601.0
π³0.00060.00120.00280.01180.02612.0
Depth Pro 2-H+0.00150.00330.00990.04430.08423.4
DVD0.00130.00360.01380.06700.12723.8

The full tables, every method and gap, are Tables E1–E3 in the supplementary ELzc.pdf.

W2: Less consistent relative performance in alternative TAE evaluation.

We find the TAE metric to be less suited for the evaluation of temporal consistency due to the static scene assumption and its handling of occlusions, which can lead to misleading results. We detail these arguments in the appendices (B.3) and in the main paper focus our temporal consistency evaluation on the more robust jitter metric.

That said, after submission, we discovered an error in our loading of camera-pose annotations, which affected the KITTI and Bonn datasets in the TAE evaluation. Following the correction of this error, we re-evaluated our method and all baselines under the TAE metric. Consistent with the handling of other non-zero-shot results in the paper, we also excluded methods that had ScanNet in their training set from the ScanNet ranking (π³, VGGT-1B, DAGE, UniK3D-L and VeloDepth, which builds on a frozen UniK3D-L).

The revised PDF contains the updated results in Table 11, and the supplementary ELzc.pdf shows the original and re-evaluated results in Table E1. With the corrections, the average rank of Depth Pro 2-H+ in Table 11 improves from 7.1 to 3.8, which is 3rd best. In our view, this result is consistent with the results we measured on jitter, and confirms the strong temporal consistency of Depth Pro 2.

Comments on the paper

1 comment

We thank the reviewers for their time and feedback. As multiple reviewers asked for clarification on the technical novelty, we provide an answer here and respond to other questions after the respective reviews. We further added a revised version of the paper (details in the individual responses), and a supplementary material with expanded versions of the tables we provide in the individual responses.

Limited technical novelty (ELzc, XT1b, QWyL).

We agree that Depth Pro 2 builds on established components, and the paper credits each to its source. No prior method, however, produces depth that is metrically accurate, sharp, and temporally consistent at once; all three reviewers note that Depth Pro 2 does. Our contribution is the architecture and training procedure that achieve this, together with controlled experiments that identify which design choices matter.

Differences to Depth Pro and Depth Anything 3 (DA3) (ELzc):

  • Depth Pro only processes single images at a fixed square resolution. It introduces several loss terms to improve boundary sharpness.
  • DA3 builds on DINOv2 with a learned spatial embedding and introduces a multi-stage training curriculum that trains a teacher model for relative depth on synthetic data and distills to a metric depth teacher.
  • Depth Pro 2, in contrast, processes a single image or video volume with the same network, and at each input's native resolution and aspect ratio. It builds on DINOv3 for spatially finer-grained features, adapts its spatial position encoding for varying aspect ratios, and introduces a resolution-preserving augmentation that keeps ground-truth depth at its original sampling.

While VGGT, Depth Pro, and DA3 may share individual components, they all perform worse across all metrics as demonstrated in Tables 1-4 of the main paper.

Our controlled experiments (Table 16 of the revised PDF) further validate our design choices in contrast to those made by prior work:

  • Training at a fixed square resolution (condition E3), as in Depth Pro, reduces metric accuracy (>48% on Waymo); this result motivates the aspect-ratio-aware position encoding.
  • Training on synthetic data alone (condition D2), as in Marigold, Lotus-2, InfiniDepth, and DA3's teacher model, reduces metric accuracy (>12% on ARKitScenes).
  • The loss terms of Depth Pro (condition I3) reduce temporal consistency (25% higher jitter on DynamicReplica).

Notably, the state of the art for sharp and temporally consistent depth is represented by DAGE and FlashDepth, see Figure 2 in our updated manuscript. The prevailing paradigm from these works is to add dedicated temporal modules and multi-stream processing at different resolutions. Depth Pro 2 shows that these additions are unnecessary. It uses a single network to process images and video volumes, and it outperforms DAGE and FlashDepth in metric accuracy, sharpness, and temporal consistency. A simpler architecture that performs better is, in our view, a useful finding for researchers and practitioners.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.