acceptodds
Under review as a conference paper at ICLR 2027

MetricTrail: Revisiting the Power of Metric Depth and Local Optimization for Long-Sequence Streaming 3D Reconstruction

Abstract

We present MetricTrail, a streaming system for accurate camera motion estimation and dense reconstruction from long monocular videos. To keep inference efficient over long sequences, we revisit DPVO's sparse local estimator. We find that single-image metric depth substantially improves its trajectory accuracy, without retraining the correspondence network or enlarging the optimization window. Metric predictions initialize patch depths and remain as soft constraints during optimization. We further improve correspondence estimation through learned adapters that augment matching features and recurrent context with frozen DINOv3 representations. The correspondence estimator and adapters are trained on diverse short clips without additional long-sequence training. This modular design also allows the depth provider to be independently upgraded and supports supplied camera intrinsics and calibrated stereo observations without retraining. Under the reported RGB-only protocols, MetricTrail achieves the lowest average trajectory error among evaluated methods on KITTI, Oxford Spires, and VBR. Pairing the estimated poses with single-image depth predictions also produces strong dense reconstruction on indoor and outdoor benchmarks and coherent reconstructions of in-the-wild videos. With supplied intrinsics, the system processes KITTI-02 at 12.70 FPS with 3.21 GB peak GPU memory.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Streaming Reconstruction, Long-Sequence Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_CzcdOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2(+1)
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents METRICTRAIL, a streaming system for long-sequence camera tracking and dense reconstruction. Starting from DPVO, the method initializes sparse patch depths using single-image metric depth predictions and retains these predictions as soft unary constraints during local bundle adjustment. Frozen DINOv3 features are further incorporated through learned adapters to improve patch matching and recurrent context. The resulting camera poses are used together with per-frame metric depth predictions to construct dense reconstructions. The paper reports strong trajectory results on KITTI, Oxford Spires, and VBR, together with dense reconstruction evaluations on 7Scenes, TUM-Dynamic, and Oxford Spires. The system is also computationally efficient and operates with a bounded local optimization window.

Critical strengths

The empirical results are strong, particularly on long-sequence trajectory estimation. The method is simple and modular, and the ablations show that metric-depth priors provide a substantial improvement over the DPVO backbone. For example, adding metric depth reduces the average calibrated KITTI ATE from 52.69 m to 12.77 m, while the subsequent multi-dataset adaptation and DINOv3 augmentation further reduce it to 9.10 m.

Critical weaknesses
  1. Limited methodological novelty over existing optimization-based VO/SLAM.

The major contributions (metric-depth regularization within BA, pretrained visual features, and local differentiable optimization) are individually well established. The paper’s main novelty is therefore largely the integration of these components into a sparse DPVO backbone and the observation that such a bounded local estimator can perform well on very long sequences. I find the empirical result interesting, but the technical advance itself appears incremental, which is a major concern for acceptance.

  1. The core metric-scale claim is not sufficiently isolated or directly evaluated.

The paper argues that persistent metric-depth constraints stabilize scale over long sequences, yet the primary trajectory metric uses per-sequence Sim(3) alignment, which removes global scale error. In addition, the ablation groups metric-depth initialization and persistent BA regularization into a single component, so it is unclear how much benefit comes specifically from the proposed persistent constraint rather than simply better initialization. Since metric-scale stabilization is central to the paper’s motivation, these missing evaluations materially weaken the evidence for the claimed contribution.

Questions for authors
  1. How do the authors distinguish the methodological contribution from published optimization-based VO/SLAM methods that already use monocular depth regularization in BA, such as DROID-W? A clearer statement of what is technically new beyond integrating these ideas into a sparse DPVO backbone would materially affect my novelty assessment.

  2. Could the authors report trajectory accuracy without Sim(3) scale alignment (e.g., SE(3)-aligned or absolute metric ATE)? Since metric-scale stabilization is a central motivation, this would materially affect whether the results support the claimed benefit of metric depth rather than only trajectory shape.

4 comments
AuthorsOpenReview ↗

Response to Reviewer Czcd

Thank you for the thoughtful feedback. We clarify the contribution and provide the required evaluations below.

W1 & Q1: Methodological novelty.

Thank you for raising this point. Our contribution is a general coupling system of feed-forward metric geometry, pretrained visual representations, and recurrent local optimization, rather than a new depth-regularization principle or a substitution of a sparse backend. The central question is how to turn pretrained predictions into a long-sequence estimator without relying solely on learning long-horizon consistency from additional long-video training data.

Our use of DINO features differs at the level of the estimator. DROID-W uses DINOv2/FiT3D feature inconsistency to estimate dynamic uncertainty and reweight BA residuals. In contrast, our learned DINOv3 residual adapters directly augment both the matching and recurrent-context branches. They change the features used to establish correspondences and the context used to predict iterative corrections—not merely the weights assigned to existing residuals. The distinction is therefore the role of the representation, not simply the choice of DINO version.

The metric and visual priors interact throughout local inference. Metric predictions initialize patch geometry and remain active as persistent BA constraints. The current geometry determines reprojections and correlation sampling; DINO-enhanced matching and context inform correspondence corrections; BA then updates poses and patch depths for the next iteration. This couples pretrained appearance and geometry through a bounded active optimization graph, rather than treating depth prediction, feature extraction, and trajectory estimation as independent stages.

The ablations also show why this combination matters. Depth-augmented DPVO yields 12.77/1.85/19.57 m ATE on KITTI/Oxford Spires/VBR, while multi-dataset DINO adaptation without depth priors yields 54.46/2.26/92.66 m; the complete system achieves 9.10/1.71/18.90 m. Retaining only metric initialization also substantially degrades accuracy, showing that better initialization alone does not explain the results.

Our contribution is thus an empirically validated system design: data-driven priors provide transferable geometry and correspondence representations, while explicit local optimization imposes sequence-specific geometric consistency at inference. Adaptation uses short clips, without additional curated long-sequence training data. We will foreground this distinction and the supporting interaction ablations in the revision.

W2 & Q2: SE(3) ATE and persistent metric-depth constraints.

Tables 16–18 of the revised paper provide the per-sequence results. And we provide the average SE(3) ATE here:

MethodKITTIOxford SpiresVBR
LingBot-Map206.5732.0793.02
HorizonStream80.079.0327.34
HorizonStream w/ LC71.138.7518.22
Ours28.303.0316.44
Ours w/ LC25.383.0312.31

Our results are 28.30/3.03/16.44 m without LC and 25.38/3.03/12.31 m with LC, lower than the corresponding LingBot-Map and HorizonStream results on all three datasets.

The following results from Table 12 separate metric initialization from persistent BA regularization. These values use Sim(3) ATE (m) and we provide the comparison here:

VariantKITTIOxford SpiresVBR
Metric initialization only36.112.4897.58
Metric initialization + persistent BA regularization12.771.8519.57
Full model9.101.7118.90

Persistent regularization reduces the average ATE of all three dataset. This separates the substantial benefit of retaining the depth constraint from the additional gains of feature adaptation and training-data diversity.

Dear Reviewer Czcd,

Thank you again for your detailed review. As the discussion period is approaching its end, we would like to kindly check whether our response and the revised manuscript address your concerns. We have provided additional metric trajectory accuracy evaluations, expanded component ablations, and the corresponding methodological novelty explanation during the rebuttal period. We believe these revisions directly address your concerns. Please let us know if there are any remaining concerns that we can address or clarify before the discussion period ends. If our revisions have resolved your concerns, we would be grateful if you would consider raising your score.

Best regards,

MetricTrail Authors

Thank you for the detailed response and additional experiments. The added experiments have addressed my concerns regarding the results.

  1. The new initialization vs. persistent-regularization ablation has addressed my concern about isolating the metric-depth contribution,
  2. the added SE(3) ATE results address my concern about evaluating metric-scale consistency.

Regarding the novelty concern, the rebuttal clarifies that the novelty is primarily in the system-level coupling of metric geometry, DINO-based correspondence features, and bounded recurrent local optimization, rather than in any individual component. I find this positioning clearer, although I still view the novelty as relatively incremental. Nevertheless, given the strong results and additional analysis, I will raise my score. I will suggest the author to keep the new positioning in the revised version.

We sincerely thank you for the constructive feedbacks and raising the final rating. We will further polish our paper and make sure we include the additional experiments and analysis in the revised paper.

Reviewer_sQCxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces MetricTrail, a method designed for long-sequence monocular 3D reconstruction, building upon the foundation of DPVO. It utilizes depth predictions from MoGe-2 to initialize the depth of image patches and incorporates persistent depth constraints into the Local Bundle Adjustment (LBA) process, further supported by DINOv3 feature enhancement. While maintaining the bounded optimization window characteristic of the original DPVO, MetricTrail achieves improved trajectory accuracy and dense reconstruction quality, all without the need for additional training on long sequences.

Critical strengths
  1. By using MoGe-2 for depth initialization and incorporating persistent depth constraints into LBA, the accuracy of dense 3D reconstruction is significantly improved, without the need to expand the optimization window.
  2. Trained using only short clips, MetricTrail demonstrates competitive trajectory accuracy on long-sequence benchmarks.
  3. Experiments assess trajectories, dense reconstruction, depth quality, and runtime across diverse datasets and conditions.
Critical weaknesses
  1. In the ablation study concerning depth priors, depth initialization and the persistent depth constraint were treated as a single, combined ablation variant; consequently, the contribution of each module remains unclear.
  2. The ablation studies are incrementally designed, failing to adequately elucidate the interactions between the approaches. Incorporating additional ablation variants, such as (i) combining depth priors with DINOv3 without expanding the training data, and (ii) combining DINOv3 with multi-dataset training without the use of depth priors, would help to further clarify the contributions of each component and the interdependencies among them.
  3. The authors do not explore the roles of the DINOv3 enhancements within the matching and context branches. Furthermore, conducting a sensitivity analysis regarding the depth-prior weights and the optimization window size would help verify the model's robustness w.r.t. the selection of these parameters.
  4. In this paper, the authors discuss related depth-assisted reconstruction methods: RoMeO and ViPE. However, they do not provide a direct evaluation of RoMeO, while the comparison with ViPE is also limited to qualitative examples.
Questions for authors
  1. In the ablation study concerning depth priors, depth initialization and the persistent depth constraint were treated as a single, combined ablation variant; consequently, the contribution of each module remains unclear.
  2. The ablation studies are incrementally designed, failing to adequately elucidate the interactions between the approaches. Incorporating additional ablation variants, such as (i) combining depth priors with DINOv3 without expanding the training data, and (ii) combining DINOv3 with multi-dataset training without the use of depth priors, would help to further clarify the contributions of each component and the interdependencies among them.
  3. The authors do not explore the roles of the DINOv3 enhancements within the matching and context branches. Furthermore, conducting a sensitivity analysis regarding the depth-prior weights and the optimization window size would help verify the model's robustness w.r.t. the selection of these parameters.
  4. In this paper, the authors discuss related depth-assisted reconstruction methods: RoMeO and ViPE. However, they do not provide a direct evaluation of RoMeO, while the comparison with ViPE is also limited to qualitative examples.
4 comments
AuthorsOpenReview ↗

Response to Reviewer sQCx

Thank you for the detailed suggestions. The expanded component ablations appear in Table 12 of the revised paper and we also provide the results here:

VariantKITTIOxford SpiresVBR
DPVO52.692.5972.25
DPVO + depth prior12.771.8519.57
DPVO + depth prior + multi-dataset10.321.7622.51
Metric initialization only36.112.4897.58
DINO + multi-dataset, no depth prior51.462.2692.66
DINO + depth, no multi-dataset13.101.8520.37
Both DINO residuals disabled12.071.7519.56
DINO matching only12.371.7420.05
DINO context only9.171.7619.27
Full model9.101.7118.90

W1 & Q1: Depth initialization versus persistent regularization.

Initialization alone yields 36.11/2.48/97.58 m. Retaining the depth prior as a soft constraint in local BA reduces these errors to 12.77/1.85/19.57 m, a 74.9% reduction in the three-dataset average. Initialization alone therefore does not account for the gains; retaining the constraint during optimization is substantially more effective.

W2 & Q2: DINOv3 and training-data diversity.

The full model improves over the depth-prior + multi-dataset variant, reducing ATE from 10.32/1.76/22.51 m to 9.10/1.71/18.90 m. Multi-dataset training alone improves KITTI and Oxford but degrades VBR; the full model with DINOv3 improves all three and reverses this VBR regression. Conversely, DINOv3 without depth priors gives 51.46/2.26/92.66 m, while DINOv3 + depth without multi-dataset adaptation gives 13.10/1.85/20.37 m. These results support the complementary roles of geometric priors, visual representations, and diverse motion supervision in the complete system.

W3 & Q3: DINOv3 Matching/context branches and sensitivity analysis.

The context branch provides most of the DINOv3 gain: the unweighted three-dataset average falls from 11.13 m with both residuals disabled to 10.07 m with context alone. Matching alone gives 11.39 m, while enabling both branches achieves 9.90 m, the best result on every dataset. Thus, the matching residual helps in combination with context.

We additionally provide sensitivity analysis to the depth-prior weight, Huber threshold, and optimization-window size on KITTI under the calibrated setting, with loop closure disabled. We use the default setting (λ, δ, W) = (1.0, 0.3, 10).

Prior weight λ. Weak regularization substantially increases both errors. Across the tested weights λ ∈ [0.8, 5.0], SE3 ATE remains within 20.74–22.23 m. Stronger weights continue to improve Sim3 ATE but yield limited, non-monotonic improvements in metric accuracy. We therefore retain λ = 1.0 as a moderate default, rather than selecting the weight that minimizes error.

Prior weight λSim3 ATE (m) ↓SE3 ATE (m) ↓
0.118.0056.52
0.2512.9733.09
0.510.4023.29
0.89.5622.23
1.0 (default)9.1021.71
1.58.3620.74
2.07.9321.00
3.07.7221.87
5.07.4821.03

Huber threshold δ. The default δ = 0.3 achieves the lowest error in both metrics among the tested thresholds. Very small thresholds substantially degrade accuracy, whereas performance remains comparatively stable for δ ∈ [0.2, 2.0], supporting the robustness of the selected setting.

Huber threshold δSim3 ATE (m) ↓SE3 ATE (m) ↓
0.0121.5175.54
0.0217.0154.11
0.0511.9032.31
0.110.5924.52
0.29.6122.23
0.3 (default)9.1021.71
0.59.3422.89
1.09.5623.92
2.09.5724.11

Optimization window W. We retain W = 10 as a compact default that balances metric accuracy and the size of the local optimization problem. Increasing the window from 8 to 10 reduces average SE3 ATE by 11.6%, whereas increasing it from 10 to 12 provides only a further 1.1% reduction. Larger windows improve Sim3 ATE more substantially, but offer diminishing gains in metric accuracy while increasing the number of jointly optimized poses. Thus, W = 10 represents a compactness–accuracy trade-off, rather than the minimum-error setting.

Optimization window WSim3 ATE (m) ↓SE3 ATE (m) ↓
410.2924.74
69.5325.30
89.4524.55
10 (default)9.1021.71
128.4721.47
167.3821.01
207.2021.13

W4 & Q4: Comparisons with ViPE and RoMeO.

ViPE. Our paper already includes quantitative, per-sequence comparisons in Appendix A.3, Tables 13 (KITTI), 14 (Oxford Spires), and 15 (VBR) of the manuscript. ViPE obtains 28.68/7.06/46.20 m, compared with our 14.05/1.77/16.36 m, under the reported uncalibrated protocols. We will add explicit cross-references to make these results easier to locate.

RoMeO. Different from RoMeo that focuses on calibrated metric visual odometry,MetricTrail focuses on a complementary objective: a complete streaming system for long-sequence 3D reconstruction without requiring camera intrinsics as input. Since an official implementation was unavailable at the time of evaluation, we compare against its published results on the same nine KITTI sequences. RoMeO reports an ATE of 6.89 m without scale alignment for its fast version. On the same nine KITTI sequences, MetricTrail achieves 7.25 m without loop closure and 6.49 m with loop closure. MetricTrail also achieves 12.7 FPS during local inference, approximately twice the 6.28 FPS reported for RoMeO. In addition, RoMeO reports nine KITTI sequences, whereas we evaluate all 11 sequences, including highway driving with limited nearby structure (01) and a long trajectory with revisits (02). This also broadens our system’s evaluated conditions.

I have carefully read the author's response, but based on comprehensive considerations, I still maintain my original attitude.

Thank you for carefully considering our response. To address your concerns, we have provided additional metric trajectory accuracy evaluations, expanded component ablations, and the required sensitivity analysis. We believe these revisions directly address your concerns. Could you please clarify which specific concerns remain unresolved after considering these additions? In particular, we would appreciate understanding whether any of these analyses require further evidence or explanation, or whether other concerns remain that we have not yet addressed. Your clarification would help us provide a focused follow-up and respond directly to the issues. If the additional evidence alleviates your concerns, we would also be grateful if you would consider raising your score accordingly.

Reviewer_SPPYOpenReview ↗
Rating
4
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper found that although sparse patch correspondence and local BA in DPVO is efficient, depths of the new coming patches come from random values or median values, which can not help to avoid the metric drifting. The paper then used MoGe-2 to initialize the metric depth of each patch and keep a constraint on them. DINOv3 features are used to enhance the matching.

Critical strengths
  • The improvement is based on a clear motivation with geometry insights. The benefits of fixing the original network are huge. The paper is inspiring in this point.

  • The experiments are complete with long sequence comparisons, per-sequence table and grey transfer.

Critical weaknesses

The metric scale is not clearly evaluated. Sim(3) loses the metric scale. Hence it's unclear whether and how much is the metric scale improved.

Questions for authors

Please add SE(3) ATE, per-segment metric drifts on KITTI, Oxford, VBR to prove that the improvement on metric scale is improved.

1 comment
AuthorsOpenReview ↗

Response to Reviewer SPPY

Thank you for highlighting the need to evaluate metric scale and inspect individual sequences.

SE(3) evaluation. We report the evaluation of average SE3 ATE here:

MethodKITTIOxford SpiresVBR
LingBot-Map206.5732.0793.02
HorizonStream80.079.0327.34
HorizonStream w/ LC71.138.7518.22
Ours28.303.0316.44
Ours w/ LC25.383.0312.31

Our method achieves 28.30/3.03/16.44 m on KITTI/Oxford Spires/VBR, compared with 80.07/9.03/27.34 m for HorizonStream without LC. With LC, our errors decrease to 25.38/3.03/12.31 m, versus 71.13/8.75/18.22 m. These results support improved metric trajectory estimation when scale errors cannot be removed by alignment.

As for the Per-sequence results. We have added the per-sequence SE(3) results to Tables 16 (KITTI), 17 (Oxford Spires), and 18 (VBR) of the revised paper.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all the reviewers for their constructive feedback. We appreciate their recognition of MetricTrail’s clear geometric motivation, simple and effective system design, and comprehensive evaluation. Our system couples pretrained geometric and visual priors with bounded local optimization to support long-sequence tracking and dense reconstruction, while requiring only short-clip adaptation. The revised manuscript adds detailed component ablations (Table 12) and per-sequence SE(3) evaluations on KITTI, Oxford Spires, and VBR (Tables 16–18). These additions are highlighted in red in the revised paper for ease of reference. Below, we summarize the average SE(3) results and the expanded ablations.

Metric trajectory accuracy. We present the average SE(3) ATE (m) ↓ here. All the results use uncalibrated monocular input and LC denotes loop closure.

MethodKITTIOxford SpiresVBR
LingBot-Map206.5732.0793.02
HorizonStream80.079.0327.34
HorizonStream w/ LC71.138.7518.22
Ours28.303.0316.44
Ours w/ LC25.383.0312.31

Our method achieves the lowest error on all three datasets, both with and without LC, supporting improved metric trajectory accuracy.

Expanded component ablations. We present average Sim(3) ATE (m ↓) with supplied intrinsics and no LC.

VariantKITTIOxford SpiresVBR
DPVO52.692.5972.25
DPVO + depth prior12.771.8519.57
DPVO + depth prior + multi-dataset10.321.7622.51
Metric initialization only36.112.4897.58
DINO + multi-dataset, no depth prior51.462.2692.66
DINO + depth, no multi-dataset13.101.8520.37
Both DINO residuals disabled12.071.7519.56
DINO matching only12.371.7420.05
DINO context only9.171.7619.27
Full model9.101.7118.90

The proposed persistent depth constraint provides improvement to the trajectory accuracy, while the complete combination of metric priors, DINOv3 adaptation, and multi-dataset training achieves the lowest error. Detailed results and responses follow below.

AuthorsOpenReview ↗

Summary of Rebuttal and Revision

Dear Reviewers, ACs, SACs, and PCs,

We sincerely thank you for your time and constructive feedback. We summarize the main progress during the rebuttal period below:

  • Metric trajectory accuracy (Reviewers SPPY and Czcd): We added per-sequence SE(3) evaluations on KITTI, Oxford Spires, and VBR (Tables 16–18) to assess trajectory accuracy without scale correction.
  • Component contributions and interactions (Reviewers sQCx and Czcd): We expanded the ablations (Table 12) to separate depth initialization from persistent constraints and examine component interactions and the DINO matching/context branches.
  • Technical contribution (Reviewer Czcd): We clarified the system-level contribution of coupling metric geometry and pretrained visual features with bounded local optimization.
  • Hyperparameter sensitivity and Quantitative comparisons (Reviewer sQCx): We provided sensitivity analyses for the depth-prior weight, Huber threshold, and optimization window. We also provided quantitative comparisons and clarified the existing results to address questions about performance relative to related methods.

The added experiments and the corresponding explanations are highlighted in red in the revised manuscript and we will further polish our final paper. We hope these additions provide a stronger basis for your assessment. Thank you again for your efforts throughout the review process.

Best regards,

MetricTrail Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.