- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents G2, a geometry-grounded feed-forward framework for temporally consistent monocular video depth estimation. The framework introduces three components to improve cross-frame feature aggregation, geometric consistency, and training-data scalability.
First, Motion-Adaptive Implicit Alignment (MAIA) uses query-conditioned sparse sampling in a spatiotemporal feature volume to aggregate information across frames without relying on external optical flow or point tracking. Second, Explicit 3D Supervision (E3DS) introduces intra-frame point-map supervision and inter-frame projection losses to improve the geometric consistency of predicted depth maps. Third, Multi-View Geometric Distillation (MVGD) transfers geometric knowledge from a pretrained multi-view reconstruction teacher to unlabeled videos through pseudo 3D supervision and feature-level distillation.
The framework is initialized from Video Depth Anything and evaluated on zero-shot video depth estimation and geometric reconstruction benchmarks. The results demonstrate competitive depth accuracy, particularly under scale-invariant evaluation, and improved geometric reconstruction compared with several existing methods.
Critical strengths
- The paper is well-written and easy to follow.
- The paper addresses an important limitation of video depth estimation by improving cross-frame geometric consistency for downstream 3D applications.
- The method achieves strong zero-shot depth and reconstruction performance, particularly on 7-Scenes and NRGBD.
- The framework coherently combines adaptive temporal aggregation, geometric constraints, and teacher-guided learning.
- Comprehensive component-level ablations clarify the effects of temporal aggregation, sampling strategies, geometric supervision, and distillation.
Critical weaknesses
-
MAIA has limited methodological novelty and closely resembles deformable attention. In particular, Deformable DETR (ICLR 2021) already introduced query-dependent sparse sampling around reference locations using learned offsets. MAIA appears to extend this formulation to a spatiotemporal feature volume, so a direct comparison with deformable attention adapted to the same video setting is needed.
-
The geometric novelty of E3DS is insufficiently justified. Because predicted and target point maps derived from depth share the same camera rays, the intra-frame point-map loss may largely reduce to geometry-weighted depth regression. Meanwhile, the inter-frame projection loss resembles established cross-view reprojection constraints. Comparisons with ray-weighted depth regression and standard reprojection losses would better isolate its contribution.
-
MAIA’s claimed ability to handle camera and object motion is not directly validated. Its offsets are predicted from feature embeddings, but the paper does not show whether they correspond to meaningful motion-displaced observations. Motion-stratified evaluations, learned-offset visualizations, and analyses of validity masks on independently moving objects would strengthen this claim.
-
Multi-View Geometric Distillation using the pretrained Pi3 teacher is not clearly distinguished from conventional pseudo-label training. The reported gains from teacher-guided geometry and feature distillation are modest, and a feature-distillation-only ablation is missing. Comparisons with simple pseudo-depth supervision, the frozen Pi3 teacher, and matched unlabeled-data baselines are needed.
-
The related-work positioning against recent multi-view geometry foundation models is incomplete. Depth Anything 3 (ICLR 2026), VGGT (CVPR 2025), and DUSt3R (CVPR 2024) are particularly relevant to spatially consistent depth, point-map estimation, and joint multi-view geometric reasoning. The paper should clarify how G2 differs from these models and what practical advantages its specialized video-depth architecture provides.
-
Computational and statistical analyses are insufficient. The paper should report MAIA’s runtime, peak memory, parameter overhead, and scaling with video length relative to the Video Depth Anything backbone. Multi-seed results, confidence intervals, or significance tests are also needed because several component-level improvements are relatively small.
Questions for authors
-
How does MAIA fundamentally differ from deformable attention, particularly Deformable DETR (ICLR 2021)? Could the authors compare it with a conventional deformable-attention module extended to the same spatiotemporal volume under matched training and computational settings?
-
Since back-projected point-map errors are determined by depth errors along known camera rays, what additional geometric information does the intra-frame point-map loss provide? Please compare it with ray-weighted depth regression using matched weights and robust penalties.
-
Do MAIA’s learned offsets represent meaningful cross-frame motion or scene-point correspondences? Offset visualizations, results stratified by static regions, moving objects, and camera motion, and statistics on dynamic pixels excluded by the validity mask would help.
-
Could the authors provide a feature-distillation-only ablation and compare MVGD with conventional pseudo-depth distillation using the same Pi3 teacher, unlabeled data, and training budget? Where applicable, please also report the frozen Pi3 teacher’s performance.
-
How does G2 compare with Depth Anything 3 (ICLR 2026) and VGGT (CVPR 2025) in geometric consistency, depth accuracy, and efficiency? If direct comparison is inappropriate, please clarify the differences in task assumptions and intended applications.
-
What parameter, FLOP, memory, and latency overhead does MAIA introduce over Video Depth Anything, and how does it scale with video length? Please also report confidence intervals or multi-seed results for the main comparisons and ablations.
9 comments
Responses to Reviewer r14i (2/6)
W2/Q2: "Since back-projected point-map errors are determined by depth errors along known camera rays, what additional geometric information does the intra-frame point-map loss provide? Please compare it with ray-weighted depth regression using matched weights and robust penalties."
A2: Thank you for asking about the supervision provided by the E3DS. For labeled data, predicted and target depths are back-projected along the same camera ray, so the resulting 3D point difference equals the depth difference scaled by ray vector. Applying the same robust penalty to each ray-scaled residual component gives the same objective when masking, normalization, and weights match. Therefore, the intra-frame supervision simply parameterizes depth supervision in camera coordinates without introducing additional geometric information. We adopt the point-map formulation to unify both intra-frame and inter-frame supervision within a common 3D representation. The inter-frame supervision projects the source prediction into the target view, where it is supervised by the target-frame ground truth. Standard reprojection instead compares projected depth with the target-frame prediction.
The E3DS and ray-weighted depth regression runs start from identical model weights and use the same robust penalty and inter-frame projection loss. The comparison varies only the formulation of the intra-frame supervision.
We report SI depth metrics and TAE, followed by reconstruction metrics.
| Supervision setting | Bonn AbsRel ↓ | Bonn δ₁ ↑ | ScanNet AbsRel ↓ | ScanNet δ₁ ↑ | Sintel AbsRel ↓ | Sintel δ₁ ↑ | Bonn TAE (%) ↓ | Sintel TAE (%) ↓ |
|---|---|---|---|---|---|---|---|---|
| E3DS(ours) | 0.0598 | 0.9800 | 0.0797 | 0.9445 | 0.3479 | 0.6321 | 2.749 | 28.612 |
| Ray-weighted depth regression | 0.0633 | 0.9782 | 0.0799 | 0.9439 | 0.3495 | 0.6178 | 2.765 | 28.970 |
| Standard reprojection loss | 0.0668 | 0.9775 | 0.0824 | 0.9421 | 0.3633 | 0.5991 | 2.804 | 29.994 |
| Supervision setting | 7-Scenes Acc. ↓ | 7-Scenes Comp. ↓ | 7-Scenes NC ↑ | NRGBD Acc. ↓ | NRGBD Comp. ↓ | NRGBD NC ↑ |
|---|---|---|---|---|---|---|
| E3DS(ours) | 0.0220 | 0.0114 | 0.7948 | 0.0581 | 0.0206 | 0.9620 |
| Ray-weighted depth regression | 0.0231 | 0.0118 | 0.7942 | 0.0582 | 0.0212 | 0.9610 |
| Standard reprojection loss | 0.0257 | 0.0121 | 0.7928 | 0.0756 | 0.0232 | 0.9611 |
E3DS lowers NRGBD completeness error to 0.0206 from 0.0212 with ray-weighted depth regression and 0.0232 with standard reprojection. E3DS also gives the lowest SI TAE on Bonn (2.749%) and Sintel (28.612%).
In the revised version, Appendix B.4 derives the ray-scaled point-map loss (Lines 915–929, Eq. 19), and Sec. 3.2 explains inter-frame supervision against target-view ground truth (Lines 223–231). Sec. 4.4 reports the comparisons with ray-weighted depth regression and standard reprojection (Lines 500–507, Table 7).
W3/Q3: "Do MAIA’s learned offsets represent meaningful cross-frame motion or scene-point correspondences? Offset visualizations, results stratified by static regions, moving objects, and camera motion, and statistics on dynamic pixels excluded by the validity mask would help."
A3: Thank you for asking how MAIA's learned offsets relate to cross-frame motion. For each clip, we visualize the final sampling locations for a single query in the second MAIA block, obtained by refining preset locations with query-conditioned residual offsets. In Fig. 4(a), several samples remain near the same junction of the Sydney Opera House sails as the viewpoint changes. In Fig. 4(b), several samples fall on or near the moving animal's torso in neighboring frames. These examples illustrate that MAIA can adaptively sample corresponding scene regions under camera and object motion.
In the residual-offset ablation, adding the learned offsets to the preset anchors increases Bonn's SI-aligned from 0.967 to 0.980.
The E3DS validity mask filters rigid projections that disagree with the target geometry. Independent motion and occlusion can cause such inconsistencies. Estimating the fraction of dynamic pixels rejected by the mask requires separate pixel-level motion labels. We count rejections by the 3D consistency test over all in-bounds projections with valid source depth and valid target depth at the projected location.
| Dataset | Rejection rate |
|---|---|
| TartanAir | 11.87% |
| MVS-Synth | 12.07% |
| Virtual KITTI | 10.98% |
| PointOdyssey | 14.76% |
In the revised version, Appendix D.2 discusses MAIA sampling under camera and object motion (Lines 987–990, Fig. 4), and Sec. 4.4 reports the residual-offset ablation (Lines 464–471, Table 5). Appendix B.3 defines eligible projections and presents the manuscript's eight-frame mask rejection diagnostic (Lines 899–911, Table 9).
Responses to Reviewer r14i (3/6)
W4/Q4-1: "Could the authors provide a feature-distillation-only ablation and compare MVGD with conventional pseudo-depth distillation using the same teacher, unlabeled data, and training budget?"
A4-1: Thank you for requesting these controls. All five settings retain the same synthetic supervision. In the new comparisons, we match all the training configuration across variants. Synthetic-only continuation sets the distillation weights to zero. [3] depth distillation uses the teacher's depth pseudo-labels. MVGD feature distillation only uses the scene-level feature relation loss. Geometry-only distillation uses the teacher-guided geometric loss , which draws on teacher depth, point maps, and camera estimates. MVGD combines the feature and geometric losses.
We report depth accuracy, TAE and reconstruction metrics under scale-invariant alignment.
| Supervision setting | Bonn AbsRel ↓ | Bonn δ₁ ↑ | ScanNet AbsRel ↓ | ScanNet δ₁ ↑ | Sintel AbsRel ↓ | Sintel δ₁ ↑ | Bonn TAE (%) ↓ | Sintel TAE (%) ↓ |
|---|---|---|---|---|---|---|---|---|
| Synthetic-only continuation | 0.0555 | 0.9820 | 0.0833 | 0.9392 | 0.2010 | 0.6839 | 2.790 | 30.144 |
| depth distillation | 0.0558 | 0.9824 | 0.0738 | 0.9501 | 0.2059 | 0.6893 | 2.606 | 33.206 |
| MVGD feature distillation | 0.0552 | 0.9821 | 0.0822 | 0.9400 | 0.1980 | 0.6920 | 2.791 | 32.569 |
| Geometry-only distillation | 0.0519 | 0.9836 | 0.0775 | 0.9455 | 0.2008 | 0.6877 | 2.635 | 28.563 |
| MVGD(ours) | 0.0508 | 0.9844 | 0.0759 | 0.9492 | 0.1976 | 0.6995 | 2.539 | 28.302 |
| Supervision setting | 7-Scenes Acc. ↓ | 7-Scenes Comp. ↓ | 7-Scenes NC ↑ | NRGBD Acc. ↓ | NRGBD Comp. ↓ | NRGBD NC ↑ |
|---|---|---|---|---|---|---|
| Synthetic-only continuation | 0.0228 | 0.0114 | 0.7926 | 0.1057 | 0.0684 | 0.9465 |
| depth distillation | 0.0227 | 0.0115 | 0.7916 | 0.0586 | 0.0271 | 0.9526 |
| MVGD feature distillation | 0.0239 | 0.0119 | 0.7932 | 0.1073 | 0.0691 | 0.9469 |
| Geometry-only distillation | 0.0223 | 0.0114 | 0.7951 | 0.0589 | 0.0193 | 0.9648 |
| MVGD(ours) | 0.0215 | 0.0113 | 0.7939 | 0.0575 | 0.0199 | 0.9622 |
Compared with pseudo-depth distillation using the teacher, MVGD demonstrates superior performance on Sintel and NRGBD—reducing Sintel TAE by 14.8% and NRGBD accuracy error by 2.0%, while only slightly trailing depth distillation on ScanNet in AbsRel and . In addition, relative to synthetic-only continuation, MVGD reduces TAE on Bonn and Sintel, as well as accuracy error across the two reconstruction benchmarks.
In the revised version, Sec. 3.4 describes the data and schedule for second-stage training (Lines 296–308). Sec. 4.4 describes the synthetic-only continuation control and compares feature distillation, geometry-only distillation, and depth-map distillation with MVGD on Bonn and 7-Scenes (Lines 509–517, Table 8).
[3] : Permutation-Equivariant Visual Geometry Learning. In ICLR 2026.
W4/Q4-2: "Where applicable, please also report the frozen teacher’s performance."
A4-2: We report the frozen teacher's depth results on Bonn and Sintel under the evaluation protocol, and its reconstruction results on 7-Scenes and NRGBD (mean / median). ScanNet is part of the teacher's training data, so we omit it from zero-shot comparison.
| Teacher | Bonn AbsRel ↓ | Bonn δ₁ ↑ | Sintel AbsRel ↓ | Sintel δ₁ ↑ |
|---|---|---|---|---|
| 0.0497 | 0.9753 | 0.2339 | 0.6642 |
| Teacher | 7-Scenes Acc. ↓ | 7-Scenes Comp. ↓ | 7-Scenes NC ↑ | NRGBD Acc. ↓ | NRGBD Comp. ↓ | NRGBD NC ↑ |
|---|---|---|---|---|---|---|
| 0.016 / 0.007 | 0.022 / 0.011 | 0.689 / 0.792 | 0.015 / 0.008 | 0.013 / 0.005 | 0.898 / 0.987 |
In the revised version, Appendix D.8 describes the Bonn and Sintel depth protocol (Lines 1107–1110) and the dense reconstruction protocol (Lines 1115–1119). The frozen teacher's depth and reconstruction results appear in Tables 17 and 18, respectively.
Responses to Reviewer r14i (1/6)
We thank Reviewer r14i for recognizing the importance of cross-frame geometric consistency, the strong reconstruction results on 7-Scenes and NRGBD, and the value of the component ablations. We appreciate your suggestions for more direct comparisons and for clarifying what each component contributes.
W1/Q1: "How does MAIA fundamentally differ from deformable attention, particularly Deformable DETR (ICLR 2021)? Could the authors compare it with a conventional deformable-attention module extended to the same spatiotemporal volume under matched training and computational settings?"
A1: Thank you for this valuable suggestion. Extending conventional deformable attention to a spatiotemporal feature volume provides a generic sampling mechanism, but it does not inherently address the geometric correspondence and spatiotemporal support required for video depth estimation. Directly applying such an extension therefore results in inferior performance. Our design builds on Deformable DETR [1] and DAT [2]. MAIA uses query-conditioned residual offsets to refine staggered spatiotemporal anchors, with content-based query–key aggregation over valid samples. We clarify the key design differences and provide the requested controlled comparisons below.
First, MAIA uses staggered spatiotemporal anchors to organize complementary spatial and temporal support. Within each attention head, the preset sampling locations share a fixed spatial radius and are distributed evenly across neighboring temporal frames, which are then refined by learned query-conditioned residual offsets (Appendix B.2, Lines 881–895, Eq. 18). By contrast, a 3D adaptation of DAT relies on a uniformly spaced spatiotemporal reference grid, whereas a 3D adaptation of Deformable DETR uses an initial radial sampling layout centered on the reference point. This prior ensures balanced coverage across multiple spatial directions and temporal strides, reducing reliance on a single spatiotemporal layout and avoiding rigid coupling between spatial angle and temporal offset.
Second, MAIA computes aggregation weights from the content of valid sampled features rather than from the query alone. Following the content-based aggregation principle in DAT, MAIA pairs query-specific offset prediction with attention weights derived from query–key similarity, and masks out sampling positions falling outside the video feature volume so that only valid samples contribute. In contrast, Deformable DETR predicts both sampling offsets and attention weights via linear projections of the query without comparing the query against the sampled keys. For video depth estimation, query–key similarity is crucial because it directly evaluates how each piece of sampled geometric evidence contributes to depth recovery. The controlled comparisons below evaluate the resulting module as a whole.
Third, to ensure a fair comparison, we extend Deformable DETR and DAT to 3D space and compare MAIA against them. All variants share the exact same configuration. We report scale-invariant (SI) aligned depth accuracy and TAE, followed by reconstrcuction metrics.
MAIA lowers SI TAE on Bonn and Sintel and improves all six reconstruction metrics relative to both 3D alternatives. Compared with 3D Deformable DETR, it lowers Bonn SI TAE by 4.9% and NRGBD accuracy error by 22.4%.
In the revised manuscript, Sec. 3.1 explains the learned offsets and aggregation over valid samples (Lines 164–195). Sec. 4.4 presents the matched Bonn and 7-Scenes comparison and the residual-offset ablation (Lines 452–471, Tables 4 and 5). Appendix D.3 gives the profiling setup and results (Lines 994–1003, Table 11).
[1] Deformable DETR: Deformable Transformers for End-to-End Object Detection. In ICLR 2021. [2] Vision Transformer with Deformable Attention. In CVPR 2022.