- Rating
- 3
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates whether dense geometric supervision can provide useful scene representations for camera-based autonomous driving. It introduces DVGT-2, a DINOv3-based multi-view model that jointly predicts ego-centric local pointmaps, relative ego-poses, and future ego-trajectories through shared features. Its streaming implementation combines factorized spatial attention, temporal causal attention, a fixed-size per-layer feature cache, and relative temporal encoding. Predicting local geometry and relative motion avoids anchoring reconstruction to the first frame, allowing older cached features to be discarded. Training uses MoGe-2-derived geometric supervision, followed by joint geometry–planning training and benchmark-specific fine-tuning. Validation includes geometry reconstruction on five driving datasets, planning on NAVSIM v1/v2 and nuScenes, component and cache-size ablations, and long-sequence accuracy and efficiency measurements. The reported results show strong planning performance and improved local ray-depth accuracy, with bounded per-frame computational and memory costs.
Critical strengths
S1. A coherent streaming design with meaningful long-sequence validation. The combination of ego-centric geometry, relative motion prediction, bounded feature caching, and relative temporal encoding addresses a concrete limitation of online multi-frame geometry models: increasing computation and memory as the observation history grows. Importantly, the evidence is not limited to an asymptotic argument or short clips. Figure 5 reports approximately constant per-frame costs over more than 1,200 frames.
S2. Strong task-level results make geometry-supervised planning a credible alternative. DVGT-2 reports 92.5 PDMS on NAVSIM v1 and 90.0 EPDMS on NAVSIM v2, the highest aggregate scores among the methods listed in Tables 3 and 4. On nuScenes, it reports an average displacement error of 0.78 m and an average collision rate of 0.19%. These results are substantial evidence that the complete geometry-supervised system can perform competitive driving planning, rather than merely improve a reconstruction objective. This empirical effectiveness is the strongest argument for the paper’s significance, although it does not by itself establish which component causes the gains.
Critical weaknesses
W1. The contribution of dense geometric supervision is not sufficiently isolated in the final streaming planner. Table 6 provides important supporting evidence: adding geometric supervision increases PDMS from 71.2 to 91.5 in the non-streaming configuration, and the authors explicitly state that the variants use the same training data and budget. However, there is no corresponding geometry-supervision ablation for the complete streaming model in row f. Consequently, the paper does not directly establish the contribution of geometric supervision to the final model’s headline benchmark results.
W2. There is a lack of experimental evidence demonstrating the relationship between window size W and planning performance. Table A.5 motivates the choice of W=3 using reconstruction accuracy and resource consumption, without reporting planning performance across window sizes. Thus, the selected history budget is justified for geometry reconstruction but not directly for the paper’s main driving task. The bounded-memory property is convincing; the planning-specific benefit and trade-off of the chosen streaming configuration are less fully established.
Other minor issues: W3. In Table 5, under the “Collision Rate” metric, the “Avg” value for DVGT-2 is both underlined and bolded—is this a mistake?
Questions for authors
Q1 — Clarification and targeted validation of the geometric-supervision claim. How is the no-geometry variant in Table 6 trained during the stage corresponding to the 160K-iteration geometry pretraining, and does it retain pose supervision? Can the authors provide a comparison between the complete streaming model and a counterpart without dense geometric supervision, keeping the backbone, planning head, training data, and optimization budget matched, preferably on a standard NAVSIM split? This would directly affect my assessment of whether the final planning gains can be attributed to dense geometric supervision.
Q2 — Targeted validation of the streaming claim. How does the proposed cached model compare against a bounded-history recomputation baseline using the same recent-frame budget, local-coordinate targets, and relative temporal encoding? Also, how do planning scores and comfort change with cache-window size? These comparisons would clarify whether the selected streaming configuration provides a favorable planning accuracy–efficiency trade-off beyond avoiding full-cache resets. Table A.5 currently reports only geometry accuracy and resource measurements.
Q3 — Clarification of benchmark evaluation and cached state. For the NAVSIM results, particularly Table 4, please specify the exact devkit version and split/scenario configuration, together with how cached state is initialized, carried forward, and reset. Is the cache reconstructed independently for each evaluated scenario, or retained across consecutive scenarios from the same log? How is the admissible observation history kept comparable with the baselines? These details materially affect the interpretation and reproducibility of the reported scores for a stateful streaming model.
Q4 — Clarification of geometric evaluation targets. What are the sources of the reference point clouds and ray depths used for geometry evaluation in Section C? Are these sensor-based measurements or model-generated labels, and are they independent of the MoGe-2-derived training supervision?
3 comments
Author Response - Part 2/2
R4. NAVSIM evaluation protocol and cache state (Q3).
We used devkits based on NAVSIM releases v1.1 for Table 3 and v2.2 for Table 4. For each version, we evaluated all scenarios in the official navtest split. We evaluated each scenario independently and built its cache from scratch, with no cache retained across consecutive scenarios from the same driving log. Following the standard NAVSIM evaluation protocol, DVGT-2 used only the current frame and the three preceding frames (W=3), matching the observation history available to the baselines.
R5. Geometric evaluation targets (Q4).
In Section C, following DVGT, we use the same MoGe-2 pseudo-labeling pipeline to generate the geometric targets for training and evaluation. We use pseudo-labels because LiDAR measurements in driving datasets are too sparse to provide dense geometric targets. As an independent check, we also compare DVGT-2’s predictions directly with raw LiDAR measurements on OpenScene and nuScenes. Although trained on pseudo-labels, DVGT-2 reconstructs scene geometry more accurately than VGGT on both datasets when evaluated directly against raw LiDAR measurements.
| Method | OpenScene Acc ↓ | OpenScene Abs Rel ↓ | nuScenes Acc ↓ | nuScenes Abs Rel ↓ |
|---|---|---|---|---|
| VGGT | 1.17 | 0.21 | 1.47 | 0.26 |
| DVGT-2 | 0.72 | 0.11 | 0.84 | 0.11 |
| MoGe-2 pseudo-labels | 0.65 | 0.11 | 0.70 | 0.10 |
R6. Typo (W3). The underline was a typo. The lowest value (0.19%) should be in bold only. We have fixed it in the revised paper.
Dear Reviewer Sprn,
Thank you for your thoughtful review and constructive suggestions, which have significantly improved the quality and clarity of our paper. We sincerely hope that our rebuttal has fully addressed all of your concerns. As the discussion period is ending soon, if you need any further clarification or additional information, please do not hesitate to contact us. We truly appreciate your time and effort devoted to reviewing our work.
Best regards, Authors
Author Response - Part 1/2
We thank the reviewer for constructive suggestions to our paper. We provide our response below.
R1. Geometric supervision in the complete streaming planner (W1, Q1).
In Table 6, the planning-only variant (row a) received neither pointmap nor pose supervision and therefore skipped the 160K-step geometry pretraining stage. Row b completed that stage, so the training budgets matched only for the two subsequent stages.
To isolate the effect of geometric supervision in the complete streaming DVGT-2, we trained two variants with the same backbone, planning head, initial weights, training data, and optimization schedule. Both skipped geometry pretraining and underwent 80K steps of streaming training followed by 40K steps of NAVSIM fine-tuning. Across both stages, one variant used only the planning loss, while the other also used pointmap and relative ego-pose losses. On NAVSIM v1
navtest, the full streaming DVGT-2 achieved 90.4 PDMS with additional geometric supervision, compared with 72.6 PDMS with planning supervision alone.navtestPDMS ↑R2. Cached inference versus recomputation at a fixed history window (Q2).
We compared two inference modes of the same DVGT-2 checkpoint on NAVSIM v1
navtest, keeping the history window (W=3), local-coordinate targets, and relative temporal encoding fixed. At each step, the baseline recomputed features for the current frame and the three preceding frames, while the cached model reused the preceding frames’ features to process the current frame. At the same history window, the full streaming DVGT-2 achieves better efficiency without sacrificing accuracy.R3. Effect of cache window size on planning (W2, Q2).
We evaluated the same DVGT-2 checkpoint with different window sizes (W) on the 200 OpenScene test sequences used in Table 6. We used these long sequences because the standard NAVSIM v1
navtestscenarios provide only the current frame and three preceding frames, limiting the history only available to W=3. Windows larger than W=3 increase latency without consistently improving PDMS, and Comfort remains at 1.00 across all window sizes.