- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes DVGT-World, which extends the streaming Vision-Geometry-Action (VGA) framework with horizon-conditioned future geometry prediction. The model jointly learns current multi-view geometry reconstruction and future pointmap and ego-pose prediction during pretraining, then fine-tunes for trajectory planning on NAVSIM. The future prediction branch is disabled during planning inference to avoid additional computational cost.
Critical strengths
-
The paper presents a straightforward extension of the VGA paradigm by explicitly supervising future 3D geometry. The future prediction branch can be disabled during planning inference, making the approach practical without incurring additional inference cost from future geometry prediction.
-
The experiments cover planning performance, future geometry prediction, prediction horizons, and inference efficiency. The ablation study shows a consistent planning benefit from future geometry supervision, with PDMS increasing from 89.9 to 93.0 on NAVSIM v1.
Critical weaknesses
-
The contribution of the scorer is not isolated. The scorer is trained directly on NAVSIM driving metrics and selects the final trajectory from 64 candidates, so it can materially affect the reported planning scores. However, the paper does not compare against a standard planning head without a scorer. It is therefore unclear how much of the performance gain comes from future geometry pretraining versus the scoring module.
-
Insufficient comparison with trajectory scoring methods. The paper lacks a controlled comparison with dedicated trajectory scoring approaches. In particular, the planning tables should include relevant methods such as DrivoR, which explicitly scores candidate trajectories. Comparisons under a common candidate generation and evaluation setup would better establish the effectiveness of the proposed planning pipeline.
-
Insufficient evidence for representation effectiveness. The planning ablations suggest that future geometry supervision benefits downstream planning, but the paper does not directly evaluate representation quality or transferability against alternative pretraining strategies under a common planning setup.
Questions for authors
-
Can the authors provide an ablation using a standard planning head without a scorer, keeping the backbone, candidate generation, and training setup fixed? This would help isolate the contribution of future geometry supervision from that of trajectory scoring.
-
Can the authors compare their scoring approach with dedicated trajectory scoring methods under a common candidate set and evaluation protocol? In particular, how much of the improvement comes from the proposed representation versus the scoring and selection strategy?
4 comments
Author Response - Part 2/2
R3. Comparison of different pretrained representations (W3).
To validate the effectiveness of our representation, we compared four variants using the same training data, optimization budget, and scorer-based planner. All variants retained current geometry reconstruction and differed only in future supervision. Representations pretrained to predict future dense geometry transferred best to planning, achieving 93.28 PDMS on NAVSIM v1 navtest, 0.73 points higher than future depth prediction and 0.80 points higher than future visual latent prediction. The visual latent variant serves as a controlled counterpart of world-action models that predict future video latents, such as Fast-WAM.
| Future supervision | NAVSIM v1 navtest PDMS ↑ |
|---|---|
| No Future | 92.29 |
| Depth | 92.55 |
| Visual latent | 92.48 |
| Dense geometry | 93.28 |
Thank you for the detailed rebuttal and additional experiments.
I appreciate the authors' efforts to address my concerns. The new controlled experiments provide stronger evidence for the effectiveness of future geometry supervision, and the revisions have clarified the experimental setup and conclusions.
Overall, I believe my main concerns have been adequately addressed, and I am updating my rating.
Official Comment by Authors
Dear Reviewer Q9ch,
Thanks for taking the time to review our paper and rebuttal.
Your suggestions helped us improve the experiments and make the paper clearer. We are glad that the additional experiments and revisions have addressed your main concerns, and we appreciate your constructive feedback and positive assessment of our work.
Best regards, The Authors
Author Response - Part 1/2
We thank the reviewer for the constructive suggestions. Sorry for the late reply, as we spent considerable time completing the dense training experiments. We first clarify a correction to Table 3 and then respond point by point.
Correction to Table 3. The submitted text stated that all Table 3 variants followed the same training protocol, which was inaccurate. The No Future baseline used the diffusion planner of DVGT-2, while all future-supervised variants used our scorer-based planner. We have retrained the No Future baseline with the same scorer-based planner under the full Table 1 configuration. Under this matched setting, future geometry supervision improves PDMS from 92.29 to 93.28 (+0.99). This matched comparison still supports our central claim that future 3D dense geometry prediction benefits downstream planning. The prediction-range variants all use the scorer-based planner, so their conclusions are unaffected.
R1. Contributions of future geometry supervision and the scorer head (W1, Q1).
To isolate the contributions of future geometry prediction and the scorer head, we trained three variants with the same initialization, training data, and optimization budget. On NAVSIM v1
navtest, replacing the diffusion planner with the scorer-based planner improved PDMS from 90.29 to 92.29 (+2.00) with future prediction disabled. With the scorer-based planner held fixed, adding future geometry prediction further improved PDMS from 92.29 to 93.28 (+0.99), supporting our central claim that predicting future dense geometry benefits downstream planning. The fourth variant would test whether this gain also holds for the diffusion planner. We could not complete that run within the available GPU time during rebuttal and will report it in future work.navtestPDMS ↑R2. Comparison with DrivoR and contribution of future geometry pretraining (W2, Q2).
We compared DVGT-World with scorer-based baselines, including DrivoR, using the standard NAVSIM
navtrainsplit for planning training. Each method generates its own candidate trajectories and selects trajectories using its own scorer, so we report results as an end-to-end comparison. Onnavtest, DVGT-World achieved 93.3 PDMS, outperforming SparseDriveV2 and trailing DrivoR (93.7) by 0.4 points. This small gap may be partly due to DrivoR’s higher camera input resolution (1148 × 672 vs. 512 × 288) and our limited exploration of scorer design, as this work focuses on geometry representations and future supervision rather than the scorer design. Our performance remains competitive with state-of-the-art planners on a benchmark where the human reference achieves 94.8 PDMS. Meanwhile, DVGT-World achieved higher NC and TTC scores than DrivoR, indicating fewer at-fault collisions and lower near-term collision risk, consistent with our motivation to learn spatially aware representations through future geometry prediction for robust planning. We have included these results in the revised paper.The scorer design is not the contribution of this paper. DrivoR improves the planner through register-based token compression and disentangled scoring, while our contribution is the pretraining objective, so the two directions are orthogonal. To assess the contribution of our pretrained representation, we kept the generator and scorer architectures, selection rule, and training budget fixed. As shown in R1, adding future geometry prediction during pretraining improved PDMS from 92.29 to 93.28 under the same planning setup. This supports our central claim that future 3D dense geometry prediction benefits downstream planning.