- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes DriveVerse, a controllable autoregressive multi-view observation generator for closed-loop driving-policy training. A structured simulation backend maintains the world state, and a world model generates synchronized multi-view future camera frames conditioned on surrounding-agent states, road geometry, and traffic signals. A four-stage training pipeline combines bidirectional adaptation, diffusion forcing, consistency distillation, and distribution matching on generated histories. Online diagnostics reject unreliable observations. Policies are optimized using an adopted T-GRPO objective that combines local plan rewards and executed-trajectory feedback. Experiments report improved video and geometry metrics and higher NAVSIM v2 EPDMS for VLA, WAM, and E2E policies. The VLA improves from 91.1/47.1 to 92.8/52.3 on navtest/navhard relative to its GRPO baseline.
Critical strengths
-
A useful integration of controllable generation and interactive policy learning.The backend explicitly owns scene dynamics and rewards, while the generator renders the camera views for planning. Perspective-map and BEV conditioning, causal generation, and training on generated histories address complementary obstacles to recurrent interaction. Tables 2 and 3 support these design choices on generation metrics. This is a consequential systems contribution toward reusing logged scenes for visual policy training.
-
Consistent policy improvements across three action representations. Table 4 reports gains over both SFT and the selected RL baseline for each policy family on both benchmarks. Table 5 also matches VLA initialization, scenes, sample counts, and update budgets, and shows that combining step and trajectory advantages outperforms either alone. These results make the full pipeline promising beyond a single policy architecture.
Critical weaknesses
- The policy experiments do not sufficiently establish which part of the pipeline produces the gains. Table 5 provides useful controls over initialization, scenes, sample counts, and update budgets, and the online step-only variant already improves over logged-observation GRPO. However, it remains unclear whether these two settings use the same EC-aware training reward. Clarifying this would strengthen the attribution of the gains to post-action observation feedback. More broadly, the generator ablations in Tables 2–3 are evaluated only on generation metrics, leaving their contribution to downstream policy improvement unestablished.
- The reliability of generated observations after the policy deviates from the recorded trajectory is not sufficiently validated. The quantitative generation evaluation uses recorded ego motion, whereas the recurrent reliability result is acceptance by the proposed diagnostics on a prefiltered subset: 65.3% of test clips are retained, and 87.2% of attempted rollouts within this pool complete four seconds. These figures transparently describe conditional coverage, but do not quantify the rate of materially incorrect observations among accepted segments. Manual inspection near the validation threshold is useful calibration, yet no quantitative independent audit or evaluation of reliability under the final RL policies is reported. NAVSIM's original and pre-generated follow-up observations provide useful policy evaluation, but do not independently validate the generator during recurrent interaction. This limits confidence that the diagnostics protect learning when optimized policies visit states different from those of the frozen SFT policy used for the reliability experiment.
Questions for authors
- Do the logged-observation GRPO baseline and the online step-only variant in Table 5 use the same EC-aware training reward and scoring implementation? If they differ, could you clarify these differences and their potential contribution to the gains? Additionally, do you have a controlled policy-training comparison using one of the generator ablations in Tables 2–3, with the remaining training settings fixed, to establish whether improved generation quality translates into better policy performance?
- Beyond the diagnostics’ own acceptance rates, have you independently assessed accepted generated observations for object or lane errors that could affect driving decisions? If so, could you report the assessment procedure and error rates? Since the reported recurrent reliability experiment uses a frozen VLA SFT policy, could you also provide acceptance rates and corresponding observation-quality assessments for rollouts collected with the final RL policies? This would help establish whether the diagnostics remain effective under the state distributions encountered after policy optimization.
2 comments
W2 and Q2: Observation Quality and Diagnostics Under Policy Optimization
We thank the reviewer for asking how generated observation quality is verified off the logged trajectory and under policy optimization.
Object and lane errors of the generated observations are measured independently of the online gate. On 1,000 navtest clips under recorded ego motion in Table 1 (L324), frozen BEVFormer for 3D detection and frozen LATR for lane detection show DriveVerse reaching LET-AP 0.407, LET-APL 0.261, LATR-F1 0.892, and a far-range lateral lane error of 0.306, the best values among all models compared in Table 1. Neither BEVFormer nor LATR is part of the online gate, which instead uses YOLOv12 for objects and CLRerNet for lanes (L248, Appendix L802), so the two sets of perception models check each other.
Once the policy leaves the logged trajectory, every generated segment is checked at the pose the policy actually reached. Because no recorded image exists at the new poses, metrics that need a reference image do not apply, and the established practice for such novel trajectories is to run perception models on the generated frames and compare the detections with the scene annotations, for which DriveDreamer4D [1] introduces NTA-IoU and NTL-IoU. Following the same design, our online object and lane agreement scores compare detections on every generated segment with the agents and lanes projected from the simulated state (Appendix L812), with tolerances that accommodate detector and projection error (Appendix L821). The acceptance threshold was calibrated on motion that leaves the log across 1,771 validation clips with 11 trajectory variants built following the navhard construction, with manual inspection of clips near the boundary (Appendix L842), and a segment is accepted only if the minimum of all its scores passes (L255).
The policy results are the direct test of whether any error left in accepted observations affects driving. DriveVerse is used only to train, and every policy is evaluated with the official NAVSIM v2 evaluation on the observations the benchmark provides (L360), with no DriveVerse-generated observation entering evaluation. If accepted segments placed objects or lanes wrongly in ways that affect driving, a policy trained on them would learn responses that do not hold on the benchmark's observations, and its benchmark score would not improve. What is observed is the opposite, as all three policy families improve over their RL baselines on both navtest and navhard in Table 4 (L378), and on the second stage of navhard, which evaluates the policy from follow-up states off the logged trajectory, EPDMS rises from 56.8 to 59.6 for the VLA, from 41.0 to 52.8 for the WAM, and from 46.9 to 54.2 for the E2E policy relative to their RL baselines (L440). The navtest EPDMS gains also hold on both sides of the initial reliability filter, which accepts 653 and rejects 347 of the 1,000 clips of Table 1:
| Policy (baseline RL -> DriveVerse T-GRPO) | Accepted scenes | Rejected scenes |
|---|---|---|
| VLA (GRPO) | 94.4 -> 95.6 | 91.3 -> 93.0 |
| WAM (FlowGRPO) | 87.8 -> 95.1 | 82.6 -> 92.2 |
| E2E (WorldEngine RL) | 90.1 -> 94.6 | 85.4 -> 91.6 |
Every family improves on both the accepted and the rejected subsets.
The gate is applied throughout training whichever policy acts, so it covers the states that optimized policies visit. The diagnostics are not tied to the SFT policy; they run online on every generated segment of every training rollout, and the object and lane checks compare the segment with the simulated state that the current policy's own action produced before the segment reaches the policy. A state far from real driving data lowers the distributional support score (Appendix L824), and the segment is rejected with no reward penalty once the reliability score falls below the threshold, while the rollout is still scored by the simulator over the full 4 s horizon (L305), so an optimized policy that moves outside the reliable range of the world model receives a simulator score, not an unreliable image. The reported experiment uses the frozen VLA SFT policy that T-GRPO training starts from, with fixed checkpoints and diagnostic parameters on reserved test clips, to give one reproducible reference figure (L405, Appendix L856). The final policies are the product of this training and are evaluated on the official benchmark, where the world model plays no part: the world model's task ends with training.
We hope these responses have brought added clarity to our work, and we welcome further comments achievable within the coming week before the discussion deadline if that would help your assessment.
[1] G. Zhao, C. Ni, X. Wang, et al. DriveDreamer4D: World Models Are Effective Data Machines for 4D Driving Scene Representation. CVPR, 2025.
W1 and Q1: Training Reward Formulation and Generator Contribution to Policy Learning
We thank the reviewer for the constructive assessment and questions. Below we explain how the training reward operates across single-step and online settings and how world-model generation quality and distillation govern policy learning.
Both settings use the same NAVSIM v2 EPDMS scorer, and extended comfort is the one term GRPO on logged observations has no second plan to compute, as in prior work. Using the benchmark's sub-scores, thresholds, and weights, NAVSIM v2 EPDMS multiplies compliance terms by a weighted average of five terms: progress, time-to-collision, lane keeping, history comfort, and extended comfort. Four of these five score one plan alone, whereas EC compares the plan at one observation with the plan at the next (Appendix L1007). Because logged-observation GRPO draws unexecuted candidate plans at one decision step, no post-action observation or second plan exists, and the baseline uses the standard reward of this line of work, EPDMS over the four defined terms with benchmark weights. Every DriveVerse rollout scores its first decision step without EC like the baseline (Appendix L1014). From the second decision step onward, the policy replans from the generated post-action observation, so both plans exist and EC is scored with its benchmark weight. The step-only variant thus differs from the baseline only in the availability of post-action observations, and EC supervision on states reached by the policy's actions is part of what this feedback contributes (L436, Appendix L1012).
Leaving EC out on a single logged scene is established practice, and the gains appear in terms that both rewards contain. FlowR2A [1] excludes the two-frame EC term because it is not defined within a single scene, and GRPO-based planners such as WAM-Flow [2] and ExploreVLA [3] likewise optimize rewards without EC and report NAVSIM v2 EPDMS. Under matched VLA initialization, scenes, sample counts, and update budgets (L429),
navtest/navhardEPDMS rises from 91.1 / 47.1 for logged-observation GRPO to 92.1 / 49.4 for DriveVerse step advantage only, 91.8 / 50.0 for trajectory advantage only, and 92.8 / 52.3 for T-GRPO (L415). For the VLA onnavtest, T-GRPO relative to GRPO raises ego progress from 90.0 to 93.4 and improves collision avoidance and drivable-area compliance, while EC rises from 82.0 to 86.0 (L423). Onnavhard, first-stage and second-stage scores rise from 82.0 to 87.6 and from 56.8 to 59.6 (L427). Progress and compliance terms are scored identically in the baseline, so these gains arise in quantities the baseline reward already optimizes.Generator quality reaches the policy through the reliability gate, while causal distillation makes interactive rollouts computationally practical. A generated segment is returned to the policy only if it passes our diagnostics; a rollout rejected at its first segment supplies no post-action observation and reduces to a logged-observation GRPO sample, moving training toward the logged-observation GRPO row of Table 5. Under the same diagnostics on our held-out test clips, the strongest baseline world model, Cosmos 3 Nano, reaches an acceptance rate of only 22.7%, against 65.3% for DriveVerse (L407), as slight shifts in perspective cause serious lane-topology mismatch and other road users move without control. Finally, causal distillation reduces denoising from 35 steps per chunk in the bidirectional teacher to four (Table 3, L340, L397), avoiding the approximately 8 times longer rollout time of the undistilled bidirectional model, which would turn RL training from days into weeks.
We hope these responses have brought added clarity to our work, and we welcome further comments achievable within the coming week before the discussion deadline if that would help your assessment.
[1] X. Li, Z. Liu, X. Ye, et al. FlowR2A: Learning Reward-to-Action Distribution for Multimodal Driving Planning. arXiv:2606.24231, 2026.
[2] Y. Xu, J. Cui, F. Cai, et al. WAM-Flow: Parallel Coarse-to-Fine Motion Planning via Discrete Flow Matching for Autonomous Driving. CVPR, 2026.
[3] Z. Sheng, X. Ye, J. Luo, et al. ExploreVLA: Dense World Modeling and Exploration for End-to-End Autonomous Driving. arXiv:2604.02714, 2026.