- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes RISE, an adaptive imagination framework for world action models that makes sequential rollout decisions based on predicted planning gains and computational cost. It also introduces CounterDrive, a generated counterfactual driving dataset for future prediction and risk learning. Experiments on NAVSIM and nuScenes demonstrate improved planning performance, while comparisons with fixed rollout depths show a better trade-off between planning quality and latency.
Critical strengths
- The paper addresses the useful question of how the benefit of future imagination varies across driving scenes. The experiments show that adaptive rollout improves planning performance while reducing latency compared with the best fixed-depth setting, supporting its practical value.
Critical weaknesses
- The distinction from existing adaptive planning methods is not clearly explained. While the fixed-depth comparisons support adaptive allocation, they do not establish the advantage over simpler learned alternatives. This makes it difficult to assess the added value of the proposed sequential stopping mechanism.
- The evidence supporting CounterDrive’s reliability is limited. Although human verification is described, the paper does not sufficiently document annotation consistency, preservation of the source scene, or source-scene-level train/test separation.
Questions for authors
- What distinguishes the proposed sequential stopping mechanism from prior adaptive imagination and lookahead methods? Could you compare it with a model that predicts the final rollout depth directly from the initial observation, using the same training targets and a comparable computation budget?
- Could you clarify CounterDrive’s verification and splitting protocols, including annotator agreement, acceptance/rejection criteria, and checks for unintended changes to scene geometry and ego motion? Are all generated variants of the same source scene kept within a single data split?
5 comments
Response to Reviewer 8RXQ (Part 2/2)
W2/Q2: CounterDrive Construction Details
CounterDrive follows the original training and test splits of NAVSIM and nuScenes, and each generated counterfactual video inherits the split assignment of its source scene. Thus, all variants derived from the same source scene remain within the same split, preventing source-scene overlap between training and test sets. We have clarified the data splitting and annotation protocols in Section 2 (p. 3), with detailed quality-control procedures provided in Appendix G.3 (p. 26).
For annotation, we adopt a sequential collaborative protocol inspired by Chain-of-Talkers (CoTalk), involving 12 annotators in total. Each sample is first annotated by one annotator and subsequently reviewed and supplemented by a second annotator, followed by a designated reviewer. When disagreements arise, a third person joins the discussion until consensus is reached; samples for which consensus cannot be achieved are discarded. Since annotations are produced sequentially rather than independently, conventional inter-annotator agreement statistics cannot be computed from independent duplicate annotations.
Regarding generation quality, our prompts explicitly constrain the camera viewpoint, road geometry, background, and initial traffic configuration. During human verification, we distinguish generation artifacts according to their potential impact on scene understanding and driving decisions. For example, severe distortions of road structures or lane geometry that may affect the model's judgment are treated as critical artifacts. We attempt to regenerate such videos multiple times, and discard them if the distortions persist. In contrast, minor deformations of task-irrelevant objects, such as leaves, are retained when they do not affect scene interpretation or annotation, considering the substantial cost of video generation. Annotators also verify ego-motion consistency and identify incident onset and generation distortions. Samples with unreliable ego-motion or severe visual distortions are excluded.
Finally, we are committed to releasing the complete CounterDrive dataset after the review process. We take full responsibility for its quality and welcome scrutiny, feedback, and suggestions from the research community to further improve the dataset.
Dear Reviewer 8RXQ,
As the discussion period draws to a close, we would greatly appreciate any feedback you may have on our rebuttal and revised manuscript. We would be happy to address any remaining questions or concerns.
Thank you again for your time and consideration!
Best regards,
Authors
Response to Reviewer 8RXQ (Part 1/2)
We sincerely thank you for your careful review and valuable suggestions. Your comments have been highly beneficial in helping us improve the clarity, rigor, and overall quality of our paper. We have carefully addressed your concerns through additional experiments and detailed clarifications. We hope these revisions satisfactorily resolve your concerns, and we would greatly appreciate it if you could reconsider your rating.
The key distinction between RISE and prior adaptive world-model methods lies in what is adapted and how the adaptation decision is made. For example, FFDC-WAM dynamically adjusts action execution length based on the consistency between imagined futures and real observations, determining when to replan. RISE focuses on adaptive imagination rollout depth during inference, where the Scheduler makes sequential ROLL/STOP decisions based on the expected Future Planning Gain and computation cost. Crucially, each newly predicted latent provides additional information about future world evolution, enabling RISE to reassess the value of continued imagination at every step. To the best of our knowledge, RISE is the first to introduce such adaptive imagination for autonomous driving World Action Models. Our results further demonstrate its effectiveness: RISE improves EPDMS from 90.1 to 90.8 over the best fixed-depth configuration while reducing latency from 318.6 ms to 287.4 ms.
To further validate the benefit of sequential reassessment, we introduce a One-shot Horizon baseline that predicts the final rollout depth directly from the initial observation, using the same all-horizon planning supervision as RISE. The results over three random seeds are summarized below.
RISE achieves a 2.97-point EPDMS improvement over One-shot Horizon, with substantially lower variance across random seeds. The additional computation is limited to 0.36 rollout steps and 21.6 ms on average. By reassessing planning utility after each predicted latent step, RISE can incorporate newly revealed future information into its stopping decisions, leading to more effective rollout allocation and consistently better planning performance.
We have extended Table 5 in Section 4.3 and clarified the differences from prior adaptive world-action modeling methods in Section 5.1.