- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PlanWAM, a novel World Action Model for end-to-end autonomous driving. Instead of directly applying future prediction supervision to the world model, the authors use planning objectives to learn future representations that are more useful for trajectory planning. Specifically, they first introduce a Temporal Register Pyramid to compress historical observations, then propose a planning-shaped future representation learned from privileged future observations through planning supervision. A Hindsight-to-Foresight Distillation strategy is further introduced to transfer this representation to a history-only world model, whose predicted future features guide trajectory generation and selection. The proposed method demonstrates strong performance on NAVSIM and HUGSIM, validating its effectiveness in both open-loop and closed-loop evaluations.
Critical strengths
-
Strong empirical performance. The authors evaluate the proposed method on multiple benchmarks, achieving state-of-the-art results on NAVSIM and HUGSIM, demonstrating strong planning capabilities.
-
Clear presentation and detailed implementation. The paper is well-written and provides comprehensive methodological and implementation details, making the proposed framework easy to understand and reproduce. Extensive experimental comparisons and ablation studies are also provided.
-
Effective methodological designs. The authors introduce a Temporal Register Pyramid for efficient historical information compression and a Hindsight-to-Foresight Distillation strategy for learning planning-shaped future representations. The effectiveness of these components is demonstrated through ablation studies.
Critical weaknesses
-
Limited generalization of the planning-shaped future representation. The main motivation of this paper is to learn future representations that are more useful for planning through planning objectives. However, compared with generic future-feature supervision, the proposed planning-shaped representation only improves PDMS by 0.3 points. Moreover, the DrivoR baseline already achieves a relatively high PDMS of 93.1 with a trajectory scorer. It would be more convincing if the authors could demonstrate the effectiveness of the proposed representation on other planning architectures, such as VADv2, to further validate its generality.
-
Limited improvement in trajectory generation. As shown in Table 6, introducing the predicted future latent into trajectory generation alone does not improve PDMS, while most of the improvement comes from trajectory selection. The failure analysis in Figure 5 also shows that the reduction in failures mainly comes from fewer selection errors. This raises the question of why the planning-shaped future representation provides limited benefits for trajectory generation, which seems to restrict the applicability of the proposed method.
-
Lack of ablation studies on HUGSIM. Although the paper reports strong zero-shot closed-loop performance on HUGSIM, the ablation studies are mainly conducted on NAVSIM. Given the relatively saturated performance and limitations of NAVSIM, it would be valuable to provide additional ablations on HUGSIM, particularly comparing generic future representations with planning-shaped representations and examining their contributions to trajectory generation and selection. Such results would further strengthen the evidence for the effectiveness of the proposed method in closed-loop driving scenarios.
Questions for authors
- As shown in Table 6, conditioning trajectory generation on the predicted future latent does not improve the final PDMS. Could the authors clarify whether the future latent improves the quality or diversity of generated trajectory candidates, for example, in terms of oracle PDMS or best-of-K trajectory error?
9 comments
(2/3) Re: the Reviewer JCcM
2. Does the Future Latent Improve the Quality or Diversity of Generated Trajectory Candidates? (W2 and Q1)
We thank the reviewer for raising this important question. To further examine the effect of future representations on trajectory generation, we conduct additional analyses of candidate quality and geometric diversity, including overall candidate quality, Oracle Top-K performance, and geometric dispersion.
(1) Candidate trajectory quality.
We evaluate all 64 candidate trajectories generated by DrivoR and PlanWAM, as reported in Table R3.
Table R3. Candidate Trajectory Quality Comparison.
| Metric | DrivoR | PlanWAM | Gain |
|---|---|---|---|
| Mean PDMS over 64 Candidates | 79.21 | 82.39 | +3.18 |
| Oracle Top-1 PDMS | 98.95 | 99.27 | +0.32 |
| Oracle Top-5 Mean PDMS | 98.09 | 98.67 | +0.58 |
| Oracle Top-10 Mean PDMS | 97.17 | 97.95 | +0.78 |
The results show that planning-shaped future representations improve the overall quality of generated candidates. Mean PDMS across all 64 candidates increases by 3.18 points, while Oracle Top-1, Top-5, and Top-10 mean PDMS improve by 0.32, 0.58, and 0.78 points, respectively. This indicates that the improvement extends beyond the best candidate to a broader set of proposals.
Regarding why generation-only conditioning in Table 6 does not improve final PDMS, DrivoR already achieves 98.95 Oracle Top-1 PDMS, leaving limited room to improve the best candidate. Moreover, the generation-only setting conditions the Planner on the predicted future latent without explicitly enhancing the Scorer. Thus, improving candidate quality alone does not guarantee better final trajectory selection or a higher PDMS.
(2) Candidate trajectory diversity.
We further measure the standard deviations of candidate endpoints, lateral positions, and headings, together with the nearest-neighbor endpoint distance, as shown in Table R4.
Table R4. Geometric Dispersion of Candidate Trajectories.
| Metric | DrivoR | PlanWAM |
|---|---|---|
| Endpoint XY Std. (m) | 2.241 | 2.142 |
| Lateral Y Std. (m) | 1.088 | 1.042 |
| Heading Std. (°) | 3.15 | 2.52 |
| Nearest-Neighbor Endpoint Distance (m) | 0.354 | 0.348 |
The results show no increase in geometric diversity, consistent with our design objective. Our goal is to improve candidate quality through planning-shaped supervision rather than explicitly expand geometric coverage. The main benefit therefore lies in higher-quality candidates, not increased diversity. Further improving trajectory diversity remains a direction for future work.
We have incorporated these candidate quality and diversity analyses into the revised manuscript.
(3/3) Re: the Reviewer JCcM
3. Additional Ablation Studies on HUGSIM (W3)
We thank the reviewer for this valuable suggestion. Following the reviewer's recommendation, we conduct additional zero-shot closed-loop ablations on HUGSIM, comparing generic and planning-shaped future representations and examining their respective contributions to trajectory generation and selection. The results are reported below as means ± standard deviations over three random seeds.
Table R5. HUGSIM Ablation Study of Planning-Shaped Future Representations.
| Configuration | Generation | Selection | HD-Score ↑ | RC ↑ |
|---|---|---|---|---|
| DrivoR Baseline | — | — | 33.5 ± 1.1 | 45.3 ± 1.9 |
| Generic Future Latent | ✓ | ✓ | 35.0 ± 1.2 | 47.6 ± 2.1 |
| Planning-Shaped Latent (Generation Only) | ✓ | × | 34.6 ± 1.2 | 47.1 ± 2.0 |
| Planning-Shaped Latent (Selection Only) | × | ✓ | 37.1 ± 1.1 | 51.3 ± 1.9 |
| Planning-Shaped Latent (Full) | ✓ | ✓ | 38.7 ± 1.3 | 53.7 ± 2.0 |
First, planning-shaped future representations outperform generic future representations in closed-loop evaluation. With both generation and selection enabled, replacing generic future supervision with planning-shaped supervision improves HD-Score from 35.0 to 38.7 (+3.7) and RC from 47.6 to 53.7 (+6.1), demonstrating that the benefits of planning-shaped representations extend to closed-loop driving.
Second, both trajectory generation and selection benefit from planning-shaped future representations. Compared with the DrivoR baseline, conditioning generation alone improves HD-Score and RC by 1.1 and 1.8 points, respectively, while conditioning selection alone yields gains of 3.6 and 6.0 points. Combining both achieves the best performance, improving HD-Score by 4.1 and 1.6 points and RC by 6.6 and 2.4 points over the generation-only and selection-only settings, respectively. These results indicate that planning-shaped representations benefit both generation and selection, with their joint use yielding the strongest closed-loop performance.
Overall, these additional HUGSIM ablations further validate the advantages of planning-shaped supervision over generic future supervision and clarify its contributions to trajectory generation and selection. We have incorporated these results and analyses into the revised manuscript.
References
[1] Y. Li et al., "End-to-End Driving with Online Trajectory Evaluation via BEV World Model," in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 27137–27146, 2025.
[2] E. Kirby et al., "Driving on Registers," in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 32058–32069, 2026.
[3] K. Guo et al., "iPad: Iterative Proposal-Centric End-to-End Autonomous Driving," IEEE Robotics and Automation Letters (RA-L), vol. 11, no. 10, pp. 11142–11149, 2026.
(1/3) Re: the Reviewer JCcM
We thank you for the careful reading of our paper and constructive comments in detail.
1. Effectiveness and Cross-Architecture Generalization of Planning-Shaped Future Representations (W1)
We thank the reviewer for this important suggestion. To further validate the effectiveness and transferability of planning-shaped representations, we conduct additional experiments on multi-seed stability, alternative future supervision objectives, and cross-architecture transfer.
(1) Performance gains and independent contributions of planning-shaped supervision.
First, to assess the stability of the 0.3 PDMS gain, we repeat the experiments with three random seeds. Generic Future Latent and Planning-Shaped Latent achieve 93.54 ± 0.05 and 93.82 ± 0.03 PDMS, respectively, yielding a consistent average improvement of 0.28 PDMS.
To further distinguish planning-shaped supervision from generic future supervision, we introduce a Future BEV Occupancy Prediction control following WoTE [1], which supervises future latents through predicted BEV occupancy. Under identical model configurations, we compare four supervision strategies:
Table R1. Future Representation Supervision Ablations.
Generic future-feature supervision and BEV occupancy supervision improve PDMS by 0.13 and 0.22, respectively, over Null Future. Planning-shaped supervision still achieves the best performance, suggesting that directly shaping future representations with planning objectives provides additional benefits beyond generic future supervision.
We also clarify that DrivoR [2] already achieves 93.1 PDMS, making a 0.3-point gain meaningful but potentially sensitive to architectural factors. The controlled comparisons above, which vary only the future supervision objective under identical model configurations, more directly isolate the contribution of planning-shaped representations.
(2) Cross-architecture transferability.
We agree that evaluation across planning architectures is important. Since PlanWAM adopts a trajectory generation-and-selection paradigm, we further integrate the proposed future representation learning mechanism into iPad [3], which follows the same paradigm but uses a different planning architecture. We compare the original iPad, generic future-feature supervision, and planning-shaped supervision:
Table R2. Transferability of Planning-Shaped Future Representations to iPad.
Under the iPad architecture, planning-shaped supervision improves performance over generic future-feature supervision by 0.6 PDMS and 2.4 HD-Score, and over the original iPad by 0.9 PDMS and 3.9 HD-Score. These results indicate that the benefits of planning-shaped representations are not specific to PlanWAM's original planning architecture, but can transfer to other architectures following the same trajectory generation-and-selection paradigm.
Overall, these additional experiments provide further evidence for the independent contribution and transferability of planning-shaped future representations, through multi-seed stability, controlled supervision comparisons, and cross-architecture evaluation. We have incorporated these results and analyses into the revised manuscript.