- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies end-to-end driving with a world-action model that co-trains a pretrained video DiT and a lightweight action DiT by joint flow matching. An isolated attention mask lets future-video prediction supervise the current-frame representation during training, while inference predicts a trajectory from the current image, ego state, and navigation command without generating future frames. Reinforcement learning then tunes the action expert on a driving reward. The central claim is that this training-time world model is enough for strong, low-latency planning. The evidence is NAVSIM Table 1 and Tables 6–9, the NAVSIM-v2 results in Tables 10–11, and the transfers in Tables 3–5.
Critical strengths
- The inference interface matches the claim. Equation 3 is the imagine-then-act factorization used by prior driving WAMs; Equation 4 removes future latents from the policy. Figure 2 and Section 4.1 keep future-frame tokens and action tokens mutually invisible, so the action expert reads only the current-frame representation. Table 6 shows the consequence: the isolated mask reaches 90.3 PDMS at 518 ms, against 90.2 PDMS at 1512 ms for bidirectional attention and 90.1 PDMS at 1499 ms for action-to-video attention.
- Future-video supervision has a measured effect on the planner, not only on video quality. In Table 9, fine-tuning the video expert with the action loss moves PDMS from 86.5 to 88.4, and adding future-video supervision moves it to 90.3. Figure 3 probes the same current-frame features: the co-trained representation lowers ADE for both surrounding-vehicle motion and the ego trajectory relative to the two ablations without that supervision.
- The decoupled interface is tested rather than only described. Table 7 swaps in LTX-Video, Wan2.1-1.3B, Cosmos-Predict2.5, and Wan2.2-5B without changing the action expert. Cosmos-Predict2.5, the driving-pretrained backbone, reaches 90.4 PDMS under imitation and 92.4 after reinforcement learning, above Wan2.2-5B at 91.9. Table 8 scales the action expert from 0.21B to 1.02B and moves imitation PDMS from 89.9 to 90.3.
- The planning result is not a single NAVSIM number. Table 1 reports 91.9 PDMS with one front camera, 0.8 above SGDrive and 1.0 above DriveVA. On the same 65K-sample, 1,000-clip PhysicalAI-AV protocol, Table 4 reports lower ADE and FDE than DriveWAM without test-time future-frame synthesis. Appendix Table 11 reports 37.6 EPDMS on reactive navhard, against 30.6 for DriveLaW. Figure 4 and Table 15 tie the reinforcement-learning gain to late-step SDE noise and the imitation anchor: removing the anchor drops PDMS to 84.3.
Critical weaknesses
- The paper frames SimWAM as a simple baseline rather than a new world-model algorithm, and the headline number should be read that way. The training-time split between video supervision and test-time imagination is the idea credited to Fast-WAM. Table 1’s 91.9 PDMS is the reinforcement-learning model in Table 9 (ID 4); the imitation model is 90.3. Table 1 does not mark the reinforcement-learning stage, and the system uses a Wan2.2-5B video expert plus a 1.02B action expert. The 0.8 PDMS margin over SGDrive is real on the reported split, but it is not a large algorithmic gap.
- The future-video contribution is isolated only before reinforcement learning. Table 9’s +1.9 PDMS (88.4 to 90.3) has no paired run that applies the same reward optimization to the action-only video fine-tune. The abstract’s 91.9 therefore mixes the world-model signal with the reward stage. Table 8’s scaling gain under imitation is 0.4 PDMS for a fourfold increase in action-expert FLOPs, so capacity alone does not explain Table 9, but the post-RL attribution remains incomplete.
- Two reported gains are too small, or too weakly held out, to carry the paper. In Table 6 the isolated mask wins by 0.1 PDMS; the supported result is the latency reduction. On WOD-E2E, reinforcement learning is run on the validation split, so the validation RFS of 8.29 is not a held-out score. On the test split the same stage moves RFS from 7.77 to 7.84 and ADE@5s from 2.69 to 2.68. No seed variance is reported for the 0.8 PDMS margin on NAVSIM.
Questions for authors
Were the Table 1 world-model and VLM rows re-evaluated in this codebase on navtest with the stated camera layouts, and does the 0.8 PDMS margin over SGDrive hold across a second seed? Some suggestions:
- Mark Table 1 as the reinforcement-learning model and report the 90.3 imitation PDMS beside it.
- Write the scalar reward used for ri in Equation 7, including which PDMS terms enter it.
- State in the Figure 1 caption whether the plotted latency is the 10-step imitation setting from Table 6.
4 comments
Response to Reviewer 4NDh 2/2
To Question: Results in Table 1 and seed variance
R: Thanks for this valuable question. Regarding the results in Table 1, we would like to clarify that they are taken from the original papers (e.g., ReCogDrive and DriveWAM) with their reported camera layouts.
Regarding the second seed, as discussed in our response to W3, the 0.8 PDMS margin over the reported 91.1 of SGDrive holds across all five seeds, with SimWAM-RL at 91.90 ± 0.05 PDMS. Notably, SimWAM-IL also surpasses SGDrive substantially on NAVSIM-v2, by 4.0 EPDMS on navtest and 10.5 EPDMS on navhard over SGDrive-IL.
To Suggestion 1: Marking IL and RL in Table 1
R: Following your suggestion, we have distinguished the IL and RL results in Table 1 and Fig. 1 of the revised manuscript.
To Suggestion 2: Scalar reward formulation
R: Thanks for pointing out this omission. The reward in Eq. 7 is the official PDM score of NAVSIM, including NC, DAC, EP, TTC and Comfort, with the EP weight raised from 5 to 10 during training. We have specified the reward in of the revised manuscript.
To Suggestion 3: Latency setting in Fig. 1
R: Thank you for this useful suggestion. We would like to clarify that the latency in Fig. 1 is measured under the 10-step IL setting of Table 6 (518 ms), and the RL model has the same latency since RL does not change the inference process.
We hope these responses address your concerns, and we would be glad to discuss any remaining questions.
I think most of my concerns are addressed. Therefore, I tend to maintain my score.
Thank you very much for your thoughtful review. We sincerely appreciate your positive assessment.
Response to Reviewer 4NDh 1/2
We thank the reviewer for the thoughtful and constructive feedback. Our responses to each point are provided below.
To W1: Comparison with SGDrive
R: Thank you for the comments. our SimWAM is a simple yet solid baseline that proves effective across multiple driving benchmarks. We have distinguished the IL and RL stages in Table 1 and compare SimWAM with SGDrive under both IL and RL. Since SGDrive does not report RL results on NAVSIM-v2, we reproduce them. The results are as follows:
On NAVSIM-v1, SimWAM outperforms SGDrive by 2.9 PDMS under IL and 0.8 PDMS after RL. More remarkably, SimWAM-IL exceeds SGDrive-IL by 4.0 EPDMS on NAVSIM-v2 navtest and substantially by 10.5 EPDMS on navhard featuring safety-critical scenarios with reactive traffic, demonstrating the superior planning performance of SimWAM. Notably, the RL of SGDrive even lowers its EPDMS on NAVSIM-v2, whereas our RL further improves it on both navtest and navhard. Additionally, SimWAM runs 32% faster than SGDrive (518 vs. 757 ms).
To W2: Future-video gain before and after RL
R: We thank the reviewer for pointing out this missing comparison. Following your suggestion, we compare the IL and RL results of the three configurations in Table 9. The results are as follows:
Across the three configurations, RL consistently improves PDMS, yet ID 2 still falls behind ID 3 after RL. Future-video supervision contributes +1.9 PDMS under IL and still +1.7 PDMS after RL (91.9 vs. 90.2). This demonstrates that future-video supervision and RL are complementary, and both contribute to the final performance of SimWAM. We have extended Table 9 with the RL results in the revised paper.
To W3: Results and seed variance
R: Thank you for these helpful comments. Regarding the mask ablation, we agree that the PDMS difference among the masks in Table 6 is marginal. More importantly, we adopt the isolated mask to avoid explicitly generating future frames at test time, thereby reducing the latency. As a result, it runs nearly 3× faster than the other masks while achieving comparable or even slightly better PDMS.
Regarding WOD-E2E, we agree that the validation RFS is not held-out, since we follow NoRD to use it as the RL reward. On the held-out test split, SimWAM achieves the highest RFS of 7.84 and the lowest ADE@5s of 2.68 among all compared methods, outperforming NoRD (7.71 RFS), which also uses the RFS for RL.
Regarding the seed variance, we re-run SimWAM with five seeds and report the mean ± std on NAVSIM-v1 navtest (PDMS), NAVSIM-v2 navtest and navhard (EPDMS).
The 0.8 PDMS margin over the reported 91.1 of SGDrive holds across all five seeds, with SimWAM-RL at 91.90 ± 0.05 PDMS. Notably, SimWAM-IL also surpasses SGDrive substantially on NAVSIM-v2, by 4.0 EPDMS on navtest and 10.5 EPDMS on navhard over SGDrive-IL.