- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents StreamWAM, a world-action model that jointly generates driving videos and actions using streaming diffusion. It overlaps the denoising of consecutive chunks to produce an action chunk after each forward pass following initialization. A latent alignment module updates partially denoised future video latents when new observations arrive. The method is evaluated on NAVSIM v1 and NAVSIM v2.
Critical strengths
- The paper addresses a practical issue in diffusion-based world-action models. Compared with the evaluated AR diffusion baseline, the streaming design reduces the reported latency from 3.650s to 0.368s while maintaining the same PDMS.
- The latent alignment module provides a reasonable way to incorporate new observations during denoising. It improves video prediction quality, reducing LPIPS from 0.227 to 0.145 compared with direct observation replacement.
Critical weaknesses
- The method builds on existing streaming diffusion methods. While extending them to joint video-action modeling is useful, the additional planning benefit is not yet clear. The latent alignment improves video quality, but the PDMS gain over direct replacement is only 0.1. Comparisons with larger chunks or full-horizon denoising under the same backbone would also help clarify the efficiency advantage.
- According to Appendix B.2, StreamWAM* receives logged images that don't change with its own actions. This limits the evidence for closed-loop driving, and it is unclear whether the compared methods receive the same observation updates. Longer rollouts would also help verify the claimed long-term planning benefit of alignment.
Questions for authors
- Have you tested larger chunks or full-horizon joint denoising with the same backbone and prediction horizon?
- Could you clarify whether the baselines receive the same observation updates, and how the generated action chunks are combined and scored?
- Is the 0.1 PDMS improvement from alignment consistent across runs? Does latent alignment also help in longer rollouts or when new observations differ substantially from the predictions?
- Could you explain the difference between the 77.4 PDMS at 2.5 Hz without alignment in Table 4 and the 91.5 PDMS with direct replacement in Table 6?
- Will you plan to evaluate the method on NAVHARD or Bench2Drive? Such results would help assess its performance beyond NAVSIM v1/v2.
3 comments
Author Response - Part 2/2
R2. Evaluation fairness, trajectory construction, and closed-loop validation (W2, Q2, Q5).
We agree that StreamWAM* receives additional logged images during rollout, and Ssandard NAVSIM baselines do not receive future logged images, so its comparison with these baselines is not input-matched. Our motivation is to build a high-frequency closed-loop World-Action Model that continually updates its trajectory from new observations. Since NAVSIM cannot render camera images resulting from the model’s executed actions, we use logged images to simulate this feedback.
To simulate observation feedback, StreamWAM* generates four waypoints at 10 Hz, covering 0.4 s per chunk, and then receives the logged camera frames for that interval. Their encoded features replace the predicted video latent for the completed chunk, and LAM updates the remaining future video latents before the next chunk is generated. We concatenate ten consecutive action chunks into a 4 s trajectory, downsample it to 2 Hz, and score it with the official NAVSIM evaluator.
For a controlled comparison, we also evaluate StreamWAM without future observations, achieving 88.0 PDMS on NAVSIM v1 and 91.0 EPDMS on v2. This provides a closer comparison with standard baselines. Note that this evaluates only the open-loop variant of our framework, whereas our primary goal is to enable real-time closed-loop driving through continuous visual feedback.
To further validate interactive closed-loop driving, we evaluate NAVSIM-trained StreamWAM, DriveVA, and DriveWAM on Bench2Drive without CARLA fine-tuning. Here, every agent receives camera images resulting from its own executed actions, providing a fair comparison under the same interactive closed-loop evaluation protocol. On the same 10 Dev10 routes, StreamWAM achieves the highest DS, SR, and RC among the three methods, supporting the effectiveness of our closed-loop framework.
| Method | CARLA fine-tuning | DS ↑ | SR ↑ | RC ↑ |
|---|---|---|---|---|
| DriveVA | No | 46.73 | 30.00% | 75.73% |
| DriveWAM | No | 22.67 | 0.00% | 50.61% |
| StreamWAM | No | 60.87 | 50.00% | 89.88% |
All methods use the same routes, camera geometry, route guidance, and official scorer. Adapters and controllers were calibrated on Dev10 while model weights remained fixed.
Dear Reviewer CoC4,
Thank you for your thoughtful review and constructive suggestions, which have significantly improved the quality and clarity of our paper. We sincerely hope that our rebuttal has fully addressed all of your concerns. As the discussion period is ending soon, if you need any further clarification or additional information, please do not hesitate to contact us. We truly appreciate your time and effort devoted to reviewing our work.
Best regards, Authors of Submission 11057
Author Response - Part 1/2
We thank the reviewer for constructive suggestions to our paper. We provide our response below.
R1. Planning benefits of our cloesd-loop framework and LAM (W1, Q1, Q3, Q4).
Our core contribution is a high-frequency closed-loop WAM that improves planning by continually updating its trajectory from new observations. StreamWAM reuses partially denoised video and action predictions, allowing new observations to guide the remaining trajectory every 0.4 s. Table 6 shows that observation feedback through Direct Replacement raises PDMS from 88.0 to 91.5, with LAM further improving it to 91.6. The efficiency of streaming enables this frequent feedback. In Table 5, streaming diffusion reduces latency per update from 3.650 s to 0.368 s while maintaining the same PDMS of 91.6.
LAM’s small additional gain on NAVSIM may partly reflect the evaluation setting. NAVSIM replays logged images that do not respond to the model’s actions and evaluates planning over only 4 s. These constraints may limit how well it captures the planning benefits of improved video consistency, despite the LPIPS reduction from 0.227 to 0.145.
We therefore compared StreamWAM with and without LAM on the same 10 Bench2Drive Dev10 routes, keeping the model weights, controller, and random seed fixed. These interactive rollouts lasted approximately 16.7–200 s, with new observations reflecting the agent’s executed actions incorporated every 0.4 s. Adding LAM improved DS and SR, although RC decreased, supporting its contribution to driving performance over longer interactive rollouts.
We also evaluated larger action chunks in Table 4, using the same backbone and 4 s prediction horizon, with 4, 8, 20, and 40 waypoints per chunk. Larger chunks increase the interval between observation updates. Under this configuration, PDMS decreases from 77.4 with four waypoints to 72.7 with forty, supporting the planning benefit of more frequent feedback. As for discrepancy between Table 4 (77.4 PDMS) and Table 6 (91.5 PDMS), we apologize for the confusion about the absolute scores. Table 4 did not use the full training configuration adopted in Table 6, which explains the difference between 77.4 and 91.5 at 2.5 Hz. We have clarified these settings in the revised paper. The conclusions drawn from comparisons within each table remain unchanged.