acceptodds
Under review as a conference paper at ICLR 2027

StreamWAM: Real-Time World-Action Modeling with Closed-Loop Streaming Diffusion

Abstract

World-action models (WAMs) have recently emerged as a promising paradigm for autonomous driving, unifying future video generation and trajectory planning in a single generative framework. However, conventional diffusion-based WAMs require multiple denoising steps to generate each action chunk, leading to high inference latency and limiting the frequency of closed-loop feedback. To address this, we propose StreamWAM, a world-action model capable of real-time driving action planning by closed-loop streaming diffusion. By overlapping the denoising processes of consecutive video-action chunks, the model is able to roll out an action chunk from each inference step, even though the complete diffusion process takes multiple steps. Furthermore, we propose a high-frequency closed-loop planning method to consider the most recent observation for the intermediate action. We employ a latent alignment module to continuously correct the denoised latent states given the new observations. Experiments on the NAVSIM benchmark show that StreamWAM achieves state-of-the-art planning performance with a PDMS of 91.6 and an EPDMS of 94.3. We demonstrate that high-frequency feedback improves planning performance while the model maintains a practical 10Hz planning frequency. We believe StreamWAM represents a concrete step toward deploying world-action models in real autonomous driving systems.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, World Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CoC4OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents StreamWAM, a world-action model that jointly generates driving videos and actions using streaming diffusion. It overlaps the denoising of consecutive chunks to produce an action chunk after each forward pass following initialization. A latent alignment module updates partially denoised future video latents when new observations arrive. The method is evaluated on NAVSIM v1 and NAVSIM v2.

Critical strengths
  • The paper addresses a practical issue in diffusion-based world-action models. Compared with the evaluated AR diffusion baseline, the streaming design reduces the reported latency from 3.650s to 0.368s while maintaining the same PDMS.
  • The latent alignment module provides a reasonable way to incorporate new observations during denoising. It improves video prediction quality, reducing LPIPS from 0.227 to 0.145 compared with direct observation replacement.
Critical weaknesses
  • The method builds on existing streaming diffusion methods. While extending them to joint video-action modeling is useful, the additional planning benefit is not yet clear. The latent alignment improves video quality, but the PDMS gain over direct replacement is only 0.1. Comparisons with larger chunks or full-horizon denoising under the same backbone would also help clarify the efficiency advantage.
  • According to Appendix B.2, StreamWAM* receives logged images that don't change with its own actions. This limits the evidence for closed-loop driving, and it is unclear whether the compared methods receive the same observation updates. Longer rollouts would also help verify the claimed long-term planning benefit of alignment.
Questions for authors
  • Have you tested larger chunks or full-horizon joint denoising with the same backbone and prediction horizon?
  • Could you clarify whether the baselines receive the same observation updates, and how the generated action chunks are combined and scored?
  • Is the 0.1 PDMS improvement from alignment consistent across runs? Does latent alignment also help in longer rollouts or when new observations differ substantially from the predictions?
  • Could you explain the difference between the 77.4 PDMS at 2.5 Hz without alignment in Table 4 and the 91.5 PDMS with direct replacement in Table 6?
  • Will you plan to evaluate the method on NAVHARD or Bench2Drive? Such results would help assess its performance beyond NAVSIM v1/v2.
3 comments
AuthorsOpenReview ↗

Author Response - Part 1/2

We thank the reviewer for constructive suggestions to our paper. We provide our response below.

R1. Planning benefits of our cloesd-loop framework and LAM (W1, Q1, Q3, Q4).

Our core contribution is a high-frequency closed-loop WAM that improves planning by continually updating its trajectory from new observations. StreamWAM reuses partially denoised video and action predictions, allowing new observations to guide the remaining trajectory every 0.4 s. Table 6 shows that observation feedback through Direct Replacement raises PDMS from 88.0 to 91.5, with LAM further improving it to 91.6. The efficiency of streaming enables this frequent feedback. In Table 5, streaming diffusion reduces latency per update from 3.650 s to 0.368 s while maintaining the same PDMS of 91.6.

LAM’s small additional gain on NAVSIM may partly reflect the evaluation setting. NAVSIM replays logged images that do not respond to the model’s actions and evaluates planning over only 4 s. These constraints may limit how well it captures the planning benefits of improved video consistency, despite the LPIPS reduction from 0.227 to 0.145.

We therefore compared StreamWAM with and without LAM on the same 10 Bench2Drive Dev10 routes, keeping the model weights, controller, and random seed fixed. These interactive rollouts lasted approximately 16.7–200 s, with new observations reflecting the agent’s executed actions incorporated every 0.4 s. Adding LAM improved DS and SR, although RC decreased, supporting its contribution to driving performance over longer interactive rollouts.

SettingDS ↑SR ↑RC ↑
Direct Replacement56.0530.00%93.03%
With LAM60.8750.00%89.88%

We also evaluated larger action chunks in Table 4, using the same backbone and 4 s prediction horizon, with 4, 8, 20, and 40 waypoints per chunk. Larger chunks increase the interval between observation updates. Under this configuration, PDMS decreases from 77.4 with four waypoints to 72.7 with forty, supporting the planning benefit of more frequent feedback. As for discrepancy between Table 4 (77.4 PDMS) and Table 6 (91.5 PDMS), we apologize for the confusion about the absolute scores. Table 4 did not use the full training configuration adopted in Table 6, which explains the difference between 77.4 and 91.5 at 2.5 Hz. We have clarified these settings in the revised paper. The conclusions drawn from comparisons within each table remain unchanged.

AuthorsOpenReview ↗

Author Response - Part 2/2

R2. Evaluation fairness, trajectory construction, and closed-loop validation (W2, Q2, Q5).

We agree that StreamWAM* receives additional logged images during rollout, and Ssandard NAVSIM baselines do not receive future logged images, so its comparison with these baselines is not input-matched. Our motivation is to build a high-frequency closed-loop World-Action Model that continually updates its trajectory from new observations. Since NAVSIM cannot render camera images resulting from the model’s executed actions, we use logged images to simulate this feedback.

To simulate observation feedback, StreamWAM* generates four waypoints at 10 Hz, covering 0.4 s per chunk, and then receives the logged camera frames for that interval. Their encoded features replace the predicted video latent for the completed chunk, and LAM updates the remaining future video latents before the next chunk is generated. We concatenate ten consecutive action chunks into a 4 s trajectory, downsample it to 2 Hz, and score it with the official NAVSIM evaluator.

For a controlled comparison, we also evaluate StreamWAM without future observations, achieving 88.0 PDMS on NAVSIM v1 and 91.0 EPDMS on v2. This provides a closer comparison with standard baselines. Note that this evaluates only the open-loop variant of our framework, whereas our primary goal is to enable real-time closed-loop driving through continuous visual feedback.

To further validate interactive closed-loop driving, we evaluate NAVSIM-trained StreamWAM, DriveVA, and DriveWAM on Bench2Drive without CARLA fine-tuning. Here, every agent receives camera images resulting from its own executed actions, providing a fair comparison under the same interactive closed-loop evaluation protocol. On the same 10 Dev10 routes, StreamWAM achieves the highest DS, SR, and RC among the three methods, supporting the effectiveness of our closed-loop framework.

MethodCARLA fine-tuningDS ↑SR ↑RC ↑
DriveVANo46.7330.00%75.73%
DriveWAMNo22.670.00%50.61%
StreamWAMNo60.8750.00%89.88%

All methods use the same routes, camera geometry, route guidance, and official scorer. Adapters and controllers were calibrated on Dev10 while model weights remained fixed.

Dear Reviewer CoC4,

Thank you for your thoughtful review and constructive suggestions, which have significantly improved the quality and clarity of our paper. We sincerely hope that our rebuttal has fully addressed all of your concerns. As the discussion period is ending soon, if you need any further clarification or additional information, please do not hesitate to contact us. We truly appreciate your time and effort devoted to reviewing our work.

Best regards, Authors of Submission 11057

Reviewer_bxPfOpenReview ↗
Rating
2(−1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces StreamWAM, a streaming world-action model for real-time end-to-end autonomous driving (E2D-AD), which achieves a practical 10 Hz planning frequency.

Specifically, the paper proposes a latent alignment module that updates the predictions of the model with GT observations, enabling grounding in long horizon planning.

In addtion, the experimental results are decent.

However, the paper lacks the key comparisons with recent WAM-based methods and some experiments on closed-loop datasets.

Critical strengths

This paper introduces StreamWAM, a streaming world-action model for real-time end-to-end autonomous driving (E2D-AD), which achieves a practical 10 Hz planning frequency.

Critical weaknesses
  1. Insufficient comparisons with recent WAM-based methods. Tables 1 and 2 lack comparisons with recent WAM-based methods [1][2]. Additionally, these methods [1][2] also claim to achieve real-time planning at 10 Hz. What are the key differences between the proposed method and these existing approaches?

  2. Inconsistent claims about planning frequency. The paper claims to achieve 10 Hz planning. However, the latency reported in Table 5 is 0.368 s, which does not correspond to 10 Hz. The authors should check the results in Table 5 and the statement in Line 427.

  3. Lack of closed-loop evaluation. The authors should evaluate the proposed method on closed-loop benchmarks such as Bench2Drive or HUGSIM, since NAVSIM is essentially an open-loop evaluation benchmark.

  4. Lack of video generation evaluation. Since the proposed method is WAM-based, why are video generation results not evaluated? The authors should provide additional video generation experiments to better demonstrate the role of the world model.

[1] WAM-Flow: Parallel coarse-to-fine motion planning via discrete flow matching for autonomous driving

[2] Metis: A Generalizable and Efficient World-Action Model for Autonomous Driving and Urban Navigation

Questions for authors

see Weakness

3 comments
AuthorsOpenReview ↗

Author Response - Part 1/2

We thank the reviewer for constructive suggestions to our paper. We provide our response below.

R1. Comparison with WAM-Flow and Metis (W1).

Our core contribution is a high-frequency closed-loop WAM that continually updates its trajectory prediction as new observations arrive. Metis reduces inference cost through action-only inference, bypassing future video generation. WAM-Flow accelerates trajectory prediction through parallel denoising of discrete trajectory tokens. Their reported inference procedures generate a plan from a fixed observation context, without incorporating incoming observations into an ongoing joint video-action denoising process.

The core distinction of StreamWAM is that it jointly streams video and actions, reusing partially denoised predictions and updating them with new observations every 0.4 s. This enables real-time closed-loop replanning while processing 10 Hz camera input. The results below show the planning benefit of this feedback mechanism. StreamWAM* receives additional logged observations during rollout, improving PDMS from 88.0 to 91.6 and EPDMS from 91.0 to 94.3. We have included these comparisons in the revised paper.

MethodNAVSIM v1 PDMSNAVSIM v2 EPDMS
WAM-Flow90.384.7
Metis89.189.5
StreamWAM (no future feedback)88.091.0
StreamWAM* (logged-observation feedback)91.694.3

R2. Planning frequency and latency (W2).

Sorry for the confusion. The 0.368 s in Table 5 is the latency of one streaming update, which produces four waypoints spaced 0.1 s apart, covering 0.4 s of driving. Thus, 10 Hz is the waypoint sampling rate, while 2.5 Hz is the closed-loop replanning rate. We have corrected the statement in the revised paper to make this distinction clear.

We measured the complete model-side inference latency, from image preprocessing through action decoding, on a single NVIDIA H20 (batch size 1, BF16, 672 × 384 input). The mean inference latency is 336.30 ms per chunk, within the 400 ms replanning interval, supporting real-time closed-loop replanning.

ComponentLatency per 0.4 s chunk
Image preprocessing + online VAE encoding43.59 ms
DiT + sampler update290.26 ms
LAM/replacement2.39 ms
Action decoding/other0.06 ms
Total model-side latency336.30 ms
AuthorsOpenReview ↗

Author Response - Part 2/2

R3. Interactive closed-loop evaluation (W3).

We evaluated NAVSIM-trained StreamWAM, DriveVA, and DriveWAM on the same 10 Bench2Drive Dev10 routes, without CARLA fine-tuning. Here, each agent receives new camera observations that reflect its own executed actions, enabling interactive closed-loop evaluation. All methods use the same routes, camera geometry, route guidance, and official scorer. Adapters and controllers were calibrated on Dev10 while model weights remained fixed.

StreamWAM achieves the highest DS, SR, and RC among the three methods, supporting the effectiveness of our closed-loop framework. We have included these results in the revised paper.

MethodBench2Drive Fine-tuningDSSRRC
DriveVANone46.7330.00%75.73%
DriveWAMNone22.670.00%50.61%
StreamWAMNone60.8750.00%89.88%

R4. Video generation quality (W4).

We evaluated StreamWAM, VaViM-S, and Epona on the same NAVSIM scenes, comparing predicted front-camera images with ground truth at 1 s and 2 s, using 256 × 448 resolution and identical metric implementations. Epona receives logged future ego motion, VaViM-S generates video without action conditioning, and StreamWAM jointly predicts motion and video. StreamWAM achieves the highest PSNR and SSIM and the lowest LPIPS, indicating more accurate future-frame prediction.

To assess LAM’s contribution, we also compare it with Direct Replacement under the same observation-feedback setting over a 4 s rollout. Adding LAM reduces LPIPS from 0.227 to 0.145, a 36.1% reduction, supporting its role in correcting future video predictions as new observations arrive. We have included these comparisons in the revised paper.

MethodMotion conditionPSNR ↑SSIM ↑LPIPS ↓
EponaLogged future ego motion18.5620.5180.267
VaViM-SAction-free20.0240.5750.255
StreamWAMJointly predicted ego motion21.4420.6300.227

Dear Reviewer bxPf,

Thank you for your thoughtful review and constructive suggestions, which have significantly improved the quality and clarity of our paper. We sincerely hope that our rebuttal has fully addressed all of your concerns. As the discussion period is ending soon, if you need any further clarification or additional information, please do not hesitate to contact us. We truly appreciate your time and effort devoted to reviewing our work.

Best regards, Authors of Submission 11057

Reviewer_wG7kOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

Summary This paper proposes StreamWAM, a world-action model for real-time autonomous driving that jointly generates future observations and driving actions. The method employs streaming diffusion to overlap the denoising processes of consecutive video-action chunks, enabling low-latency action generation. A latent alignment module is further introduced to incorporate new observations into partially denoised future latents. Experiments on NAVSIM v1 and v2 report competitive planning performance, achieving 91.6 PDMS and 94.3 EPDMS under the proposed closed-loop evaluation setting.

Overall The paper presents a practically motivated and technically reasonable framework for low-latency world-action modeling, supported by promising experimental results. However, the non-standard evaluation protocol raises substantial concerns about the fairness of the reported improvements, particularly for the claimed closed-loop performance. The unexplained differences in the ablation results and the limited evidence connecting latent alignment to improved planning further weaken the experimental justification. While the approach is promising, these issues should be clarified before the main conclusions can be considered sufficiently supported.

Critical strengths

Strengths

  1. Practical Motivation. The paper addresses an important limitation of diffusion-based world-action models: high inference latency and insufficient feedback frequency, which are relevant to real-time autonomous driving.
  2. Effective Framework. The combination of streaming diffusion, unified video-action modeling, and latent alignment provides a coherent framework for continuous action generation. The proposed method significantly reduces the interval between consecutive action chunks while preserving planning quality.
  3. Promising Experimental Results. StreamWAM reports competitive planning performance on both NAVSIM benchmarks. The ablations demonstrate the effectiveness of causal attention and substantial improvements in video generation consistency through latent alignment.
Critical weaknesses

Weaknesses

  1. Concerns About Evaluation Fairness. The closed-loop evaluation uses future ground-truth observations from recorded driving logs, which are unavailable to standard NAVSIM baselines. Therefore, directly comparing StreamWAM* against these methods may be misleading. Moreover, the observations do not reflect the consequences of the model's executed actions, limiting the evidence for interactive closed-loop driving. The use of additional 10 Hz sensor data also raises questions about comparability with the standard NAVSIM input protocol.
  2. Incomplete Real-Time Latency Analysis. Although the paper emphasizes 10 Hz planning, new action chunks and observation-conditioned updates occur every 0.4 seconds, corresponding to a 2.5 Hz closed-loop update frequency. Furthermore, the reported latency primarily reflects a single DiT forward pass, without a complete breakdown of VAE processing, latent alignment, and initialization overhead.
  3. Unclear Ablations and Limited Planning Benefits of Alignment. Table 4 reports a PDMS of 77.4 at 2.5 Hz without latent alignment, while Table 6 reports 91.5 for direct replacement, an apparently similar setting. This substantial discrepancy requires clarification. Additionally, latent alignment improves PDMS by only 0.1 over direct replacement, despite significantly improving LPIPS. Its claimed benefits for long-horizon planning remain insufficiently demonstrated.
Questions for authors

Questions for Authors

  1. Could the authors clarify how StreamWAM* trajectories are constructed and evaluated under NAVSIM? How can direct comparisons with standard baselines be considered fair when additional future observations are used? Does the 10 Hz sensor input also follow the same evaluation protocol?
  2. What explains the substantial PDMS discrepancy between the 2.5 Hz setting in Table 4 (77.4) and direct replacement in Table 6 (91.5)? Are there differences in their experimental configurations or evaluation procedures?
  3. Could the authors report complete end-to-end inference latency, including initialization, VAE processing, and latent alignment? Additionally, can longer-horizon evaluations demonstrate that improved video consistency translates into meaningful planning improvements?
3 comments
AuthorsOpenReview ↗

Author Response - Part 1/2

We thank the reviewer for constructive suggestions to our paper. We provide our response below.

R1. Evaluation fairness, trajectory construction, and interactive evidence (W1, Q1).

We agree that StreamWAM* receives additional logged images during rollout, so its comparison with standard NAVSIM baselines is not input-matched. Our motivation is to build a high-frequency closed-loop World-Action Model that continually updates its trajectory from new observations. Since NAVSIM cannot render camera images resulting from the model’s executed actions, we use logged images to simulate this feedback. Regarding input frequency, standard NAVSIM uses 2 Hz historical input to reduce computation, while StreamWAM efficiently processes native 10 Hz nuPlan frames within the same available history window. We consider this comparison fair in terms of history horizon coverage, as both settings use the same driving logs and evaluation scenes. The denser visual history makes practical use of our model’s efficiency.

To simulate observation feedback, StreamWAM* generates four waypoints at 10 Hz, covering 0.4 s per chunk, and then receives the logged camera frames for that interval. Their encoded features replace the predicted video latent for the completed chunk, and LAM updates the remaining future video latents before the next chunk is generated. We concatenate ten consecutive action chunks into a 4 s trajectory, downsample it to 2 Hz, and score it with the official NAVSIM evaluator.

For a controlled comparison, we also evaluate StreamWAM without future observations, achieving 88.0 PDMS on NAVSIM v1 and 91.0 EPDMS on v2. This provides a closer comparison with standard baselines. Note that this evaluates only the open-loop variant of our framework, whereas our primary goal is to enable real-time closed-loop driving through continuous visual feedback.

To further validate interactive closed-loop driving, we evaluate NAVSIM-trained StreamWAM, DriveVA, and DriveWAM on Bench2Drive without CARLA fine-tuning. Here, every agent receives camera images resulting from its own executed actions, providing a fair comparison under the same interactive closed-loop evaluation protocol. On the same 10 Dev10 routes, StreamWAM achieves the highest DS, SR, and RC among the three methods, supporting the effectiveness of our closed-loop framework.

MethodBench2Drive Fine-tuningDSSRRC
DriveVANone46.7330.00%75.73%
DriveWAMNone22.670.00%50.61%
StreamWAMNone60.8750.00%89.88%

All methods use the same routes, camera geometry, route guidance, and official scorer. Adapters and controllers were calibrated on Dev10 while model weights remained fixed.

R2. Replanning frequency and complete inference latency (W2, Q3).

StreamWAM outputs four waypoints spaced 0.1 s apart and updates its trajectory prediction with new observations every 0.4 s. Thus, 10 Hz is the waypoint sampling rate, while 2.5 Hz is the closed-loop replanning rate. We have clarified this distinction in the revised paper.

We measured the complete model-side inference latency, from image preprocessing through action decoding, on a single NVIDIA H20 (batch size 1, BF16, 672 × 384 input). The mean inference latency is 336.30 ms per chunk, within the 400 ms replanning interval, supporting real-time closed-loop replanning. The breakdown below includes Image preprocessing, VAE encoding, DiT forward, latent alignment, and action decoding steps.

ComponentLatency per 0.4 s chunk
Image preprocessing + online VAE encoding43.59 ms
DiT + sampler update290.26 ms
LAM/replacement2.39 ms
Action decoding/other0.06 ms
Total model-side latency336.30 ms
AuthorsOpenReview ↗

Author Response - Part 2/2

R3. Discrepancy between Tables 4 and 6 (W3, Q2).

Sorry for the confusion. The loop-frequency ablation in Table 4 did not use the full training configuration adopted in Table 6. This configuration difference explains the lower PDMS at 2.5 Hz in Table 4 (77.4), compared with Direct Replacement in Table 6 (91.5). We will clarify the experimental settings in the revised paper. The conclusions drawn from the comparisons within the table remain unchanged.

R4. LAM's planning contribution and long-horizon behavior (W3, Q3).

NAVSIM replays logged images that do not reflect the model’s executed actions. Its samples are also short, covering approximately 7 s, with planning evaluated over a 4 s horizon. These constraints may limit how well the evaluation captures the planning benefits of improved video consistency, helping explain the small PDMS gain from 91.5 to 91.6 despite the LPIPS improvement from 0.227 to 0.145.

To assess LAM’s contribution during longer interactive driving, we compared StreamWAM with and without LAM on the same 10 Bench2Drive Dev10 routes, keeping the model weights, controller, and random seed fixed. The rollouts lasted about 16.7–200 s, with new observations reflecting the agent’s actions incorporated every 0.4 s. Adding LAM improved DS from 56.05 to 60.87 and SR from 30.00% to 50.00%, although RC decreased from 93.03% to 89.88%. These results support LAM’s contribution to driving score and successful route completion over longer interactive closed-loop rollouts.

StreamWAM settingDSSRRC
Direct Replacement (no LAM)56.0530.00%93.03%
LAM60.8750.00%89.88%

Dear Reviewer wG7k,

Thank you for your thoughtful review and constructive suggestions, which have significantly improved the quality and clarity of our paper. We sincerely hope that our rebuttal has fully addressed all of your concerns. As the discussion period is ending soon, if you need any further clarification or additional information, please do not hesitate to contact us. We truly appreciate your time and effort devoted to reviewing our work.

Best regards, Authors of Submission 11057

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

We thank the reviewers for their constructive feedback. Sorry for the late reply, as we spent considerable time setting up Bench2Drive and completing the closed-loop experiments.

Our additional results strengthen the paper in three main areas. First, StreamWAM achieves a full model-side inference latency of 336.30 ms, within the 400 ms replanning interval, supporting real-time closed-loop replanning. Second, we evaluate zero-shot transfer to Bench2Drive using NAVSIM-trained weights without CARLA fine-tuning. Here, each agent receives new observations that reflect its own actions, enabling interactive closed-loop evaluation. On the 10-route Dev10 split, StreamWAM achieves 60.87 DS, compared with 46.73 for DriveVA and 22.67 for DriveWAM. Third, we evaluate StreamWAM with and without LAM under the same Bench2Drive settings. Adding LAM improves DS from 56.05 to 60.87, demonstrating its benefit to driving performance over longer interactive closed-loop rollouts.

AuthorsOpenReview ↗

Summary of the Author-Reviewer Discussion

Dear PCs, SACs, ACs and all of our reviewers,

We sincerely thank you for your time and effort in reviewing our paper, as well as your constructive feedback. We are grateful for your comments highlighting the strengths of our work:

  • A streaming world-action model that addresses limitations of current diffusion-based WAMs, including high inference latency and inability to promptly adjust to visual feedbacks. The design combines streaming diffusion with latent alignment to achieve not only real-time inference, but also closed-loop driving.
  • Promising experimental results, including competitive planning performance on NAVSIM v1 and v2 benchmarks, as well as low inference latency.

We have provided additional experiments and explanations to address the reviewers' concerns, and updated the manuscript accordingly:

  • Evaluation fairness and closed-loop validation. Our evaluation on NAVSIM uses future logged video, which is unavailable in the standard evaluation. For a controlled comparison, we evaluate StreamWAM without future observations, achieving 88.0 PDMS and 91.0 EPDMS. We further validate closed-loop driving on Bench2Drive without finetuning. Our model outperforms DriveVA and DriveWAM with a Driving Score of 60.87, compared to 46.73 and 22.67 respectively (wG7k, bxPf, CoC4).
  • Clarification about frequency and latency. We clarify that 10 Hz is the sampling rate of video and action, while 2.5 Hz is the closed-loop replanning rate. We also measure the complete model-side inference latency, which had an average of 336.30 ms per chunk (wG7k, bxPf).
  • Planning benefits of LAM. We assess LAM's contribution during long-term interactive driving on 10 Bench2Drive Dev10 routes. We find that compared to Direct Replacement, LAM increases DS by 4.82 points, SR by 20.00% but decreases RC by 3.15%. These results support LAM's contribution to planning over long interactive closed-loop rollouts (wG7k, CoC4).
  • Discrepancy between Tables 4 and 6. We clarify that Table 4 was not tested on the full training configuration as Table 6, resulting in the lower PDMS. However, since the configurations within the table is consistent, the conclusion drawn from the comparisons remain unchanged (wG7k).
  • Comparison with WAM-Flow and Metis. We compare StreamWAM to WAM-Flow and Metis. First, we explain that the core distinction of our work is that it jointly streams video and actions. Second, we compare the PDMS and EPDMS results, finding them to be superior to the two competing methods (bxPf).
  • Video Generation Quality. We evaluate the video generation quality of StreamWAM on NAVSIM scenes and compare it to VaViM-S and Epona. StreamWAM achieves the highest PSNR and SSIM and the lowest LPIPS, indicating more accurate future video prediction (bxPf).

Thank you for your time and efforts.

Best regards, Authors of Submission 11057

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.