acceptodds
Withdrawn from ICLR 2027

Towards Self-Improving Autonomous Driving through Imagined Interaction

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-Language-Action (VLA) models have shown strong potential for autonomous driving, yet their learning remains largely confined to fixed offline demonstrations. Such offline training provides little supervision from the states and consequences induced by the policy’s own decisions, limiting continual self-improvement. We present InteractVLA, a framework for self-improving autonomous driving through imagined interaction. Rather than using predicted futures only for reasoning or action evaluation, InteractVLA converts action-conditioned future transitions into multi-step policy-induced experience for policy optimization. To enable efficient imagined interaction, we introduce an Asymmetric Action-Transition Dual Transformer that jointly performs action generation and transition modeling within a single forward pass, while compact transition tokens capture action-induced changes without reconstructing complete future observations. We further propose interaction-aware feedback to maintain reliable supervision as imagined states deviate from logged traffic, together with an interaction-specific return-to-go objective for temporal credit assignment under GRPO. Experiments on NAVSIM and Bench2Drive demonstrate strong planning performance and consistent gains under successive interaction. InteractVLA achieves 93.2 PDMS on NAVSIM and a 91.13 Driving Score on Bench2Drive, and reaches 88.3 PDMS under 5-second continuation on NAVSIM. These results demonstrate the effectiveness of imagined interaction for self-improving autonomous driving.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, Vision-Language-Action Models, World Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Zc8eOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents InteractVLA, a learning framework to enable self‑improving vision‑language‑action (VLA) driving models via imagined interaction. The core components include an Asymmetric Action‑Transition Dual Transformer that produces actions and compact scene‑change transition tokens in one forward pass, interaction‑aware adaptive feedback fusion for drifted imagined states, and an interaction‑specific return‑to‑go objective optimized with GRPO. Transition‑related modules are discarded during inference so that no extra overhead is introduced at deployment. Evaluations are conducted on NAVSIM and Bench2Drive. InteractVLA achieves strong standard planning performance.

Critical strengths
  1. By converting imagined multi‑step rollouts into usable training signals rather than only using them for single‑step reasoning, this work provides a promising direction for model‑based self‑supervised improvement in autonomous driving.

  2. The proposed Asymmetric Action‑Transition Dual Transformer leverages compact transition tokens to represent scene dynamics instead of reconstructing full pixel‑level future observations, which significantly cuts the computation cost of imagined rollouts.

  3. The interaction‑aware feedback mechanism adaptively mixes replay‑based ground‑truth evaluation and learned reward‑model estimates according to visual correspondence.

  4. The framework separates training‑time imagined‑interaction machinery from inference pipelines. Ablations demonstrate favorable latency compared to competing world‑model‑enhanced baselines, making it potentially applicable for real‑world driving VLA development.

Critical weaknesses
  1. After Stage III the transition pathway, decoder, and Reward Model are discarded. Inference keeps only the vision encoder, VLA, and action de-tokenizer. There is no test-time imagination, no online update, and no multi-round improvement curve. That is one-shot model-based policy optimization, not self-improving autonomous driving.

  2. The Reward Model is trained with replay derived NC/TTC while the action transition model stays fixed. Fusion (Eq. 7) trusts the RM when visual cosine similarity is low, the regime in which replay was declared invalid. DAC, EP, and Comfort always remain replay. Stage III still supervises transition and reward on logged data. GRPO can exploit world-model and RM error; that is unmeasured.

  3. Table 1 reports a PDMS of 93.6 for ChainFlow-VLA, compared with 93.2 for InteractVLA. This relatively small margin alone does not appear sufficient to support a broad state-of-the-art claim, particularly when several reported experimental results do not consistently establish superiority over competing methods.

  4. The InteractVLA results reported in Table 1 appear inconsistent with the corresponding results reported in its original work, raising concerns about the reliability of this baseline comparison.

  5. Although adaptive feedback alleviates evaluation drift, the paper does not systematically quantify how transition‑model prediction errors accumulate over longer imagined rollouts. It also lacks controlled ablations on how rollout length trades off between training‑data diversity and simulation‑error contamination.

  6. Despite being a VLA‑driving work, all evaluations focus on general trajectory‑planning tasks. The paper does not test whether imagined interaction improves language‑instructed driving performance, which is a natural use case for VLA systems.

Questions for authors
  1. Does the method actually support self-improving driving at inference time, or does it only produce a better policy through offline training?

  2. Can the policy exploit errors in the learned transition model or Reward Model, and how is this risk controlled?

  3. Are the reported baseline results and SOTA claims sufficiently reliable and reproducible?

  4. Does the proposed imagination mechanism improve the language-conditioned driving capabilities that motivate the use of a VLA in the first place?

4 comments
AuthorsOpenReview ↗

Response to Reviewer Zc8e (1/2)

We thank the reviewer for the detailed and technically insightful feedback. We particularly appreciate the positive assessment of our imagined-interaction framework, compact transition representations, adaptive feedback mechanism, and inference efficiency. We address the concerns below and clarify the scope of the proposed method.

W1 / Q1. Clarification and Evidence of Self-Improving Autonomous Driving

Definition of Self-Improving Autonomous Driving. We define self-improvement as a policy improving its capabilities by learning from self-generated experiences rather than relying solely on fixed expert demonstrations, consistent with prior work [1, 2]. Our framework focuses on training-time self-improvement, rather than test-time imagination or online adaptation. Specifically, at each GRPO iteration, the updated policy generates fresh action-conditioned imagined interactions, learns from their consequences, and uses the improved policy to generate new experiences for subsequent optimization (Algorithm 1). This establishes an iterative interaction-learning process rather than one-shot optimization over fixed imagined experiences.

Evidence of Progressive Improvement. We provide additional experimental evidence in Figs. 4 and 5 of the revised manuscript. Fig. 4 shows that increasing the training interaction horizon from one to four steps improves PDMS from 90.8 to 93.4, demonstrating the benefit of multi-step imagined experiences. More importantly, Fig. 5 directly tracks policy performance throughout Stage III optimization: 4-s PDMS progressively increases from 86.4 to 93.2 (+6.8), while 5-s PDMS improves from 79.5 to 88.3 (+8.8). These results provide direct evidence that the policy progressively improves through iterative learning from imagined interactions.

References

[1] Chen et al. Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models. ICML 2024.

[2] Pang et al. Language Model Self-improvement by Reinforcement Learning Contemplation. ICLR 2024.


W2 / Q2. Reliability of Imagined Feedback and Reward Exploitation

We agree that errors in learned transition and reward models may be exploited during policy optimization. We mitigate this risk through perturbation-aware reward learning, adaptive feedback, and additional reliability evaluations.

Reward Model Reliability. To improve reward estimation beyond expert demonstrations, we employ perturbation-aware supervision with four trajectory perturbations: speed reduction, speed increase, yaw deviation, and unnecessary stops. On the unseen NAVSIM v1 navtest split, the demonstration-only model predicts nearly uniform high rewards, whereas our model closely matches ground-truth PDMS across different perturbations (Appendix D.6, Table 12):

PerturbationGT PDMSDemo-only RMOurs
Speed Reduction0.9190.9320.921
Speed Increase0.8620.9410.856
Yaw Deviation0.7160.9340.723
Unnecessary Stop0.6270.9310.637

We further evaluate candidate ranking reliability using 32 trajectories per scene on NAVSIM v1 navtest. Our Reward Model achieves a Spearman correlation of 0.570 and a pairwise ranking accuracy of 78.46%, compared with 0.551 and 77.11% reported by WAC [1], respectively. These results provide additional evidence that our reward estimates can distinguish trajectory quality on unseen scenes.

Adaptive Feedback. Eq. 7 adaptively combines replay-based and learned NC/TTC estimates according to visual correspondence, reducing reliance on replay evaluation as imagined states diverge from logged observations. DAC, EP, and Comfort remain replay-based. As shown in Tables 3 and 5, our adaptive fusion achieves 88.3 PDMS, outperforming replay-only (81.4), learned-only (84.9), and fixed fusion (86.5), demonstrating the benefit of combining complementary feedback sources.

Transition Model Reliability. We additionally conduct an action-matched ground-truth replacement experiment to quantify the effect of transition prediction errors on continued planning below. Predicted transitions achieve 88.6 PDMS, only 0.5 points below GT transition tokens, indicating limited downstream degradation from transition prediction errors under the evaluated conditions (Table 6 in manuscript).

Second-step State5s PDMS ↑
GT Visual Tokens (Oracle)89.7
GT Transition Tokens + Decoder89.1
Predicted Transitions (Ours)88.6
Randomized Transition Outputs41.4

Together, these results demonstrate the reliability of our reward supervision and imagined transitions, while providing empirical evidence that perturbation-aware supervision and adaptive feedback effectively mitigate model-induced errors during imagined-interaction optimization.

References

[1] Zhang et al. World Action Critic (WAC) Model: Joint Foresight and Criticism for Policy Learning. arXiv:2610.22825, 2026.

AuthorsOpenReview ↗

Response to Reviewer Zc8e (2/2)

W3 / Q3. Comparison with ChainFlow-VLA and SOTA claims

We would like to highlight an important difference in the comparison with ChainFlow-VLA. ChainFlow-VLA uses four camera views and a learned trajectory scorer to select among multiple refined candidates, whereas InteractVLA achieves 93.2 PDMS using only a single front-view camera, without additional trajectory scoring at inference time. Despite this more restrictive configuration, InteractVLA achieves the highest PDMS among the single-front-view methods in Table 1, only 0.4 points below ChainFlow-VLA (93.6).

To further assess performance on more challenging benchmarks, we evaluate InteractVLA on NAVSIM v2 (navtest) and NAVHARD, comparing against recent VLA-based and world-model-based methods, as detailed in Table 8 of Appendix D.1 in the revised manuscript. InteractVLA achieves 91.7 EPDMS on NAVSIM v2 and 41.2 EPDMS on NAVHARD, outperforming all compared methods on both benchmarks. These results further demonstrate the effectiveness of InteractVLA under more challenging planning settings.

Moreover, InteractVLA achieves a 91.13 Driving Score and 76.27% Success Rate on Bench2Drive, demonstrating strong closed-loop driving performance. Overall, these results highlight the competitiveness of InteractVLA across different benchmarks, with minimal visual input and no additional inference-time trajectory scoring.


W4. Possible inconsistency in baseline results

We have carefully re-examined the baseline results in Table 1 against their original publications and updated the reported results of DriveVA and SimWAM according to the latest versions of their respective papers. These corrections have been incorporated into Table 1 of the revised manuscript to ensure that our comparisons accurately reflect the most recent reported results.


W5. Error accumulation and interaction-horizon analysis

To quantify error accumulation, we compare GT-Conditioned Prediction (GT) and Self-Conditioned Rollout (SC) using identical ground-truth actions in the Table below (Appendix D.8, Table 14 (a)). GT predicts each future observation from its corresponding ground-truth observation, whereas SC recursively conditions on its own predicted observations. GT maintains approximately 23 dB, while SC decreases to 18.39 dB, quantifying prediction-error accumulation during recursive imagination.

Prediction DepthGT PSNR ↑SC PSNR ↑
123.1423.14
223.0322.57
322.7621.02
423.4018.39

We further evaluate training-data diversity using the group-wise average pairwise L2 distance between sampled trajectories over a fixed time window, ensuring that differences are not simply caused by longer trajectory durations (Appendix D.8, Table 14 (b)). As the interaction horizon increases, trajectory diversity rises substantially, while planning improvements gradually saturate. Extending from three to four steps yields only 0.2 additional PDMS points, despite increasing training time from approximately 3 to 10 days.

Training Interaction StepsTrajectory Diversity (L2, m) ↑PDMS ↑
10.3690.8
21.2692.4
32.3193.2
44.5893.4

Together, these results quantify the trade-off between richer imagined experiences and accumulating prediction errors, motivating our three-step interaction horizon. We further mitigate these risks through logged-state initialization, horizon curriculum, and interaction-aware feedback.


W6 / Q4. Language-conditioned driving capabilities

We clarify that language-based navigation instructions are provided as conditioning inputs to InteractVLA in both NAVSIM and Bench2Drive evaluations. Our method focuses on improving driving decisions through imagined interactions under language-conditioned instructions, rather than enhancing language understanding in isolation. To further demonstrate this improvement, we compare the policy before and after imagined-interaction optimization across five Bench2Drive driving skills (Table 9, Appendix D.2). The results show that imagined-interaction optimization improves all five driving skills, increasing the mean success rate from 67.73% to 74.45%, with particularly notable gains in overtaking and give-way scenarios. These results demonstrate that imagined interactions effectively improve driving performance under language-conditioned navigation instructions. We have clarified the role of language conditioning and the scope of our evaluation in the revised manuscript.

AuthorsOpenReview ↗

Follow-up on Rebuttal

Thank you again for your detailed and insightful feedback. We have carefully addressed your concerns and substantially strengthened the revised manuscript with additional evidence on progressive self-improvement, reward and transition reliability, rollout error accumulation, baseline comparisons, and language-conditioned driving performance.

We would greatly appreciate it if you could revisit our rebuttal and revised manuscript and let us know whether your main concerns have been resolved. We would welcome any further questions or suggestions during the remaining discussion period.

Thank you very much for your time and consideration!

Dear Reviewer Zc8e,

We would like to gently follow up on our previous responses. We would be grateful if you could revisit our revised manuscript and share whether any major concerns remain. We would be happy to provide further clarification before the discussion period ends.

Thank you very much for your time and consideration!

Reviewer_4BbxOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

InteractVLA combines a world model and a reward model into a unified and effective closed-loop training pipeline, an important direction in autonomous driving. However, its claimed self-improvement process has not been sufficiently discussed.

Critical strengths
  1. Closed-loop driving is a key problem in autonomous driving, especially closed-loop training, which can be very costly. The paper presents a pipeline for closed-loop training with a world model.

  2. The experiments are fairly detailed, and the model performs well on the benchmarks. The writing and figures are easy to follow.

Critical weaknesses
  1. The paper presents self-improvement as its main claim, but the experiments do not show how performance changes as the policy is updated. It remains unclear which driving abilities and policy modules improve through the imagined interactions. The paper could analyze improvements across different driving scenarios or freeze selected policy modules to identify which modules benefit most. This analysis is essential to understanding self-improvement; reporting only the final results is not enough.

  2. The method is essentially a world-model-based approach to closed-loop training. It should therefore be compared with closed-loop training directly based on a simulator in terms of both performance and efficiency. Bench2Drive, which the paper already uses, provides a complete simulation-based environment for such a comparison.

  3. The training pipeline includes a learned reward model. The paper should evaluate the accuracy of its outputs and discuss its generalization. For example, it could test whether the reward model can reliably select better trajectories on unseen data. This would help show that the reward model has not overfit.

Questions for authors

Please refer to the weaknesses section. I am willing to raise the score if the authors can address my concerns.

6 comments
AuthorsOpenReview ↗

Response to Reviewer 4Bbx (1/2)

We thank the reviewer for the thoughtful and constructive assessment. We particularly appreciate the recognition of the importance of closed-loop training, the effectiveness of our framework, and the quality of the experimental presentation. We address the three concerns below.

W1. Analysis of Policy Self-Improvement

To directly demonstrate progressive policy self-improvement, we evaluate intermediate policy checkpoints throughout Stage III training (Fig. 5 in the revised paper). The 4s PDMS consistently increases from 86.4 to 93.2 (+6.8 points), while the 5s PDMS improves from 79.5 to 88.3 (+8.8 points), demonstrating continuous performance gains through iterative learning from imagined interactions.

To further investigate which driving capabilities improve, we compare the policy before (Stage I) and after (Full) imagined-interaction optimization across five Bench2Drive driving skills, alongside representative published methods (Appendix D.2, Table 9).

MethodVenueMerge ↑Overtake ↑EmBrake ↑GiveWay ↑TSign ↑Mean ↑
SimLingoCVPR 202553.7867.4181.6750.0077.2066.01
LinkVLACVPR 202660.0080.0093.3350.0083.6873.40
BLUEEMNLP 202661.4480.0093.2750.0084.7473.89
InteractVLA (Stage I)Ours58.7566.6791.6740.0081.5867.73
InteractVLA (Full)Ours62.5082.2293.3350.0084.2174.45

Imagined-interaction optimization improves all five driving skills, increasing the mean success rate from 67.73% to 74.45% (+6.72 points). The largest gains occur in overtaking (+15.55) and give-way (+10.00), which particularly involve interactions with surrounding traffic.

We further analyze the contribution of auxiliary modules during Stage III by disabling their supervised updates while retaining VLA optimization through GRPO (Sec 4.4, Table 7). Freezing the transition pathway reduces 4s/5s PDMS from 93.2/88.3 to 91.9/85.5, while freezing the Reward Model yields 92.2/87.8. Freezing both further reduces performance to 90.4/83.1. The larger degradation from freezing the transition pathway, particularly at 5s, is consistent with the importance of adapting transition prediction to the evolving VLA representations.

Together, these results demonstrate progressive policy improvement across training iterations, identify the driving capabilities that benefit from imagined interactions, and highlight the importance of jointly updating the training-time components during policy optimization.


W2. Comparison with Simulator-Based Closed-Loop Training

We compare InteractVLA with CLEAR (arXiv 2607.02841), which performs closed-loop RL training directly in parallel CARLA simulators, considering both Bench2Drive performance and training efficiency. InteractVLA achieves 4.33 points higher DS and 6.77 percentage points higher SR than CLEAR. For a consistent comparison, the training time and GPU-hours reported below refer specifically to the RL optimization stage. Our Stage III GRPO optimization takes approximately 3 days (2,304 A100 GPU-hours), compared with CLEAR's reported 6 days (5,760 aggregate GPU-hours) for simulator-based RL training.

Our approach generates policy-induced experiences through compact transition tokens without repeatedly executing CARLA during RL optimization. Although differences in backbones and GPU architectures prevent a strictly controlled comparison, these results support the competitiveness of imagined-interaction training in both performance and RL optimization efficiency (Appendix D.3, Table 10).

MethodTraining ApproachGPUsTimeGPU-hoursDS ↑SR (%) ↑
CLEARSimulator-based RL32 V100 + 8 H100~6 days5,76086.8069.50
InteractVLAImagined-interaction RL (Stage III)32 A100~3 days2,30491.1376.27
AuthorsOpenReview ↗

Response to Reviewer 4Bbx (2/2)

W3. Reward Model Reliability and Generalization

We would like to further clarify our reward-training design. To improve reward estimation beyond expert demonstrations, we employ perturbation-aware supervision with four trajectory perturbations: speed reduction, speed increase, yaw deviation, and unnecessary stops. These perturbations expose the Reward Model to diverse suboptimal driving behaviors, improving its ability to estimate trajectory quality beyond expert demonstrations. We evaluate reward prediction on NAVSIM v1 navtest split, comparing our model against a demonstration-only Reward Model (Appendix D.6, Table 12):

PerturbationGT PDMSDemo-only RMOurs
Speed Reduction0.9190.9320.921
Speed Increase0.8620.9410.856
Yaw Deviation0.7160.9340.723
Unnecessary Stop0.6270.9310.637

While the demonstration-only model predicts nearly uniform high rewards, our model closely matches ground-truth scores across all four perturbation categories, demonstrating improved discrimination of suboptimal driving behaviors on unseen scenes.

We further evaluate whether the Reward Model can reliably distinguish candidate trajectory quality on NAVSIM v1 navtest, using 32 candidate trajectories per scene. Our model achieves a Spearman rank correlation of 0.570 and a pairwise ranking accuracy of 78.46%, compared with 0.551 and 77.11% reported by WAC [1], respectively. Although the methods use independently generated candidate sets, these results demonstrate competitive ranking consistency with simulator-computed trajectory scores.

Together with the feedback ablation in Table 5, these results support the accuracy, ranking reliability, and generalization of our Reward Model for imagined-interaction policy optimization.

References

[1] Zhang et al. World Action Critic (WAC) Model: Joint Foresight and Criticism for Policy Learning. arXiv:2610.22825, 2026.

Thank you for the response and the additional experimental results. However, my concerns remain unresolved.

  1. The authors have added results showing how performance changes across different scenarios. What is more important now is whether the performance of each module improved accordingly. This was the reason for my original Question 3: whether the judge improved. The paper can show how the performance of both the world model and the judge improves over the course of training.

  2. The authors added a comparison with simulator-based closed-loop training, but this does not resolve my concern. What we want to know is how the same policy performs after direct closed-loop training in a simulator versus closed-loop training through imagined interaction. This comparison is needed to demonstrate the benefits of the proposed method.

  3. Regarding the judge’s performance, a Spearman rank correlation of 0.570 is relatively low; a correlation of around 0.8 is generally considered good. In their response to W3, the authors also say that the judge’s predictions closely match the ground-truth PDMS scores, which I find puzzling. My understanding is that the judge is intended to evaluate driving trajectories after surrounding agents react, whereas the ground-truth PDMS here does not account for those reactions. If the two are close, could that indicate that the judge is inaccurate?

Reviewer_ZWQGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors proposed InteractVLA, a framework for self-improving autonomous driving through imagined interaction. It uses a learned transition-token world model to generate multi-step imagined” experience and scores it. GRPO is also used to improve. It reports 93.2 PDMS on NAVSIM v1 and 91.13 DS on Bench2Drive.

Critical strengths
  1. The idea of the pipeline loop is motivated.

  2. The presentation is clear.

Critical weaknesses

Weaknesses:

  1. In the experiments, including the NAVSIM v2 and Navhard results would be more supportive. In addition, PDMS-5s, it seems the second decision is conditioned on an observation generated by the method’s own transition model, and in Stage III, the model is trained on these imagined states. Which world model generates DynVLA’s second observation? If it is InteractVLA’s, the comparison favors the method, and if it is DynVLA’s own, the two numbers are not comparable.

  2. Usually, for the Bench2Drive experiments, we need to run several times with different seeds to provide variance of those runs. And also only the DS is the best among the baselines.

  3. Is there any way to measure the quality of the imagined states? How about directly using the ground truth instead to see the performance change? This could provide some insights about the quality of future.

  4. In the title and abstract, the "self-improvement" is emphasized a lot, but Stage III is a single round of GRPO with a fixed interaction horizon. There is no experiment with repeated rounds, and the continuation evaluation covers only two decisions. In my opinion, the experimental settings only support “multi-step imagined RL fine-tuning” rather than self-improvement.

Minors:

  • Some confusion notations: A^\hat{A} denotes both action tokens and advantages
Questions for authors

Please see the Weaknesses section. The main concerns are the experiments, the self-improvement claim.

4 comments
AuthorsOpenReview ↗

Response to Reviewer ZWQG (1/2)

We sincerely thank the reviewer for the constructive feedback and for recognizing the motivation of our imagined-interaction framework and the clarity of our presentation. We address the concerns below.

W1. Fairness of Comparison and Generalization to Challenging Benchmarks

Fairness of the 5-Second Comparison. We thank the reviewer for raising this important concern. We clarify that DynVLA does not use InteractVLA's world model to generate subsequent observations. The original DynVLA autoregressively predicts transition tokens before action tokens, treating future transitions as chain-of-thought reasoning. To enable action-conditioned future-state prediction, we retrain DynVLA to generate transition tokens after action tokens while preserving its autoregressive architecture. Both methods use the same transition-token representation and independently generate their own action-conditioned future states.

To further ensure a fair and comprehensive comparison, we evaluate both architectures under both training pipelines (Appendix D.7, Table 13):

ArchitectureTraining Pipeline4s PDMS ↑5s PDMS ↑
DynVLA (adapted)DynVLA91.779.9
Asymmetric Dual TransformerDynVLA91.580.3
DynVLA (adapted)InteractVLA91.982.8
Asymmetric Dual TransformerInteractVLA93.288.3

Under DynVLA's training scheme, the two architectures achieve comparable performance (79.9 vs. 80.3 in 5s PDMS). With our imagined-interaction training pipeline, both improve, but our Asymmetric Dual Transformer achieves a substantially larger gain (+8.0 vs. +2.9 PDMS). These results provide a more comprehensive comparison and highlight the complementary contributions of our architecture and training strategy to continued planning.

Additional Evaluation on NAVSIM v2 and NAVHARD. We further evaluate InteractVLA on NAVSIM v2 navtest and NAVHARD using EPDMS, comparing against representative VLA-based and world-model-based methods (Appendix D.1, Table 8).

MethodVenueNAVSIM v2 ↑NAVHARD ↑
ReCogDriveICLR 202683.625.7
DriveVLA-W0ICLR 202686.924.4
CAR-VLAarXiv 202690.335.0
SimWAMarXiv 202690.237.6
PhysWAMarXiv 202690.439.8
SUVarXiv 202691.036.9
VITALarXiv 202691.138.6
InteractVLA (Ours)ICLR 2027 Submission91.741.2

InteractVLA achieves 91.7 EPDMS on NAVSIM v2 and 41.2 EPDMS on NAVHARD, outperforming all listed baselines on both benchmarks. These results further demonstrate the effectiveness of imagined-interaction training under more challenging planning settings.

Together, these comparisons support the fairness of our DynVLA evaluation and demonstrate the effectiveness of InteractVLA beyond the original NAVSIM v1 setting.


W2. Multi-Seed Evaluation and Overall Performance on Bench2Drive

We thank the reviewer for the valuable comments. We address both the evaluation variance and the overall performance of InteractVLA.

Multi-Seed Evaluation. We conduct three independent Bench2Drive evaluations using three different CARLA random seeds. The results below report the mean and standard deviation of Driving Score (DS) and Success Rate (SR), alongside previously reported multi-seed baselines (Sec 4.2, Table 2):

MethodVenueDS (Mean ± Std) ↑SR (Mean ± Std) ↑
SimLingoCVPR 202585.07 ± 0.9567.27 ± 2.11
BeVAD-MCVPR Findings 202688.11 ± 0.9872.73 ± 1.98
BLUEEMNLP 202690.58 ± 0.1276.18 ± 0.64
InteractVLA (Ours)ICLR 2027 Submission91.13 ± 0.1476.27 ± 0.67

InteractVLA achieves the highest mean DS (91.13 ± 0.14) and SR (76.27 ± 0.67) among these multi-seed baselines, with low variance across evaluation runs.

Overall Performance. DS is the principal aggregate driving-performance metric on Bench2Drive, where InteractVLA achieves the highest reported score of 91.13. Moreover, our method achieves the second-highest SR (76.27%) among all compared methods. Compared with LinkVLA, which achieves a similar DS (91.01), InteractVLA obtains a higher SR (76.27% vs. 74.55%). These results demonstrate strong closed-loop driving performance, with leading DS, competitive SR, and consistent results across multiple evaluation seeds.

AuthorsOpenReview ↗

Response to Reviewer ZWQG (2/2)

W3. Quality of Imagined States and Ground-Truth Comparison

We conduct additional experiments to evaluate the prediction fidelity and planning effectiveness of imagined states.

Future-state prediction quality. Under logged-action conditions, our world model predicts transition tokens and decodes them into next-frame images, achieving a PSNR of 23.14 dB against ground-truth observations. For reference, PWM [1] and DriveDreamer-Policy [2] report PSNR values of 21.57 dB and 21.05 dB, respectively, on NAVSIM future-image generation.

Ground-truth replacement. To directly assess the impact of prediction errors on continued planning, we conduct an action-matched oracle evaluation, where the first executed action is fixed to the logged action to ensure consistency with ground-truth future observations (Sec 4.4, Table 6).

Second-step State5s PDMS ↑
GT Visual Tokens (Oracle)89.7
GT Transition Tokens + Decoder89.1
Predicted Transitions (Ours)88.6
Randomized Transition Outputs41.4

Predicted transitions achieve 88.6 PDMS, only 0.5 points below GT transition tokens, suggesting that imagined states preserve most task-relevant information for subsequent planning. Randomizing transition outputs reduces PDMS to 41.4, further highlighting the importance of meaningful state transitions. Together, these results demonstrate both the prediction fidelity and downstream planning utility of imagined states.

References

[1] Zhao et al. From Forecasting to Planning: Policy World Model for Collaborative State-Action Prediction. NeurIPS 2025.

[2] Zhou et al. DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning. arXiv 2026.


W4. Clarification of Self-Improvement and Long-Horizon Evaluation

We clarify that Stage III is not a single round of GRPO fine-tuning on fixed imagined experiences. Instead, at each GRPO iteration, the updated VLA policy generates fresh action-conditioned imagined interactions, which are evaluated and used for subsequent policy optimization (Algorithm 1). To directly demonstrate progressive self-improvement, we evaluate intermediate policy checkpoints throughout Stage III training (Fig. 5). The 4s PDMS consistently increases from 86.4 to 93.2 (+6.8), while the 5s PDMS improves from 79.5 to 88.3 (+8.8). These results provide direct evidence of continuous performance improvement through iterative learning from imagined experiences. Furthermore, Fig. 4 shows that increasing the training interaction horizon from one to four steps improves PDMS from 90.8 to 93.4, supporting the benefits of multi-step interaction.

Regarding long-horizon evaluation, extending imagined rollouts under NAVSIM v1's non-reactive logged traffic would increasingly compromise evaluation reliability due to accumulated prediction errors and action–state inconsistencies. We therefore use the 5s experiment to specifically assess continued planning from imagined observations. For broader evaluation, InteractVLA achieves 91.7 EPDMS on NAVSIM v2, 41.2 EPDMS on NAVHARD, and 91.13 DS on Bench2Drive's reactive closed-loop benchmark. These complementary evaluations support the effectiveness of our approach across different planning settings.


Minor. Notation

Thank you for pointing out the notation ambiguity. We have revised the notation throughout the manuscript to consistently distinguish action tokens A^t,k\hat{A} _{t,k} from advantage estimates Advt,k\mathrm{Adv} _{t,k}, as clarified in Appendix C.3 and Algorithm 1.

AuthorsOpenReview ↗

Follow-up on Rebuttal

Thank you again for your constructive feedback. We have provided additional experiments and revised the manuscript to address your concerns, including controlled DynVLA comparisons, NAVSIM v2/NAVHARD evaluations, multi-seed Bench2Drive results, ground-truth replacement experiments, and progressive self-improvement analysis.

We would greatly appreciate it if you could revisit our responses and let us know whether your main concerns have been addressed. If any questions remain, we would be happy to provide further clarification.

Thank you for your time and consideration!

Dear Reviewer ZWQG,

We would like to kindly follow up on our rebuttal and revised manuscript. As the discussion period is approaching its end, we would greatly appreciate it if you could let us know whether our additional experiments and clarifications have addressed your main concerns.

Thank you again for your time and consideration!

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely thank the reviewers for their constructive feedback and recognition of InteractVLA. We have revised the manuscript and conducted additional experiments addressing self-improvement, reliability, comparison fairness, and generalization. We summarize the main points below.


1. Research Perspective.

InteractVLA turns world-model predictions into policy-induced training experience, rather than using them only for decision-time reasoning or trajectory selection. Existing world-model-augmented VLA approaches primarily use imagined futures to improve individual decisions, without explicitly using the resulting interactions to update the policy.

Our central perspective is that a self-improving driving policy should learn from the states and consequences induced by its own decisions, rather than relying solely on fixed expert demonstrations. InteractVLA realizes this by closing the loop of action, consequence, feedback, and policy update. At each GRPO iteration, the updated policy generates fresh multi-step imagined interactions, learns from their consequences through interaction-aware feedback and return-to-go optimization, and uses its improved capabilities to generate new experiences in subsequent iterations. This enables progressive improvement through learning from policy-induced experiences.

To make this process practical, our Asymmetric Action-Transition Dual Transformer enables efficient imagined interaction through compact transition tokens. The interaction-learning process operates during training, while all world-modeling and feedback components are removed at inference.


2. Additional Experimental Evidence.

We appreciate ZWQG's recognition of the motivation and presentation, 4Bbx's positive assessment of closed-loop training and experimental quality, and Zc8e's recognition of compact transition modeling, adaptive feedback, and inference efficiency.

Following the reviewers' suggestions, we added:

  • Self-improvement analysis (ZWQG, 4Bbx, Zc8e): Stage III learning curves (Fig. 5), five-skill Bench2Drive analysis (Table 9), and module-update ablations (Table 7).
  • Reliability analysis (ZWQG, 4Bbx, Zc8e): Ground-truth replacement and prediction fidelity (Table 6), reward estimation and trajectory ranking (Table 12), and rollout error accumulation (Table 14).
  • Fairness and generalization (ZWQG, Zc8e): Controlled DynVLA architecture/training comparisons (Table 13), NAVSIM v2/NAVHARD evaluation (Table 8), and three-seed Bench2Drive results (Table 2).
  • Efficiency and horizon trade-offs (4Bbx, Zc8e): Simulator-based RL comparison (Table 10), compact transition ablation (Table 11), and rollout diversity versus error analysis (Table 14).

3. Key Findings.

  • Progressive self-improvement. During Stage III, 4-s PDMS increases from 86.4 to 93.2 (+6.8) and 5-s PDMS from 79.5 to 88.3 (+8.8). All five Bench2Drive driving skills improve, especially overtaking (+15.55) and give-way (+10.00), supporting learning from policy-induced experiences.
  • Useful and reliable imagined feedback. Predicted transitions achieve 88.6 PDMS, only 0.5 below ground-truth transition tokens in an action-matched evaluation. On unseen NAVSIM scenes, the Reward Model achieves 0.570 Spearman correlation and 78.46% pairwise ranking accuracy. Feedback and horizon ablations further characterize the benefits and limits of imagined rollouts.
  • Strong performance with practical efficiency. InteractVLA achieves 91.7 EPDMS on NAVSIM v2 and 41.2 on NAVHARD, exceeding the compared baselines. On Bench2Drive, it reaches 91.13 ± 0.14 DS and 76.27 ± 0.67% SR over three seeds. Imagined-interaction RL is competitive with simulator-based RL in reported training cost, without extra world-modeling overhead at inference.

4. Other Revisions.

We clarified the scope of self-improvement, evaluation protocols, DynVLA comparison, Reward Model supervision, and rollout limitations, and updated baseline results from recent papers.

We thank the Area Chair and reviewers for their consideration and welcome further discussion.

Sincerely,

The Authors

AuthorsOpenReview ↗

Follow-up on Rebuttal and Invitation for Further Discussion

Dear Area Chair and Reviewers,

We sincerely thank you again for your valuable feedback and constructive suggestions. We have carefully addressed the reviewers' concerns in our detailed responses and revised the manuscript with additional experiments and analyses.

As the discussion period is approaching its end, we would greatly appreciate it if the reviewers could revisit our responses and share any remaining concerns or questions. We would be happy to provide further clarifications and engage in additional discussion.

Thank you very much for your time and consideration.

Sincerely,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.