- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents InteractVLA, a learning framework to enable self‑improving vision‑language‑action (VLA) driving models via imagined interaction. The core components include an Asymmetric Action‑Transition Dual Transformer that produces actions and compact scene‑change transition tokens in one forward pass, interaction‑aware adaptive feedback fusion for drifted imagined states, and an interaction‑specific return‑to‑go objective optimized with GRPO. Transition‑related modules are discarded during inference so that no extra overhead is introduced at deployment. Evaluations are conducted on NAVSIM and Bench2Drive. InteractVLA achieves strong standard planning performance.
Critical strengths
-
By converting imagined multi‑step rollouts into usable training signals rather than only using them for single‑step reasoning, this work provides a promising direction for model‑based self‑supervised improvement in autonomous driving.
-
The proposed Asymmetric Action‑Transition Dual Transformer leverages compact transition tokens to represent scene dynamics instead of reconstructing full pixel‑level future observations, which significantly cuts the computation cost of imagined rollouts.
-
The interaction‑aware feedback mechanism adaptively mixes replay‑based ground‑truth evaluation and learned reward‑model estimates according to visual correspondence.
-
The framework separates training‑time imagined‑interaction machinery from inference pipelines. Ablations demonstrate favorable latency compared to competing world‑model‑enhanced baselines, making it potentially applicable for real‑world driving VLA development.
Critical weaknesses
-
After Stage III the transition pathway, decoder, and Reward Model are discarded. Inference keeps only the vision encoder, VLA, and action de-tokenizer. There is no test-time imagination, no online update, and no multi-round improvement curve. That is one-shot model-based policy optimization, not self-improving autonomous driving.
-
The Reward Model is trained with replay derived NC/TTC while the action transition model stays fixed. Fusion (Eq. 7) trusts the RM when visual cosine similarity is low, the regime in which replay was declared invalid. DAC, EP, and Comfort always remain replay. Stage III still supervises transition and reward on logged data. GRPO can exploit world-model and RM error; that is unmeasured.
-
Table 1 reports a PDMS of 93.6 for ChainFlow-VLA, compared with 93.2 for InteractVLA. This relatively small margin alone does not appear sufficient to support a broad state-of-the-art claim, particularly when several reported experimental results do not consistently establish superiority over competing methods.
-
The InteractVLA results reported in Table 1 appear inconsistent with the corresponding results reported in its original work, raising concerns about the reliability of this baseline comparison.
-
Although adaptive feedback alleviates evaluation drift, the paper does not systematically quantify how transition‑model prediction errors accumulate over longer imagined rollouts. It also lacks controlled ablations on how rollout length trades off between training‑data diversity and simulation‑error contamination.
-
Despite being a VLA‑driving work, all evaluations focus on general trajectory‑planning tasks. The paper does not test whether imagined interaction improves language‑instructed driving performance, which is a natural use case for VLA systems.
Questions for authors
-
Does the method actually support self-improving driving at inference time, or does it only produce a better policy through offline training?
-
Can the policy exploit errors in the learned transition model or Reward Model, and how is this risk controlled?
-
Are the reported baseline results and SOTA claims sufficiently reliable and reproducible?
-
Does the proposed imagination mechanism improve the language-conditioned driving capabilities that motivate the use of a VLA in the first place?
4 comments
Response to Reviewer Zc8e (2/2)
W3 / Q3. Comparison with ChainFlow-VLA and SOTA claims
We would like to highlight an important difference in the comparison with ChainFlow-VLA. ChainFlow-VLA uses four camera views and a learned trajectory scorer to select among multiple refined candidates, whereas InteractVLA achieves 93.2 PDMS using only a single front-view camera, without additional trajectory scoring at inference time. Despite this more restrictive configuration, InteractVLA achieves the highest PDMS among the single-front-view methods in Table 1, only 0.4 points below ChainFlow-VLA (93.6).
To further assess performance on more challenging benchmarks, we evaluate InteractVLA on NAVSIM v2 (navtest) and NAVHARD, comparing against recent VLA-based and world-model-based methods, as detailed in Table 8 of Appendix D.1 in the revised manuscript. InteractVLA achieves 91.7 EPDMS on NAVSIM v2 and 41.2 EPDMS on NAVHARD, outperforming all compared methods on both benchmarks. These results further demonstrate the effectiveness of InteractVLA under more challenging planning settings.
Moreover, InteractVLA achieves a 91.13 Driving Score and 76.27% Success Rate on Bench2Drive, demonstrating strong closed-loop driving performance. Overall, these results highlight the competitiveness of InteractVLA across different benchmarks, with minimal visual input and no additional inference-time trajectory scoring.
W4. Possible inconsistency in baseline results
We have carefully re-examined the baseline results in Table 1 against their original publications and updated the reported results of DriveVA and SimWAM according to the latest versions of their respective papers. These corrections have been incorporated into Table 1 of the revised manuscript to ensure that our comparisons accurately reflect the most recent reported results.
W5. Error accumulation and interaction-horizon analysis
To quantify error accumulation, we compare GT-Conditioned Prediction (GT) and Self-Conditioned Rollout (SC) using identical ground-truth actions in the Table below (Appendix D.8, Table 14 (a)). GT predicts each future observation from its corresponding ground-truth observation, whereas SC recursively conditions on its own predicted observations. GT maintains approximately 23 dB, while SC decreases to 18.39 dB, quantifying prediction-error accumulation during recursive imagination.
| Prediction Depth | GT PSNR ↑ | SC PSNR ↑ |
|---|---|---|
| 1 | 23.14 | 23.14 |
| 2 | 23.03 | 22.57 |
| 3 | 22.76 | 21.02 |
| 4 | 23.40 | 18.39 |
We further evaluate training-data diversity using the group-wise average pairwise L2 distance between sampled trajectories over a fixed time window, ensuring that differences are not simply caused by longer trajectory durations (Appendix D.8, Table 14 (b)). As the interaction horizon increases, trajectory diversity rises substantially, while planning improvements gradually saturate. Extending from three to four steps yields only 0.2 additional PDMS points, despite increasing training time from approximately 3 to 10 days.
| Training Interaction Steps | Trajectory Diversity (L2, m) ↑ | PDMS ↑ |
|---|---|---|
| 1 | 0.36 | 90.8 |
| 2 | 1.26 | 92.4 |
| 3 | 2.31 | 93.2 |
| 4 | 4.58 | 93.4 |
Together, these results quantify the trade-off between richer imagined experiences and accumulating prediction errors, motivating our three-step interaction horizon. We further mitigate these risks through logged-state initialization, horizon curriculum, and interaction-aware feedback.
W6 / Q4. Language-conditioned driving capabilities
We clarify that language-based navigation instructions are provided as conditioning inputs to InteractVLA in both NAVSIM and Bench2Drive evaluations. Our method focuses on improving driving decisions through imagined interactions under language-conditioned instructions, rather than enhancing language understanding in isolation. To further demonstrate this improvement, we compare the policy before and after imagined-interaction optimization across five Bench2Drive driving skills (Table 9, Appendix D.2). The results show that imagined-interaction optimization improves all five driving skills, increasing the mean success rate from 67.73% to 74.45%, with particularly notable gains in overtaking and give-way scenarios. These results demonstrate that imagined interactions effectively improve driving performance under language-conditioned navigation instructions. We have clarified the role of language conditioning and the scope of our evaluation in the revised manuscript.
Follow-up on Rebuttal
Thank you again for your detailed and insightful feedback. We have carefully addressed your concerns and substantially strengthened the revised manuscript with additional evidence on progressive self-improvement, reward and transition reliability, rollout error accumulation, baseline comparisons, and language-conditioned driving performance.
We would greatly appreciate it if you could revisit our rebuttal and revised manuscript and let us know whether your main concerns have been resolved. We would welcome any further questions or suggestions during the remaining discussion period.
Thank you very much for your time and consideration!
Dear Reviewer Zc8e,
We would like to gently follow up on our previous responses. We would be grateful if you could revisit our revised manuscript and share whether any major concerns remain. We would be happy to provide further clarification before the discussion period ends.
Thank you very much for your time and consideration!
Response to Reviewer Zc8e (1/2)
We thank the reviewer for the detailed and technically insightful feedback. We particularly appreciate the positive assessment of our imagined-interaction framework, compact transition representations, adaptive feedback mechanism, and inference efficiency. We address the concerns below and clarify the scope of the proposed method.
Definition of Self-Improving Autonomous Driving. We define self-improvement as a policy improving its capabilities by learning from self-generated experiences rather than relying solely on fixed expert demonstrations, consistent with prior work [1, 2]. Our framework focuses on training-time self-improvement, rather than test-time imagination or online adaptation. Specifically, at each GRPO iteration, the updated policy generates fresh action-conditioned imagined interactions, learns from their consequences, and uses the improved policy to generate new experiences for subsequent optimization (Algorithm 1). This establishes an iterative interaction-learning process rather than one-shot optimization over fixed imagined experiences.
Evidence of Progressive Improvement. We provide additional experimental evidence in Figs. 4 and 5 of the revised manuscript. Fig. 4 shows that increasing the training interaction horizon from one to four steps improves PDMS from 90.8 to 93.4, demonstrating the benefit of multi-step imagined experiences. More importantly, Fig. 5 directly tracks policy performance throughout Stage III optimization: 4-s PDMS progressively increases from 86.4 to 93.2 (+6.8), while 5-s PDMS improves from 79.5 to 88.3 (+8.8). These results provide direct evidence that the policy progressively improves through iterative learning from imagined interactions.
References
[1] Chen et al. Self-Play Fine-Tuning Converts Weak Language Models to Strong Language Models. ICML 2024.
[2] Pang et al. Language Model Self-improvement by Reinforcement Learning Contemplation. ICLR 2024.
We agree that errors in learned transition and reward models may be exploited during policy optimization. We mitigate this risk through perturbation-aware reward learning, adaptive feedback, and additional reliability evaluations.
Reward Model Reliability. To improve reward estimation beyond expert demonstrations, we employ perturbation-aware supervision with four trajectory perturbations: speed reduction, speed increase, yaw deviation, and unnecessary stops. On the unseen NAVSIM v1 navtest split, the demonstration-only model predicts nearly uniform high rewards, whereas our model closely matches ground-truth PDMS across different perturbations (Appendix D.6, Table 12):
We further evaluate candidate ranking reliability using 32 trajectories per scene on NAVSIM v1 navtest. Our Reward Model achieves a Spearman correlation of 0.570 and a pairwise ranking accuracy of 78.46%, compared with 0.551 and 77.11% reported by WAC [1], respectively. These results provide additional evidence that our reward estimates can distinguish trajectory quality on unseen scenes.
Adaptive Feedback. Eq. 7 adaptively combines replay-based and learned NC/TTC estimates according to visual correspondence, reducing reliance on replay evaluation as imagined states diverge from logged observations. DAC, EP, and Comfort remain replay-based. As shown in Tables 3 and 5, our adaptive fusion achieves 88.3 PDMS, outperforming replay-only (81.4), learned-only (84.9), and fixed fusion (86.5), demonstrating the benefit of combining complementary feedback sources.
Transition Model Reliability. We additionally conduct an action-matched ground-truth replacement experiment to quantify the effect of transition prediction errors on continued planning below. Predicted transitions achieve 88.6 PDMS, only 0.5 points below GT transition tokens, indicating limited downstream degradation from transition prediction errors under the evaluated conditions (Table 6 in manuscript).
Together, these results demonstrate the reliability of our reward supervision and imagined transitions, while providing empirical evidence that perturbation-aware supervision and adaptive feedback effectively mitigate model-induced errors during imagined-interaction optimization.
References
[1] Zhang et al. World Action Critic (WAC) Model: Joint Foresight and Criticism for Policy Learning. arXiv:2610.22825, 2026.