acceptodds
Under review as a conference paper at ICLR 2027

CausalDrive: Breaking Motion Inertia in VLA-based Autonomous Driving by Aligned Semantic Intervention

Abstract

As End-to-End autonomous driving advances into the Vision-Language-Action (VLA) era, models increasingly rely on natural language to generate human-aligned control. Despite this progress, a critical vulnerability remains: inertial driving, defined as the tendency to maintain historical motion patterns even when facing sudden hazards. We identify three factors underlying this issue: memory bottlenecks in temporal fusion, inertial bias in implicit temporal modeling, and decoupling of reasoning and action, which together allow misleading temporal inertia to dominate decision-making. To address this issue, we introduce CausalDrive, a novel VLA framework that explicitly bridges semantic reasoning and temporal action generation through language-guided causal intervention. Concretely, our architecture features three core innovations: 1) Streaming Sparse Temporal Fusion (SSTF) for maintaining efficient, long-term context via lightweight memory slots; 2) Task-Query-Guided Temporal Modeling to generate kinematically consistent ego-trajectories; and 3) Causal-Intervention Attention (CIA), a mechanism that dynamically overrides historical momentum when triggered by safety-critical language cues (e.g., “ignore momentum”). We further optimize CausalDrive through supervised fine-tuning and constrained Group Relative Policy Optimization (GRPO) guided by learned Reward/Cost Models. Extensive experiments demonstrate its effectiveness, achieving 87.41% accuracy on DriveAction, 0.30m trajectory error on DriveLM, and 0.0142 steering MAE on BDDA. Qualitative results further show its ability to suppress inertial bias during critical events, paving the way for highly reliable and interpretable autonomous driving systems.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action Models, Autonomous Driving, Causal Intervention, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_b42aOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

CausalDrive addresses “motion inertia” in VLA-based autonomous driving, where a model recognizes a hazard but continues to follow historical motion patterns. The framework combines sparse historical memory through SSTF, Ego Query-based temporal modeling, and language-guided Causal-Intervention Attention (CIA) to adjust actions by suppressing historical features and emphasizing current hazard cues. It is trained using supervised fine-tuning (SFT) and constrained GRPO. The paper reports action classification, trajectory prediction, and vehicle control results on DriveAction, DriveLM, and BDDA.

Critical strengths

The problem is practically relevant. The failure to promptly adjust actions despite recognizing hazards warrants investigation, and directly incorporating language signals into action generation is well motivated.

  1. The approach is intuitive. Selectively downweighting historical information based on the current level of risk is well suited to sudden hazards, compared with prioritizing temporal continuity alone.
  2. The evaluation covers multiple output levels. The reported trajectory error of 0.30 m on DriveLM and steering MAE of 0.0142 on BDDA provide preliminary evidence of effectiveness.
Critical weaknesses
  1. The core contributions are insufficiently differentiated from existing techniques. SSTF primarily consists of a FIFO cache of frozen features, while Ego Query attends to historical queries; DriveTransformer already employs sparse task queries and streaming history propagation. CIA appears closer to language-conditioned key/value gating, and the paper does not establish its substantive advantages over standard gating, history resets, or attention bias. The use of the term “causal” also lacks support from controlled intervention experiments.
  2. The lack of decisive component ablations prevents attribution of the reported gains. The paper does not individually remove SSTF, Ego Query, or CIA, nor does it compare SFT-only training against GRPO using the same backbone. The modality ablations in Table 1 cannot distinguish whether improvements arise from a stronger backbone, additional training, reinforcement learning, or the proposed inertia suppression mechanism.
  3. The experiments do not directly validate the central safety claims. Aggregate accuracy, L2 error, and MAE do not establish that the model overrides motion inertia more quickly in emergencies. The paper lacks evaluation on a dedicated hazard-event subset, braking latency, collision rate, TTC, and closed-loop performance. Figure 3 presents only a scene sequence, without baseline trajectories, speed profiles, or comparisons with the intervention enabled versus disabled; it therefore cannot substantiate the claim that the model performs “causal reasoning.” Likewise, keeping the learned cost below a threshold does not directly establish real-world safety.
  4. The performance gains are limited, and the comparison settings lack transparency. On DriveAction, the reported accuracy of 87.41% is lower than GPT-4o’s 88.84% and Qwen-Max’s 91.32% in the same table. On DriveLM, the improvement over EMMA is only 0.02 m, with no variance estimates or statistical significance analysis reported. On BDDA, the method is compared against only a single FCN-LSTM baseline, whose results are presented as ranges without a clearly specified, consistent reproduction protocol. The data splits, prediction horizons, and units of the control outputs are also insufficiently specified.
Questions for authors
  1. With the same backbone, data, and training budget, how much additional benefit does CIA provide over standard gating, hazard-triggered history resets, and attention enhancement applied only to the current frame?
  2. Could you hold the current observation fixed while varying historical motion states in sudden-hazard scenarios, and report braking latency, collision rate, and false braking rate in normal scenarios, to verify whether the gains can be attributed to suppressing misleading motion inertia?
  3. Please clarify the DriveAction input protocol, the DriveLM data splits and prediction horizon, and the source of the BDDA labels. Please also explain the relationship between the 0.05 m error reported in the appendix and the 0.30 m error in the main results table.
Reviewer_Y5fhOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

CausalDrive aims to reduce reliance on historical motion when current observations indicate danger. It combines a frozen Qwen3-VL backbone with SSTF memory, Ego Query temporal attention, and semantic-conditioned Causal-Intervention Attention (CIA), trained through SFT and constrained GRPO. Evaluation covers offline action classification, trajectory prediction, and continuous control on DriveAction, DriveLM, and BDDA.

Critical strengths

The mismatch between hazard recognition and timely action is a relevant driving problem. Using current semantics to regulate historical information is a plausible research direction, although its specific benefit remains insufficiently validated here.

Critical weaknesses
  1. The claimed technical contribution is not established.

Section 3.1 describes a FIFO feature cache, temporal query attention, and semantic-conditioned K/V modulation, but does not establish how these operations provide the claimed semantic preservation, kinematic consistency, or causal intervention. FIFO temporal queries and sparse memory already appear in DriveTransformer and OmniReason. The introduction also describes reasoning-based methods, including Alpamayo-R1, as structurally decoupling reasoning from action. However, Alpamayo-R1's action expert reads a VLM KV cache containing reasoning (Sec. 5.1). This does not guarantee faithful reasoning, but it contradicts the stated structural distinction. The contribution beyond existing temporal memory and reasoning-conditioned policies therefore remains unclear.

  1. The experiments do not validate the central behavioral claim.

The paper does not establish that a baseline recognizes danger but continues because of misleading history, then demonstrate that CIA corrects this failure. No controlled core-module ablation isolates the proposed mechanism from the combined architecture and GRPO; modality comparisons and training curves cannot provide this attribution. DriveAction compares general-purpose VLMs, while BDDA uses only FCN-LSTM. Although the trajectory table includes driving models, it does not resolve the contribution relative to closely related temporal or reasoning-conditioned policies. Offline Accuracy/L2/MAE cannot establish improved emergency responses, and no closed-loop or equivalent targeted behavioral test is reported. Figure 3 shows scene snapshots without baseline responses, predicted trajectories, or intervention measurements. The evidence thus supports offline prediction performance, not the stronger claim of breaking motion inertia through causal intervention.

Questions for authors
  1. How does CIA determine what to intervene on?

How does it distinguish misleading historical motion from still-valid motion information? Are the relevant tokens or masks predefined or learned, and how do hazard semantics determine the location and strength of suppression?

  1. Are the main experimental results directly comparable?

Were baselines rerun or taken from prior work? Please clarify the relationship between Table 2's DriveLM evaluation and published nuScenes results, and the source of Table 3's FCN-LSTM error ranges. Does each comparison use consistent data splits, input conditions, and metric definitions?

Reviewer_J1WXOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The author proposed three mechanisms to counter the "inertial driving" failure mode in VLA-based autonomous driving, including Streaming Sparse Temporal Fusion, Task-Query-Guided Temporal Modeling, and Causal-Intervention Attention. After two-stage training (SFT with GRPO), proposed approach was evaluated on three open-loop benchmarks.

Critical strengths

A relevant, well-motivated problem based on VLA-based autonomous driving for "inertial driving"; Efficient temporal design coupled with CIA is an intuitive and interesting combination

Critical weaknesses

Factors identified for "inertial driving" behavior may just be intuitively plausible but are presented without empirical evidence, therefore ideas stemmed from them may not tackle the key variables leading to "inertial driving";

Selection of the three benchmarks seems random without a clear rationale explained;

Benchmarking presented in the paper lacks closed-loop metrics or ablation study.

Questions for authors

Can you help to supplement more empirical evidence about the factors identified for "inertial driving" as well as provide a more systematic benchmarking results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.