acceptodds
Withdrawn from ICLR 2027

PROACTIVEDRIVE: FUTURE-AWARE ADAPTIVE THINKING FOR VISION-LANGUAGE-ACTION DRIVING

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Anticipating near-future scene evolution has shown strong potential for autonomous driving by enabling policies to foresee how their surroundings may change before acting. However, bringing this capability into vision-language-action (VLA) driving remains challenging, as future prediction and language reasoning can both increase inference cost in closed-loop deployment. We present ProActiveDrive, an adaptive-thinking framework for efficient VLA autonomous driving. Built on a SimLingo-style language-command policy, ProActiveDrive combines lightweight future-aware representation learning with selective reasoning at inference time. Instead of relying on dense generative rollouts, the model uses a compact feature-level future predictor to provide anticipatory context to the driving policy. At the same time, adaptive thinking avoids unnecessary language generation by activating explicit reasoning only when it is useful. On Bench2Drive-220, ProActiveDrive improves over the SimLingo language-command baseline, increasing the driving score from 86.08 to 88.20 and the success rate from 67.27% to 71.30%. Ablation studies show that adaptive reasoning improves inference efficiency and that compact near-future prediction is more effective than heavier multi-horizon prediction in closed-loop driving. Evaluation on Fail2Drive further highlights the remaining challenge of out-of-distribution generalization. Our results suggest that combining efficient reasoning with lightweight future-aware prediction can improve both efficiency and decision quality in embodied VLA systems.

Primary area:
applications to robotics, autonomy, planning
Keywords:
world models, autonomous driving, vision language action models, vision language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PbHuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ProActiveDrive, a vision-language-action (VLA) autonomous driving framework built upon SimLingo that integrates a LiDAR-BEV encoder, a DeltaTok-based single-step latent future predictor, and heuristic-supervised fast/slow adaptive reasoning to improve closed-loop driving performance on Bench2Drive-220 and Fail2Drive.

Critical strengths
  • The paper addresses a practical challenge in deploying VLA policies in closed-loop autonomous driving by attempting to balance test-time deliberation with inference throughput.
  • The data-enhancement pipeline in Appendix D identifies and corrects semantic misalignments between rule-based commentary templates and expert trajectories (such as misclassifying yielding as car-following), which is a useful engineering insight for VLA dataset curation.
Critical weaknesses
  • Unfair Modality Comparison and Missing Component Ablations: The main claim that ProActiveDrive outperforms its closest architectural baseline SimLingo (88.20 vs. 86.07 DS in Table 1) is confounded by sensor modality differences, as SimLingo is strictly vision-only (1x Camera) whereas ProActiveDrive adds a LiDAR-BEV encoder alongside the DeltaTok predictor and 170K relabeled CoT annotations. There is no step-by-step ablation isolating the individual gains of adding LiDAR-BEV, adding DeltaTok future tokens, and relabeling the reasoning dataset.
  • Contradictory Empirical Results Between Main Text and Appendix: In Appendix D.8, the authors report that training with the template-based commentary already achieves 87.52 DS, and the first round of grounded CoT annotation reaches 88.93 DS and 73.2% SR—which actually surpasses the final reported ProActiveDrive model in Table 1 (88.20 DS and 71.30% SR). This indicates that the heavy two-stage fusion and second round of annotation may degrade performance and that the primary gains stem from offline data cleaning rather than the proposed architecture.
  • Conflated Ablation of Adaptive Thinking vs. Data Relabeling: Table 5 compares "Full CoT" against "Adaptive thinking" to claim a +3.1 mean ability gain from selective reasoning, but the text states that "Full CoT" uses unrefined template-based supervision while "Adaptive thinking" uses the Qwen3-VL-2B grounded supervision. This conflates the effect of selective fast/slow inference with the effect of fixing corrupted supervision labels; furthermore, adaptive thinking degrades performance in Merging (-2.1), Emergency Braking (-1.7), and Traffic Sign handling (-2.8).
  • Unsubstantiated Efficiency Claims and Hidden Visual Backbone Overhead: While claiming lightweight anticipation, the DeltaTok module requires running an additional frozen DINOv3 ViT-B/16 encoder on 512x512 images plus a 6-layer transformer predictor at every timestep on top of the main InternViT encoder. Table 4 only reports an opaque "Mean Ratio" without wall-clock latency (ms/frame), FPS, or isolated overhead of the DINOv3/DeltaTok and LiDAR branches, and in slow mode (critical scenarios like emergency braking), the agent still suffers from full autoregressive token generation latency when low reaction time is most critical.
Questions for authors
  1. What is the closed-loop Driving Score and Success Rate of ProActiveDrive on Bench2Drive-220 when trained without the LiDAR-BEV branch (i.e., strictly 1x Camera like SimLingo)?
  2. Please provide a clean, decoupled ablation table starting from the SimLingo baseline and sequentially adding: (a) LiDAR-BEV fusion, (b) DeltaTok single-step prediction, (c) grounded CoT relabeling (always-on), and (d) fast/slow adaptive switching.
  3. Why does the first-round grounded annotation model in Appendix D.8 (88.93 DS, 73.2% SR) outperform the final full ProActiveDrive model in Table 1 (88.20 DS, 71.30% SR)?
  4. What is the exact wall-clock inference latency (in ms) and GPU memory footprint of each module (InternViT, LiDAR-BEV encoder, DINOv3 + DeltaTok predictor, and Qwen2.5-0.5B under both fast and slow modes)?
Reviewer_mb99OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ProActiveDrive, a VLA driving model built on SimLingo. It adds LiDAR fusion and near-future features predicted with DeltaTok, and uses fast/slow output modes to selectively generate language reasoning. The authors also improve reasoning annotations using simulator information. The method is evaluated on Bench2Drive-220 and Fail2Drive, with additional experiments on reasoning supervision, prediction horizons, and inference efficiency.

Critical strengths
  1. The paper studies a useful problem: how to use reasoning and future information without making VLA driving too expensive. Evaluating on both Bench2Drive and Fail2Drive provides a useful setting to examine closed-loop performance and generalization. The reported overall driving performance is competitive.

  2. I also appreciate the analysis of mismatches between language annotations and expert actions, such as describing a yielding action as car following. This identifies a concrete problem in language–action supervision, and correcting such errors could be useful beyond this particular model.

Critical weaknesses
  1. I am not convinced that the contribution is sufficiently distinguished from closely related work. AutoVLA already studies fast/slow reasoning for driving, while DeepSight combines future latent prediction with adaptive CoT. The paper cites these works, but does not clearly explain what new capability or finding its particular design provides beyond them. This makes the significance of the proposed combination difficult to assess.

  2. The experiments do not convincingly establish the benefit of the main mechanisms. Table 5 changes both annotation quality and reasoning frequency, while the comparison with SimLingo also introduces LiDAR. A closed-loop comparison without future prediction is missing.

  3. More importantly, the paper provides no visualized examples showing when the model chooses fast or slow mode, what reasoning it generates, and how this relates to its action. There are also no statistics on mode usage or missed critical situations. These gaps leave it unclear whether adaptive thinking allocates computation usefully, or whether most gains come from better supervision and additional inputs.

Questions for authors
  1. With the same sensors, grounded annotations, and training budget, how do always-fast, always-CoT, and adaptive reasoning compare?

  2. How often does the model choose slow mode, and in which situations? Could the authors provide representative driving sequences showing the selected mode, generated reasoning, and resulting action, including cases where the mode selection fails?

  3. What exactly does “Mean Ratio” measure, and which parts of the pipeline are included? Please clarify why adaptive thinking is faster even than SimLingo without CoT.

  4. Are the SimLingo baseline numbers from a new reproduction? The original paper reports DS/SR of 85.07/67.27 for target-point conditioning and 86.08/65.78 for language-command conditioning, whereas this submission uses 86.07/67.27. Please clarify the source and navigation setting.

Reviewer_YrbUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

ProActiveDrive extends a SimLingo-style vision-language-action driving policy with LiDAR-BEV fusion, lightweight future-feature prediction, and adaptive language reasoning. A DeltaTok-based predictor provides near-future features, while fast/slow supervision teaches the policy to predict waypoints directly or first generate a reasoning trace. The method also improves reasoning annotations using privileged simulator information during offline annotation.

On Bench2Drive-220, the method reports a Driving Score of 88.20 and a Success Rate of 71.30%, compared with 86.07 and 67.27% for SimLingo. Fail2Drive results show a modest generalization improvement. Ablations compare adaptive reasoning with full CoT and single-step with multi-horizon prediction.

Critical strengths
  1. The paper addresses a relevant trade-off in closed-loop VLA driving. Anticipatory information can support interactive driving decisions, while generating language reasoning at every timestep can add unnecessary computation. Combining compact future representations with selective reasoning is a practically motivated direction.

  2. The evaluation includes closed-loop driving and distribution-shift testing. The improvements on Bench2Drive and Fail2Drive provide encouraging evidence for the complete system. The annotation analysis also identifies a concrete problem—language explanations that contradict expert driving intent—and describes a grounded correction procedure while preserving expert waypoint targets.

Critical weaknesses
  1. The experiments do not sufficiently isolate the central contributions. Relative to the camera-only SimLingo baseline, the proposed system introduces LiDAR, future-feature prediction, improved reasoning annotations, and additional training. The adaptive-thinking comparison also changes the reasoning supervision, making it difficult to separate selective computation from annotation quality. Similarly, showing that single-step prediction outperforms multi-horizon prediction does not establish that future prediction improves over an otherwise identical policy without prediction. Controlled comparisons holding sensors, annotations, and training budget fixed are important for attributing the reported improvements to the proposed mechanisms.

  2. The efficiency evidence is insufficiently specified for assessing the deployment claim. Table 4 reports a “Mean Ratio” and a 2.96× relative speedup over full CoT, but the metric and timing boundaries are not clearly defined. Absolute policy latency, tail latency, and the frequency of slow-mode activation are not reported. It is therefore difficult to determine the practical benefit after including feature extraction, future prediction, fusion, and language generation. The gains are also not uniform across driving abilities: compared with SimLingo, Emergency Braking decreases from 81.67% to 80.00%, and Traffic Sign handling from 77.20% to 67.70%. These regressions warrant discussion before interpreting the aggregate gains as broadly improved driving reliability.

Questions for authors
  1. With LiDAR inputs, annotation quality, and training budget held fixed, how much improvement comes separately from future prediction and adaptive reasoning? In particular, can you compare against a no-future-prediction variant and compare direct, always-on, and adaptive reasoning using the same grounded supervision?

  2. How is “Mean Ratio” defined, and which operations are included in its measurement? Please report absolute mean and tail policy latency, slow-mode activation frequency, and the evaluation repetition protocol underlying the reported standard deviations. Can you also clarify whether the Traffic Sign regression is consistent across repeated evaluations?

Comments on the paper

1 comment
Area_Chair_jthLOpenReview ↗

Reminder: Author Response and Reviewer Discussion

Dear Reviewers,

We are halfway through the discussion period. Please review the authors’ responses, address any remaining concerns, and engage in the discussion where appropriate. If your assessment remains unchanged, please briefly acknowledge that you have considered the authors’ responses.

Thank you for your time and contributions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.