acceptodds
Under review as a conference paper at ICLR 2027

EvoDriveVLA: Evolving Driving VLA Models via Collaborative Perception-Planning Distillation

Abstract

Vision-Language-Action models have shown great promise for autonomous driving, yet they suffer from degraded perception after unfreezing the visual encoder and struggle with accumulated instability in long-term planning. To address these challenges, we propose EvoDriveVLA—a novel collaborative perception-planning distillation framework that integrates self-anchored perceptual constraints and future-informed trajectory optimization. Specifically, self-anchored visual distillation leverages self-anchor teacher to deliver visual anchoring constraints, regularizing student representations via trajectory-guided key-region awareness. In parallel, future-informed trajectory distillation employs a future-aware oracle teacher with coarse-to-fine trajectory refinement and Monte Carlo dropout sampling to synthesize reasoning trajectories that model future evolutions, enabling the student model to internalize the future-aware insights of the teacher. EvoDriveVLA achieves SOTA performance in nuScenes open-loop evaluation and significantly enhances performance in NAVSIM closed-loop evaluation. Code is provided in the supplementary material.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous drive, VLA model, Knowledge distillation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_F951OpenReview ↗
Rating
2(−1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces EvoDriveVLA, a novel distillation framework for end-to-end autonomous driving (E2D-AD).

Specifically, the paper designs self-anchored visual distillation to impose visual anchoring constraints on trajectory-guided key regions, enhancing perceptual capabilities.

In addition, the paper proposes future-informed trajectory distillation to enable the student to internalize the oracle teacher’s future-aware insights, which also enable coarse-to-fine trajectory refinement.

However, the paper lack the comparison with existing coarse-to-fine methods and the complete ablation studies.

Critical strengths

This paper introduces EvoDriveVLA, a novel distillation framework for end-to-end autonomous driving (E2D-AD).

Specifically, the paper design self-anchored visual distillation to impose visual anchoring constraints on trajectory-guided key regions, enhancing perceptual capabilities.

In addition, the paper propose future-informed trajectory distillation to enable the student to internalize the oracle teacher’s future-aware insights, which also enable coarse-to-fine trajectory refinement.

Critical weaknesses
  1. Insufficient comparison with existing coarse-to-fine methods. One of the core contributions of this paper is coarse-to-fine trajectory refinement. What are the differences between the proposed method and existing coarse-to-fine methods [1]? The authors should provide experimental comparisons. And discuss these differences in the Introduction or Related Work section.

  2. Limited comparisons with recent methods. The authors should include more comparisons with methods [1][2][3][4] published in 2026 in Tables 1 and 2.

  3. Lack of evaluation in interactive closed-loop environments. The authors should evaluate the proposed model on closed-loop benchmarks such as Bench2Drive or HUGSIM, since NAVSIM and nuScenes are both open-loop benchmarks.

  4. Lack of comparisons with newer VLM backbones. In Table 2, the authors should adopt the latest Qwen3.5 or Qwen3.8 models as baselines to strengthen the conclusions of the paper.

  5. Insufficient ablation studies. The authors should conduct the ablation experiments in Table 3 on additional datasets, as the current results do not clearly demonstrate the effectiveness of each proposed module.

[1] WAM-Flow: Parallel Coarse-to-Fine Motion Planning via Discrete Flow Matching for Autonomous Driving

[2] Recogdrive: A reinforced cognitive framework for end-to-end autonomous driving

[3] WAM-Diff: A masked diffusion vla framework with moe and online reinforcement learning for autonomous driving

[4] Wa-jepa: Rethinking the video jepa paradigm for world-action modeling in autonomous driving

Questions for authors

Are the images in Figure 1 and Figure 2 AI-generated? Regardless, please refine the color scheme.

17 comments

Thank you for the constructive review and for recognizing the contributions of our self-anchored visual distillation and future-informed trajectory distillation. We address your concerns below.

Q1. Differences from existing coarse-to-fine methods.

WAM-Flow uses coarse-to-fine refinement as part of its inference-time planning procedure through parallel discrete flow matching. In contrast, our refinement is performed exclusively by the privileged teacher during training to construct stronger distillation targets. The student does not perform coarse-to-fine refinement at inference, and its inference procedure remains unchanged.

These mechanisms therefore serve different purposes: inference-time trajectory generation versus training-time supervision construction. A direct module-level comparison would not isolate the same functionality, although system-level benchmark comparisons remain useful. We will clarify this distinction in the Introduction and Related Work of the camera-ready version.

Q2. Comparisons with recent methods.

Thank you for bringing these recent works to our attention. We will expand the discussion and comparison to include WAM-Flow, RecogDrive, WAM-Diff, and WA-JEPA in the camera-ready version. We will incorporate their reported results into Tables 1 and 2 where the datasets, evaluation protocols, and input settings are comparable, and explicitly identify differences in experimental settings.

Q3. Evaluation in interactive closed-loop environments.

NAVSIM evaluates trajectories using non-reactive simulation and simulation-based driving metrics, going beyond trajectory-displacement evaluation alone. Nevertheless, we agree that this does not substitute for fully interactive closed-loop evaluation.

Importantly, Appendix G, Table 11 already reports additional closed-loop results on Bench2Drive:

MethodOpen-loop avg. L2 ↓Closed-loop Driving Score ↑Closed-loop Success Rate (%) ↑
Matched baseline1.0142.3216.08
EvoDriveVLA0.9744.4616.74

These results demonstrate improvements in both closed-loop Driving Score and Success Rate. We will make this evaluation more prominent and clarify the distinction between NAVSIM's non-reactive evaluation and interactive closed-loop evaluation.

Q4–Q5. Validation on a newer backbone and additional ablations.

Following your suggestions, we further evaluated our framework on Qwen3.5-4B, including separate ablations of visual and language-model distillation:

MethodL2 1s ↓L2 2s ↓L2 3s ↓Avg. L2 ↓Collision 1s ↓Collision 2s ↓Collision 3s ↓
Qwen3.5-4B0.160.471.010.540.030.080.42
Only Vision KD0.160.481.020.550.000.020.30
Only LLM KD0.160.440.940.520.050.080.38
Ours0.150.430.940.510.000.080.33

The complete framework reduces average L2 from 0.54 to 0.51 m and three-second collision rate from 0.42% to 0.33%, demonstrating effectiveness on a newer backbone. The ablations further distinguish the branches' contributions: visual distillation improves collision rates at all three horizons, while LLM distillation primarily improves trajectory accuracy. Their combination achieves the lowest average L2, although visual distillation alone performs better on some collision metrics. These experiments extend our branch-level ablations across backbones; the NAVSIM and Bench2Drive results additionally support effectiveness across benchmarks, although they do not constitute full component-wise ablations on additional datasets.

Figures 1 and 2.

Neither Figure 1 nor Figure 2 is AI-generated. Thank you for the suggestion regarding their appearance. We will refine the color scheme to improve readability and visual consistency in the camera-ready version.

We hope these clarifications and additional results address your concerns, and we would appreciate your consideration of raising your score.

AuthorsOpenReview ↗

A Sincere Request to Reconsider the Rating Change

May I sincerely ask why the rating was lowered from 3 to 2 before our rebuttal, without an accompanying explanation? This unexpected change has been deeply discouraging, particularly because we do not understand what prompted it.

We have worked hard to address your concerns and provide additional evidence. Could you please consider our rebuttal and reconsider restoring your original rating of 3? If concerns remain, we would be grateful for the opportunity to address them.

Thank you sincerely for your time and consideration.

The driving score (DS) of EvoDriveVLA on Bench2Drive makes me somewhat skeptical of this approach, even though the work achieved promising results in open-loop evaluations on NAVSIM.

Reviewer_m27KOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes EvoDriveVLA, a collaborative perception-planning distillation framework for autonomous driving Vision-Language-Action (VLA) models. Motivated by visual representation degradation during fine-tuning and instability in long-horizon trajectory prediction, the method introduces two complementary distillation strategies. Self-anchored visual distillation uses a frozen copy of the pretrained visual encoder to preserve perceptual representations, with trajectory-guided anchoring weights emphasizing driving-relevant regions. Future-informed trajectory distillation employs an oracle teacher with access to privileged future observations, combining coarse-to-fine trajectory refinement and Monte Carlo dropout sampling to generate high-quality supervision targets. The student is subsequently trained through hidden-state and logit alignment alongside the original trajectory prediction objective. Experiments on nuScenes and NAVSIM demonstrate improvements in trajectory prediction and planning performance, supported by component ablations and qualitative analyses.

Critical strengths
  1. Relevant motivation addressing both perception and planning challenges. The paper identifies two practical limitations of driving VLA training: degradation of pretrained visual representations and insufficient planning guidance from conventional teacher models. Addressing these challenges within a unified distillation framework is meaningful for improving autonomous driving performance.

  2. Intuitive integration of complementary distillation strategies. The proposed framework combines trajectory-guided visual feature preservation with future-informed trajectory supervision. AnchorFormer provides adaptive constraints on driving-relevant visual regions, while the oracle teacher employs privileged future observations, trajectory refinement, and candidate sampling to provide stronger planning supervision.

  3. Competitive results with supporting ablations. EvoDriveVLA achieves strong trajectory prediction performance on nuScenes and improves PDMS on NAVSIM across both 3B and 8B backbones. The component ablations demonstrate progressive improvements from trajectory distillation, refinement, MC-Dropout, and visual distillation, providing empirical support for the overall framework.

Critical weaknesses
  1. Limited methodological novelty of the individual components. The framework primarily combines established techniques, including frozen-teacher feature distillation, privileged-information learning, coarse-to-fine trajectory refinement, and MC-Dropout sampling. Although their integration into driving VLA training is reasonable, the paper does not sufficiently establish which aspects constitute a substantial methodological advancement beyond existing knowledge distillation approaches.

  2. Insufficient isolation of the benefits of future-informed distillation. The oracle teacher has access to future observations, and the optimal candidate is selected using ground-truth trajectories. These mechanisms provide substantially stronger supervision than conventional teacher-student settings. However, the current experiments do not clearly separate the benefits of privileged future information, ground-truth-based candidate selection, and the proposed refinement and sampling strategies. Consequently, the specific contributions of these mechanisms remain difficult to assess.

  3. Incomplete validation of perceptual preservation and planning robustness. Although AnchorFormer is designed to prevent visual representation degradation, the evidence mainly consists of downstream planning metrics and attention visualizations rather than direct perceptual evaluations. Moreover, performance improvements are not consistent across all metrics: for example, the collision rate under the UniAD protocol remains worse than DiMA despite improved trajectory accuracy. More comprehensive analyses would strengthen the claims regarding perception preservation and robust planning.

Questions for authors
  1. Could the authors further clarify the methodological novelty relative to existing privileged-information distillation and visual feature regularization approaches? In particular, which components or interactions are essential to the proposed framework beyond combining established distillation techniques?

  2. Could the authors provide a controlled experiment in which the teacher operates without privileged future observations while retaining the same coarse-to-fine refinement, MC-Dropout sampling, and ground-truth-based candidate selection? This would help isolate the contribution of future information from the benefits of the trajectory selection and distillation mechanisms.

  3. Could the authors provide more direct evidence that self-anchored visual distillation preserves pretrained perceptual capabilities, such as feature similarity measurements or downstream perception evaluations? Additionally, can the authors explain why the proposed method achieves a higher collision rate than DiMA under the UniAD evaluation protocol despite improving trajectory prediction accuracy?

8 comments

Thank you for the thoughtful review and for recognizing the motivation and effectiveness of EvoDriveVLA. We clarify our contributions and supporting evidence below.

Q1. Methodological novelty.

Our contributions lie in three task-specific designs:

  • Trajectory-conditioned visual anchoring. AnchorFormer uses trajectory, instruction, and ego-state information to redistribute representation-preservation constraints across visual tokens. Appendix E, Table 9 validates this design against uniform and MLP-based weighting, achieving lower average L2 error (0.54 vs. 0.56/0.55 m).
  • Future-informed supervision construction. The oracle teacher combines future-conditioned coarse-to-fine trajectory refinement with hidden-state sampling, then distills the selected candidate's paired hidden states and logits. This converts privileged future information into supervision that the student can learn from available observations.
  • Complementary perception–planning distillation. The framework jointly addresses visual representation degradation and trajectory supervision. Appendix F, Table 10 shows that combining both branches achieves 0.52 m/0.12% average L2/collision, outperforming trajectory guidance alone (0.53 m/0.17%) and AnchorFormer alone (0.54 m/0.15%).

Appendix H, Table 12 summarizes the conceptual distinctions from related distillation paradigms. We will highlight these distinctions more clearly in the revised manuscript.

Q2. Contribution of privileged future information.

Following your suggestion, we evaluated teacher variants without future observations while retaining ground-truth-based candidate selection where applicable. The reference baseline below uses future observations.

MethodFuture observationsAvg. L2 (m) ↓Avg. collision (%) ↓
Baseline without distillation.-0.550.19
Refinement onlyNo0.560.19
Sampling onlyNo0.540.20
Refinement + samplingNo0.550.20

The baseline uses privileged future observations, whereas the three subsequent variants do not. Their comparable performance suggests that refinement and sampling offer little benefit without future observations, consistent with a predictive bottleneck imposed by the available information. Their role in our framework is to exploit the relationship between historical observations and privileged future information, producing more informative supervision for the student.

Q3. Perceptual preservation and planning robustness.

Beyond attention visualizations, the supplementary material provides quantitative evidence under challenging visual conditions:

EvaluationMetric ↓Without visual KDWith visual KD
Rainy/night, ST-P3Avg. L2 (m)0.240.22
Rainy/night, UniADAvg. L2 (m)0.430.40

These results (Appendix B, Table 7) support improved planning under challenging visual conditions. We agree that they do not replace direct feature-similarity measurements or perception benchmarks, and will distinguish the representation-preservation objective from the downstream planning robustness demonstrated experimentally.

Regarding DiMA, EvoDriveVLA achieves better results on three of the four aggregate metrics:

MethodST-P3 avg. L2 (m) ↓ST-P3 avg. collision (%) ↓UniAD avg. L2 (m) ↓UniAD avg. collision (%) ↓
DiMA0.270.080.570.07
EvoDriveVLA0.260.060.520.12

Only the average collision rate under UniAD is worse than DiMA. Average waypoint error does not directly measure obstacle avoidance, so lower L2 does not necessarily imply fewer collisions. Nevertheless, our framework improves both UniAD metrics over its matched baseline, reducing average L2 from 0.55 to 0.52 m and average collision rate from 0.19% to 0.12%. NAVSIM and the Bench2Drive evaluation in Appendix G provide further evidence of improved planning performance.

We hope these clarifications and additional results address your concerns, and we would appreciate your consideration of raising your score.

Thanks for the additional results. For the no-future ablation, what is the result with future observations under the same refinement and sampling setup? Are these teacher or student results?

AuthorsOpenReview ↗

Complete Ablation Results and the Matched Future-Aware Comparison

Dear Reviewer m27K,

Thank you for the follow-up. All results in our no-future ablation table are evaluations of the student after distillation, not the teacher. The corresponding future-aware results are already reported in Table 3. We present the original and additional ablations together below.

ConfigurationTeacher future observationsRefinementMC-DropoutStudent visual encoderVisual KDStudent avg. L2 (m) ↓Student avg. collision (%) ↓
Baseline without distillation (Table 3)———TrainableNo0.550.19
Trajectory KD only (Table 3)YesNoNoTrainableNo0.540.17
Trajectory KD + refinement (Table 3)YesYesNoTrainableNo0.530.16
Trajectory KD + refinement + sampling (Table 3)YesYesYesTrainableNo0.530.14
Above configuration with frozen encoder (Table 3)YesYesYesFrozenNo0.560.18
Full EvoDriveVLA (Table 3)YesYesYesTrainableYes0.520.12
No-future trajectory KD + refinementNoYesNoTrainableNo0.560.19
No-future trajectory KD + samplingNoNoYesTrainableNo0.540.20
No-future trajectory KD + refinement + samplingNoYesYesTrainableNo0.550.20

The direct answer to your question is 0.53 m average L2 and 0.14% average collision rate for the future-aware teacher under the same refinement and sampling setup. Its no-future counterpart achieves 0.55 m and 0.20%, respectively.

These two configurations use the same teacher architecture, refinement and sampling procedures, GT-based candidate selection, and training budget. Both retain a trainable student visual encoder and exclude visual distillation. The difference is whether the teacher receives privileged future observations. This matched comparison therefore supports the contribution of future information beyond refinement, sampling, and GT-based selection alone.

We hope these results resolve your questions about the experimental settings and the contribution of privileged information. If your concerns have now been addressed, we would sincerely appreciate an upward revision of your score as soon as possible during the discussion period. We remain available to address any further questions.

Comments on the paper

5 comments
Area_Chair_TQyCOpenReview ↗

Reminder: Author Response and Reviewer Discussion

Dear Reviewers,

We are halfway through the discussion period. Please review the authors’ responses, address any remaining concerns, and engage in the discussion where appropriate. If your assessment remains unchanged, please briefly acknowledge that you have considered the authors’ responses.

Thank you for your time and contributions.

Thank you for the thoughtful review and for recognizing the motivation, complementary benefits, and empirical effectiveness of EvoDriveVLA. We appreciate your suggestions for better understanding the interactions between the distillation objectives and the roles of teacher refinement and sampling.

Q1. Interaction between perception and planning objectives.

Appendix F, Table 10 already compares independently applied visual and trajectory distillation with their joint application:

MethodAvg. L2 (m) ↓Avg. collision (%) ↓
Baseline0.550.19
Trajectory guidance only0.530.17
AnchorFormer only0.540.15
Joint distillation0.520.12

Joint distillation outperforms either branch alone on both metrics, supporting their complementary contributions. The visual objective is also explicitly conditioned on the planning task: AnchorFormer uses trajectory information to determine where visual representations should be preserved. During joint training, these constraints act on the same encoder that is updated through the trajectory objective, allowing visual preservation and planning adaptation to occur simultaneously.

We agree that these results do not establish gradient-level synergy or superiority over sequential distillation. We have not conducted a budget-matched sequential comparison. Running two full training stages would increase training cost, while keeping the total budget fixed would require allocating steps between stages and selecting their order. Our joint formulation avoids this additional scheduling complexity. We will clarify that “collaborative” refers to task-conditioned visual anchoring and complementary joint supervision, rather than a demonstrated absence of gradient conflicts.

Q2. Contribution of privileged future information.

Following your suggestion, we evaluated teacher variants without future observations while retaining ground-truth-based candidate selection where applicable. The reference baseline below uses future observations.

MethodFuture observationsAvg. L2 (m) ↓Avg. collision (%) ↓
Future-aware reference baselineYes0.550.19
Refinement onlyNo0.560.19
Sampling onlyNo0.540.20
Refinement + samplingNo0.550.20

The baseline uses privileged future observations, whereas the three subsequent variants do not. Their comparable performance suggests that refinement and sampling offer little benefit without future observations, consistent with a predictive bottleneck imposed by the available information. Their role in our framework is to exploit the relationship between historical observations and privileged future information, producing more informative supervision for the student.

Q3. Coarse-to-fine refinement versus additional candidate generation.

Coarse-to-fine refinement and MC-Dropout sampling perform different operations. Our MC-Dropout perturbs the output hidden states of existing trajectories and generates candidates through the LM head. Increasing the sample count therefore primarily expands the search around existing representations. In contrast, refinement feeds the coarse trajectory back into the teacher as an additional condition and performs a new conditional generation pass. It can revise the trajectory using both the privileged observations and the preceding prediction, rather than simply producing more perturbations of the same representation.

Figure 3 provides evidence consistent with these complementary roles: the left panel shows that refinement shifts the trajectory-loss distribution toward lower errors and reduces the high-error tail, while the right panel shows further improvements through MC-Dropout-based candidate selection. Refinement thus improves the representations from which candidates are generated, while sampling further explores those representations.

We hope these clarifications and additional results address your concerns, and we would appreciate your consideration of raising your score.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal and Invitation to Further Discussion

Thank you again for your time and thoughtful reviews. We have provided point-by-point responses and additional experimental results addressing your concerns.

We would appreciate it if you could review our rebuttal and share your updated assessment before the discussion period ends. If any concerns remain unresolved, please let us know as soon as possible so that we have time to respond and clarify them.

Thank you for your continued engagement.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.