acceptodds
Withdrawn from ICLR 2027

MindDrive: A Map-Informed Adaptive Decision Method for Efficient End-to-End Driving

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Recent advances in Vision-Language-Action (VLA) models have demonstrated the potential of combining multimodal understanding, reasoning, and trajectory generation for end-to-end autonomous driving. However, existing methods often infer traffic context directly from multi-view images, making them susceptible to hallucinations about planning-critical scene elements. Moreover, their reasoning strategies are typically insensitive to scene complexity, leading to insufficient deliberation in challenging scenarios and unnecessary reasoning overhead in routine ones. In this paper, we propose MindDrive, a ap-formed Adaptive ecision VLA model for efficient driving. First, we design a language-map construction and retrieval pipeline that converts structured map elements into spatially indexed natural-language annotations, providing accurate, decision-relevant scene priors.Then, during supervised fine-tuning stage, the model learns to assess scenario complexity and accordingly perform either direct action generation or deliberative reasoning. Finally, we introduce a reinforcement learning-based post-training strategy that contrasts different decision modes within the same scene, further improving planning performance and reasoning efficiency. Extensive experiments on NAVSIM and nuScenes benchmarks demonstrate the competitive performance of MindDrive.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Vision-Language-Action model, end-to-end autonomous driving, adaptive decision making, reinforcement learning post-training

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_r3UGOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces MindDrive, a framework that converts HD maps into retrievable text descriptions. Built on the Qwen2.5-VL-3B backbone, the model dynamically selects between direct trajectory generation and Chain-of-Thought reasoning according to the scenario. Planning performance is further optimized through reinforcement fine-tuning (RFT) with a dual-mode paired-sampling strategy. The experiments use the NAVSIM and nuScenes datasets. After RFT, the NAVSIM PDMS improves from 83.0 to 90.0; on nuScenes, the average L2 error decreases from 0.67 to 0.51 and the average collision rate decreases from 0.14% to 0.08%. The approach has practical engineering value, but the paper does not yet establish its incremental contribution over existing adaptive-reasoning methods or provide direct measurements supporting the claim of improved efficiency.

Critical strengths
  1. The textual map representation has clear practical value and is supported by preliminary ablations. Map descriptions can be generated offline and retrieved online with limited storage overhead. In Table 3, adding map data improves the SFT model's PDMS from 79.5 to 81.9, and adding CoT reasoning further increases it to 83.0. These results indicate that map information contributes to planning performance.

  2. The complete system shows substantial internal improvements on two benchmarks. RFT improves NAVSIM PDMS by 7.0 points and also improves trajectory error and collision metrics on nuScenes. The analysis of 548 failure scenarios provides useful information about the system's limitations.

Critical weaknesses
  1. The core framework overlaps substantially with existing work, and the remaining contributions are not independently validated. AutoVLA already combines direct action and Chain-of-Thought modes with reinforcement fine-tuning. AdaThinkDrive (2025) (https://arxiv.org/html/2509.13769v1) uses the planning quality of both modes to train adaptive reasoning. Converting map topology into text has also been explored in prior work such as PlanAgent. The authors should clarify the incremental contribution of their particular text-based map construction and forced paired-sampling mechanism. The current experiments do not compare different representations of the same map information, nor do they compare forced paired sampling against standard RFT under matched backbone, data, and training budgets. Consequently, the paper does not yet show why these specific design choices are important.

  2. The main experiments do not establish a fair comparison or substantiate the claim of efficiency. The original NAVSIM protocol (https://www.cvlibs.net/publications/Dauner2024NEURIPS_supplementary.pdf) designates HD map data as privileged information for simulation and metric computation and makes it inaccessible to the driving agent. MindDrive consumes map-based text during inference, yet Table 1 labels its input simply as Cam. Its PDMS of 90.0 therefore cannot be directly compared with baselines using standard non-privileged inputs. Results under equivalent map-input conditions are needed. In addition, the map and CoT ablations are limited to SFT and do not isolate the corresponding gains in the final RFT model. The paper reports no direct measurements of latency, throughput, or generated-token count, and it does not provide a performance-versus-compute comparison for the final model across direct-planning, reasoning, and adaptive modes. CoT trigger rates in Table 4 are insufficient to establish end-to-end efficiency. Finally, the TTC metric decreases from 93.7 to 91.8 in Table 1, so the results should not be described as an across-the-board improvement in safety.

Questions for authors
  1. Input protocol and fairness: Where do the map and global-localization data used during inference originate? Please provide baselines using the same map inputs, together with results for the final RFT model when map inputs are removed.

  2. Incremental innovation: With the backbone, data, and training budget held constant, what performance gain does forced pairwise sampling provide over standard grouped sampling? What advantages does the text-based map representation offer over structured map encodings containing the same information?

  3. Training mechanism: Are the mandated mode tokens included in the policy-gradient loss? If not, how does RFT directly optimize the model's own mode-selection process?

  4. Efficiency and trade-offs: For the final model under always-direct, always-reasoning, and adaptive settings, what are the PDMS, generated-token count, and end-to-end latency? Does the latency include map retrieval, input prefill, and trajectory refinement?

Reviewer_2wmhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

MindDrive proposes a map-informed vision-language-action framework for autonomous driving. It retrieves local natural-language road descriptions from an offline map library and adaptively selects between direct trajectory generation and explicit chain-of-thought (CoT) reasoning. The training pipeline combines teacher-derived scenario-difficulty supervision with reinforcement fine-tuning (RFT), using paired rollouts from different reasoning modes and planning-based rewards. A trajectory refiner produces the final planning output. The paper reports a PDMS of 90.0 on NAVSIM and an average collision rate of 0.08% on nuScenes.

Critical strengths
  1. Practical integration of static map knowledge. The anchor-based language-map construction and retrieval pipeline provides a concrete way to incorporate structured road information into VLA planning. The design is clearly motivated and reasonably specified. Table 3 provides supporting evidence for its utility: adding map descriptions improves PDMS from 79.5 to 81.9 under the reported SFT setting.

  2. Competitive planning results and useful supplementary material. The paper evaluates on both NAVSIM and nuScenes and reports encouraging aggregate planning performance, particularly in collision avoidance on nuScenes. The authors also acknowledge the weaker trajectory accuracy relative to several baselines. The supplied implementation and supplementary analysis of turning-related failures provide useful additional context.

Critical weaknesses
  1. The central claim of adaptive, efficient reasoning is not directly validated.

    The paper does not provide a controlled comparison of always-direct, always-CoT, and learned adaptive inference using the same trained model. It also does not report end-to-end latency, completion lengths, or reasoning-mode usage after RFT. Table 4 examines different CoT supervision ratios rather than whether the learned routing policy allocates reasoning effectively. Furthermore, the length penalty in Equation 7 applies only beyond a fixed threshold and does not distinguish reasoning costs below that threshold. Consequently, the reported planning improvements do not establish that adaptive mode selection achieves the claimed planning-quality versus efficiency trade-off. It also remains unclear whether the model selects CoT specifically in scenarios where reasoning provides a measurable planning benefit.

  2. The methodological distinction from prior adaptive-reasoning approaches and the contribution of paired-mode RFT remain insufficiently established.

    AutoVLA (NeurIPS 2025) already combines direct and CoT-based planning with GRPO-based reinforcement fine-tuning, while AdaThinkDrive (ICRA 2026) explicitly compares trajectory quality across reasoning modes to optimize adaptive reasoning and evaluates the associated performance–latency trade-off. The paper should clarify what is substantively new about its paired-mode rollout formulation relative to these approaches.

    A related concern is how the proposed RFT objective actually improves mode selection. Section 3.4 constructs paired rollouts by forcing different reasoning modes, but Equations 7–8 do not make clear whether the forced mode tokens participate in the policy-gradient update or are treated as fixed conditioning prefixes. This distinction matters because optimizing trajectory generation under a prescribed mode does not necessarily optimize the policy that selects the mode.

    Moreover, the map-conditioning and CoT ablations are reported under SFT rather than the final RFT setup. Without RFT-stage ablations and a matched comparison against standard planning-reward RFT, it is difficult to isolate the contributions of map conditioning and paired-mode training from the general benefits of reinforcement fine-tuning.

Questions for authors
  1. Adaptive reasoning and efficiency: For the same RFT-trained model under identical decoding settings, how do always-direct, always-CoT, and adaptive inference compare in PDMS, collision metrics, average completion length, and mean/P95 end-to-end latency? What fraction of scenes uses each reasoning mode after RFT? Does the learned mode selection correlate with the scene-level planning improvement obtained from CoT?

  2. Optimization of mode selection: When constructing paired rollouts by forcing direct and deliberative modes, are the mode tokens included in the policy-gradient objective or masked as conditioning prefixes? How does the proposed RFT objective specifically improve the learned mode-selection policy rather than only improving trajectory generation conditioned on a given mode?

  3. Contribution isolation: What performance remains when map conditioning or paired-mode training is removed from the final RFT setup? How does paired-mode training compare with standard PDMS-based RFT under matched training budgets and sampling configurations?

Reviewer_3yvxOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose MindDrive. MindDrive builds an offline language-map library from HD maps, retrieved by ego pose, and trains a Qwen2.5-VL-3B VLA to choose between action and CoT. It then uses RFT to contrast the two modes. It reports 90.0 PDMS on NAVSIM v1 and 0.51m on nuScenes.

Critical strengths
  1. It introduced a language-map library, which is simple yet efficient.
Critical weaknesses
  1. The PDMS 90.0 on NAVSIM v1 and 0.51 on nuScenes are not competitive. RecogDrive has 86.5 after SFT and 90.8 after RFT. I am also curious about the results of EPDMS on NAVSIM v2 or NAVHard.

  2. It claimed "efficient" in the title, but there is no evidence to support that. I expect to see some results on inference latency or some other points to prove that.

  3. The proposed method used map information to generate actions, while in Table 1, to the best of my knowledge, those VLA-based baselines don't use map information. I think the comparison is not fair, and the second column notes "Cam" only.

Questions for authors

Please see the Weaknesses section. The main concerns are the results, the efficiency experiments, and the fairness of the comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.