- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes MindDrive, an end-to-end autonomous driving framework that combines a Future-aware Trajectory Generator (FaTG) powered by a Transformer-Mamba World Action Model (WAM) for ego-conditioned BEV scene rollouts with a VLM-oriented Evaluator (VLoE) based on Tiny-LLaVA-1B to score and select trajectory candidates, evaluated on the NAVSIM-v1 and NAVSIM-v2 benchmarks.
Critical strengths
- Coupling ego-conditioned world model rollouts with multi-objective candidate evaluation addresses a practical trade-off between trajectory generation quality and selection accuracy in end-to-end planning.
- The empirical evaluation covers both NAVSIM-v1 and the newer NAVSIM-v2 benchmarks (including the safety-critical Navsafe and two-stage Navhard splits), demonstrating consistent improvements in aggregate PDMS and EPDMS metrics.
Critical weaknesses
- Unaddressed Computational Overhead of 256-Way BEV Rollout and 1B VLM Inference: According to Section 3.2.1 and Appendix A.2, WAM constructs a separate BEV feature map for each of the N=256 trajectory anchors (forming a 4D tensor of shape N by H by W by C) and rolls out a Transformer-Mamba-Transformer network across multiple future steps, followed by running a 1B-parameter VLM (Tiny-LLaVA-1B) for candidate scoring. Despite this heavy pipeline, the paper provides zero analysis of inference latency (ms per frame), memory footprint, or FLOPs, making it unclear whether the framework is practical for real-time closed-loop driving.
- Disconnect Between Claimed Cognitive Reasoning and Actual Scorer Distillation: While the paper claims VLoE performs interpretable semantic reasoning, the VLM does not generate any textual explanations or chain-of-thought deliberation. Instead, it processes a static template prompt, extracts the hidden state of a single score token, and uses an MLP head to regress ground-truth simulator scores precomputed on the 256 fixed anchors (Appendix A.2). This essentially uses a 1B VLM as a black-box feature extractor to distill the rule-based NAVSIM scorer (similar to Hydra-MDP and WoTE), and introduces a mismatch between scoring fixed anchors and evaluating the dynamically offset trajectories produced by FaTG.
- Complete Omission of Training Objectives and Loss Formulations: Neither the main paper nor the appendix provides the mathematical definitions or weighting schemes for the training objectives, including the simulator BEV supervision loss for WAM, the trajectory offset regression loss for FaTG, and the multi-metric scoring loss for VLoE. In addition, Equation 13 compresses all N=256 candidates into K metric queries and Equation 15 extracts a single score token hidden state, leaving it unclear how per-candidate multi-metric scores are decoded and matched to the 256 candidates.
- Sensor Modality Contradictions on Navhard and Marginal Gains on Navtest: MindDrive is designed as a Camera-LiDAR fusion model, yet Table 3 reports Stage II results on Navhard where synthetic future observations are strictly camera-only (as noted in Appendix A.3), without explaining how the missing LiDAR input is handled in Table 3 or why the sensor setup differs from Table 8 and Table 9. Moreover, on NAVSIM-v1 Navtest (Table 1), adding the 1B VLM and 256-way world model rollout yields only a 0.6 PDMS gain over WoTE and DIVER while slightly trailing them on No-at-fault Collision (98.4 vs. 98.5) and Comfort (99.9 vs. 100.0).
Questions for authors
- What is the exact per-frame inference latency (in ms), peak GPU memory usage, and parameter count of MindDrive broken down by Perception, WAM (for N=256 scene variants), and VLoE (Tiny-LLaVA-1B), and how does its runtime compare to WoTE and DiffusionDrive?
- In Equation 4, does WAM process all N=256 full-resolution BEV feature maps independently through the Transformer and Mamba blocks, and how does VLoE map a single score token hidden state (Equation 15) to multi-objective scores for 256 distinct trajectory candidates?
- Could the authors provide the explicit mathematical equations and loss weights for all training objectives in Stage 1 and Stage 2 (Appendix A.2), and clarify how VLoE accounts for the trajectory offsets added in Equation 9 when its supervision is precomputed on the 256 fixed anchors?
- How is the Camera-LiDAR MindDrive model evaluated in Table 3 where Stage II of Navhard provides only camera-based synthetic observations without LiDAR point clouds, and what is the exact sensor configuration difference between Table 3 and Table 8?