- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes FocusDrive, a structured multimodal reasoning framework for autonomous-driving planning. Instead of generating only a textual rationale before a trajectory, the model generates: (i) a description of a decision-relevant object; (ii) a reference to an image patch corresponding to that object; (iii) an action plan; and (iv) eight future waypoints. The patch-reference interface is explicitly adapted from PaDT’s dynamic visual-reference mechanism: current-image patch prototypes are added as autoregressive output choices and subsequently used as input embeddings when selected. The planning model is based on Qwen3-VL-2B and is trained through visual-reference warm-up, planning SFT, and PDMS-rewarded GRPO.
Critical strengths
- A coherent and practically meaningful integration of spatial references into driving reasoning
The central design is intuitive: a generated focus object is explicitly linked to a current-image patch before the system generates a driving response. The implementation is technically clean because the same image-conditioned prototype functions as both an output candidate and a subsequent input embedding. This is more structured than merely appending text coordinates, and the Text-2D/reference-ID/patch-prototype ablations are directionally consistent with the intended interpretation.
- Strong empirical breadth and unusually detailed implementation reporting
The paper evaluates both dense driver-attention prediction and end-to-end planning, reports component-level NAVSIM metrics, includes matched internal baselines at both SFT and RL stages, and documents prompts and optimization settings. FocusDrive leads most reported W3DA gaze metrics, while the RL planner reaches 91.1 PDMS/EPDMS with deterministic single-trajectory decoding rather than test-time reranking.
Critical weaknesses
- The experiments do not isolate the causal contribution of visual patch-reference reasoning
The headline comparison against text CoT conflates several changes: additional decision-relevant object descriptions; spatial focus labels on 13.1% of Navtrain; a new dynamic patch-reference vocabulary; separate object-reference warm-up data and training; image-conditioned selected-region embeddings; and, for the final model, outcome-based RL. Thus, the 1.3/1.4-point SFT improvement over text CoT does not establish that patch references themselves improve planning. Table 5 is more diagnostic: Text-2D already improves over text CoT; replacing coordinates with patch references yields only 0.4 points without warm-up, while the full warmed reference model gains 0.6 points over warmed Text-2D. The reported image-conditioned prototype advantage over the learned-ID variant is 0.3 points. These are potentially meaningful, but no seeds, variance, confidence interval, or paired-scene significance test is provided.
- Annotation bias, baseline comparability, and statistical robustness are insufficiently controlled
The focus labels are produced after candidate screening using risk cues and Qwen3.7-Plus with the front image, candidate boxes, and recorded future ego path. Two annotators then revise/select a single patch. The resulting set is highly concentrated: 84.43% of focus objects are vehicles and 89.2% of focus-annotated examples involve deceleration; no inter-annotator agreement, independent duplicate-label rate, rejection rate, or ambiguity analysis is reported. Using future ego motion for training-label construction is not Navtest leakage as described, and the policy does not receive it at inference. Nevertheless, it is privileged offline supervision correlated with future behavior. Since the text-CoT baseline does not receive an equivalently rich focus-label channel, some gain may come from the annotation pipeline rather than the patch-reference representation. The strong deceleration skew is especially relevant because the reported RL model improves collision/TTC-related components while EPDMS progress falls relative to the SFT model. This could represent a safety–progress tradeoff or conservative bias rather than uniformly better planning. External comparisons should also be interpreted cautiously. The paper itself notes that some earlier EPDMS results may not align with the updated NAVSIM implementation and that its results use human-penalty filtering plus an official aggregation correction. Consequently, the external leaderboard tables are supportive context, not clean evidence of superiority. The internal matched comparisons should carry the main burden, but those comparisons currently lack repeated runs and uncertainty analysis.
- novelty and causal attribution.
The paper’s high-level premise that decision-relevant visual focus should guide autonomous-driving action is substantially preceded by AEGIS (CHI 2025) [1], which uses human-attention supervision to align policy attention with task-relevant objects and directly maps the resulting attended representation to vehicle control. Meanwhile, FocusDrive’s defining patch-reference interface, including decoding an image-dependent patch token and feeding its prototype back into subsequent reasoning, is adopted from PaDT (ICLR 2026) [2], while FSDrive[3], LLada[4], CoT-VLA[5], AutoVLA[6], and DriveTeach-VLA[7] already cover complementary parts of visual driving reasoning, attention-object explanation, and spatial policy guidance. The remaining contribution therefore appears to be a useful driving-specific composition of these ideas rather than a substantially new reasoning mechanism. Moreover, the current ablations conflate focus annotation, sequence restructuring, reference warm-up, spatial indexing, and image-conditioned feedback; the isolated gain of patch prototypes over learned reference IDs is only 0.3 PDMS/EPDMS. Random/wrong/shuffled-focus and no-feedback controls are needed to establish that the selected visual evidence causally drives the planning improvement rather than functioning as auxiliary supervision or a vehicle/deceleration shortcut.
[1] AEGIS: Human Attention-based Explainable Guidance for Intelligent Vehicle Systems. CHI 2025. [2] Patch-as-Decodable-Token: Towards Unified Multi-Modal Vision Tasks in MLLMs (PaDT). ICLR 2026. [3] FutureSightDrive: Thinking Visually with Spatio-Temporal CoT for Autonomous Driving (FSDrive). NeurIPS 2025. [4] Where, What, Why: Towards Explainable Driver Attention Prediction (LLada). ICCV 2025. [5] CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models. 2025. [6] AutoVLA: A Vision-Language-Action Model for End-to-End Autonomous Driving with Adaptive Reasoning and Reinforcement Fine-Tuning. NeurIPS 2025. [7] Teaching Vision-Language-Action Models What to See and Where to Look (DriveTeach-VLA). ECCV 2026.
Questions for authors
-
Repeated-run robustness: Can the authors report at least three seeds, or paired bootstrap confidence intervals over Navtest, for text CoT, Text-2D, reference-ID, and patch-reference models? Are the 0.3–0.6-point component gains statistically distinguishable from training/evaluation variance?
-
Matched-supervision control: What happens if the text-CoT baseline receives the same focus-object descriptions and the same object-reference warm-up examples, but patch tokens are removed or masked? This would determine whether the gain is attributable to explicit patch references rather than extra semantic supervision or training.
-
Focus intervention: How does planning change when the generated/reference patch is replaced at inference with a random patch, a nearby patch, or a plausible wrong-object patch while preserving all text? If performance is unchanged, the claim that selected visual content guides subsequent reasoning would be weakened substantially.
-
Annotation reliability and privileged information: How many scenes were independently labeled by both annotators, what was agreement on object/category/patch, and how often were VLM candidates rejected or revised? Please also provide an ablation in which candidate construction does not use the recorded future ego path.
-
Peripheral vision and global scene awareness: How does FocusDrive preserve and utilize peripheral visual information when reasoning is explicitly anchored to a selected image patch? In driving, safety-critical cues may arise outside the primary focus region, such as pedestrians entering from the image periphery or vehicles approaching from adjacent lanes. Does the patch-reference mechanism risk over-concentrating attention on a single object while overlooking other relevant hazards? Please clarify whether global visual context remains accessible during subsequent reasoning, and provide ablations or failure-case analyses involving multiple simultaneous hazards, peripheral objects, and unexpected events outside the selected focus region.