acceptodds
Withdrawn from ICLR 2027

GEOVLDRIVE: GEOMETRY-GROUNDED VISION-LANGUAGE REASONING FOR AUTONOMOUS DRIVING

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Autonomous driving involves continuous interaction between the ego vehicle and surrounding agents in a dynamic 3D environment, requiring geometrically grounded reasoning about current spatial relationships and future scene evolution for safe and efficient planning. Despite advances in geometric grounding and future prediction, reasoning about the implications of evolving 3D spatial relationships for ego planning remains a fundamental challenge. We introduce GeoVLDrive, a geometry-grounded vision-language-action (VLA) framework for 3D spatiotemporal reasoning and planning. Multi-layer geometric features from a 3D foundation model complement the semantic representations of a visionlanguage backbone. A causal latent chain of thought (CoT) reasons from future scene geometry through key-agent motion to meta action. An action expert grounded in geometric reasoning jointly generates and scores trajectory proposals conditioned on these latent states and multi-depth scene features. We further introduce NAVSIM 3DQA to support training and evaluation of metric 3D spatial understanding and agent motion reasoning. GeoVLDrive outperforms the compared supervised VLA baselines on NAVSIM v1 with 91.8 PDMS, while attaining 90.9 EPDMS on NAVSIM v2. On NAVSIM 3DQA, it achieves an aggregate score of 89.2, outperforming a matched fine-tuned baseline by 4.6 points. It also achieves 68.6 on LingoQA.

Primary area:
applications to robotics, autonomy, planning
Keywords:
vision-language-action model, autonomous driving, latent chain-of-thought, 3D geometric grounding, spatio-temporal reasoning, trajectory planning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_19UVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces GeoVLDrive, which aims to address the lack of effective integration between geometric understanding, future dynamics prediction, and trajectory planning within Vision-Language-Action (VLA) models for autonomous driving. The core workflow involves enhancing Vision-Language Models (VLMs) with 3D geometric features, establishing a causal latent reasoning chain that links the geometry of future scenes, the motion of key traffic participants, and driving intentions, and subsequently generating and scoring trajectory candidates via an action expert.

Critical strengths
  1. The architectural design is comprehensive. There are clear technical connections between geometric augmentation, latent state supervision, trajectory generation, and candidate scoring.

  2. NAVSIM 3DQA is of significant value. The dataset contains approximately 1.28 million question-answer samples covering categories, metric-based locations, vehicle dimensions, and motion behaviors. It addresses the limitations of traditional spatial relationship evaluations, which often lack coverage of absolute metric geometry and temporal information.

Critical weaknesses
  1. Insufficient core innovation. The proposed method primarily combines geometric feature injection, future state prediction, latent Chain-of-Thought, and trajectory scoring—approaches that overlap significantly with existing works such as VGGDrive, CoWorld-VLA, and GeoWorldAD. The independent innovative value of its core causal reasoning mechanism has not been sufficiently demonstrated.
  2. Lack of thorough validation for key mechanisms. Ablation studies focus mainly on the roles of various latent states; however, they lack direct comparisons regarding causal attention masks and reasoning sequences, failing to prove that a structured chain of reasoning is superior to standard multi-task supervision.
  3. Unclear sources of performance gains in planning. Improvements in planning performance could stem from additional supervision—such as future geometry or agent motion—or from the trajectory scoring module; the contributions of individual components have not been adequately isolated. Furthermore, certain safety metrics have not improved in tandem with the overall performance score.
  4. Limitations in the benchmark. The NAVSIM 3DQA benchmark relies on annotated projection boxes to specify targets, failing to evaluate full-scale object localization capabilities. Additionally, there is a severe scarcity of samples for critical motion categories like "cut-in" and "cut-out" maneuvers, and the focus on a single "key agent" is insufficient to capture complex multi-vehicle interactions.
  5. Insufficient comparative analysis and evidence for real-world deployment. The study lacks direct comparisons with several relevant methods, as well as evaluations of inference latency, computational overhead, and closed-loop interaction. Consequently, there is insufficient evidence to substantiate the method's generalizability and practical utility in complex driving scenarios.
Questions for authors
  1. How much improvement does structured causal attention provide compared to a model that uses no causal masking but is otherwise identical in terms of parameter count and supervision?

  2. What components are missing from related methods like DriveWorld-VLA? Are the sensor inputs, training data, and NAVSIM protocol consistent across the different models being compared?

  3. Your approach is intended to focus specifically on enhancing safety; however, the experimental results show that efficiency is gained at the expense of safety to achieve an overall performance boost—is this a reasonable outcome?

Reviewer_4AvLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes GeoVLDrive, a geometry-grounded VLA framework for autonomous driving. It integrates 3D geometric features into a VLM and introduces a causal latent CoT that connects future scene geometry, key-agent motion, and meta-actions. An action expert then generates and scores multiple trajectory proposals. The paper also introduces NAVSIM 3DQA to evaluate 3D spatial understanding and agent motion reasoning. Experiments on NAVSIM v1/v2, NAVSIM 3DQA, and LingoQA demonstrate improvements in planning and spatial understanding.

Critical strengths
  1. The proposed reasoning structure is clear, and the overall architecture is well organized. The progression from future geometry to agent motion and meta-actions provides an intuitive framework for connecting spatial understanding with planning.

  2. The experimental evaluation is relatively comprehensive, covering both planning performance and spatial understanding. The introduction of NAVSIM 3DQA also provides a useful benchmark for evaluating metric 3D understanding and motion reasoning.

Critical weaknesses
  1. The contribution of the causal latent CoT is not sufficiently demonstrated. The structured attention mask defines the information flow from future geometry to agent motion and meta-actions, but it remains unclear whether the model actually relies on this reasoning process for planning. The observed improvements may also come from the additional supervision rather than the proposed reasoning structure itself.

  2. The methodological novelty and intermediate predictions need stronger validation. Latent reasoning supervision, future dynamics modeling, and multi-proposal planning have been explored in prior work, and the specific contribution of their combination is not sufficiently clear.

Questions for authors
  1. Although future depth and agent motion are explicitly supervised, the paper does not provide sufficient quantitative evaluation of these intermediate predictions.
  2. Could the authors clarify the main technical differences between GeoVLDrive and existing methods involving latent reasoning, future dynamics modeling, and multi-proposal trajectory planning?
Reviewer_9DTGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

GeoVLDrive integrates StreamVGGT features into a VLM and connects future geometry, key-agent motion, and driving intent through supervised latents for trajectory generation and scoring. It evaluates planning on NAVSIM v1/v2 and introduces NAVSIM 3DQA for spatial and motion understanding.

Critical strengths
  1. Geometry integration improves spatial understanding. Against Qwen3-VL-2B trained with the same data and schedule, NAVSIM 3DQA improves from 84.6 to 89.2, position accuracy within 1 m from 66.8% to 76.7%, and motion accuracy from 85.8% to 90.9% (Table 3).
  2. Scoring ablations show practical benefits. The full scoring mechanism improves PDMS from 90.2 to 91.8 over pooling plus an MLP; cross-proposal score attention contributes the increase from 90.8 to 91.8 (Table 6).
Critical weaknesses
  1. The specific reasoning order is insufficiently justified. The geometry → motion → meta-action chain is intuitive, but the action expert can access scene context directly, so decisions need not pass through these latents. The paper should clarify whether its main contribution is structured representation learning, prediction-assisted planning, or faithful decision explanation.
Questions for authors
  1. Are planning gains robust, and what are the safety trade-offs? Against GeoWorldAD, EPDMS improves by 0.5 and EP by 2.3, while NC and TTC decrease by 0.3 and 0.4 (Table 2). Table 4 shows a similar trade-off. Could you provide confidence intervals using paired scene comparisons or grouping by log, plus results on critical interaction scenarios?
  2. Efficiency and deployment feasibility remain unverified. Avoiding textual CoT decoding does not establish an end-to-end speedup given the additional geometry encoder and four trajectory refinement stages.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.