acceptodds
Under review as a conference paper at ICLR 2027

GeoWAM: Visual Geometry World Action Models for Autonomous Driving

27 views

Abstract

World action models (WAMs) have recently gained increasing attention as a framework for jointly modeling scene evolution and ego actions in autonomous driving. Most existing WAMs learn scene dynamics in pixel space by combining a video-generation backbone for future-observation prediction with an action head for ego-trajectory prediction. Pixels, however, provide only an indirect representation of these dynamics: they entangle geometry and motion with appearance, texture, and illumination, forcing the model to infer three-dimensional transformations from two-dimensional observations. We argue that point-based geometry provides a more natural state space for driving. It explicitly captures spatial structure and both rigid and non-rigid scene dynamics while remaining aligned with the 3D space of driving actions. Building on this insight, we introduce GeoWAM, a visual geometry world action model for autonomous driving. Rather than predicting future images, GeoWAM is pretrained to forecast future scene geometry, yielding representations that jointly encode spatial structure and temporal evolution. A geometry-conditioned action head then leverages these learned geometric dynamics to predict future ego-trajectories. Extensive experiments show that GeoWAM outperforms image-based alternatives, achieving an EPDMS of 90.2 on NAVSIM v2 and a combined EPDMS of 36.6 on navhard without PDMS supervision. Scaling geometry pretraining raises the navhard score by 8.2% to 39.6 and enables strong zero-shot transfer to nuScenes, yielding an average L2 error of 0.89 m and a collision rate of 0.12%. Together, these results establish geometry as an effective state representation and geometry pretraining as a general, scalable strategy for downstream planning.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous Driving, Geometry Predicition, End-to-End Planning

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.50 over 2 reviews, above 99% of the others.
Reviewer_MSuKOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents GeoWAM, a geometry-centric world-action model for autonomous driving. A pretrained visual geometry encoder and a future decoder predict dense future point maps and features, which are used by a trajectory decoder. The method is evaluated on geometric forecasting, NAVSIM v2, and zero-shot nuScenes planning, with additional experiments on pretraining data scale.

Critical strengths

(+) The use of explicit metric geometry provides a clear connection between future prediction and motion planning. The architecture and its supervision are reasonably motivated, and the current-only versus future-geometry ablation directly examines an important design choice.

(+) The planning results are promising. Without additional PAI data, future geometry improves the reported navhard score from 31.7 to 36.6; the full model reaches 39.6. The zero-shot nuScenes results also show improvements in average displacement and collision metrics over the reported DVGT-2 baseline.

Critical weaknesses

(-) Explicit future-geometry prediction introduces additional computation and memory requirements, but its practical cost remains unclear. Table 7 reports parameter counts, while Table 6 demonstrates planning gains from longer prediction horizons without quantifying the corresponding inference overhead. Could the authors report end-to-end latency and peak GPU memory for the current-only baseline and different prediction horizons under the same hardware and input settings? Please also clarify whether dense point-map decoding is required during planning inference or whether the planner operates solely on future geometry tokens. These results would help establish whether the planning gains justify the additional computational cost.

(-) Could the authors explore whether the planning benefits of future-geometry modeling can be retained through training-time distillation, inspired by future-prior distillation approaches such as AMap? Specifically, a student planner could learn from the full GeoWAM teacher while omitting the future-geometry prediction branch at inference. Comparing this student with the full model in planning performance, latency, and peak memory would help clarify how much of the benefit requires explicit future prediction at deployment.

[1] AMap: Distilling Future Priors for Ahead-Aware Online HD Map Construction, CVPR 2026

Questions for authors

Q1: What are the end-to-end inference latency and peak memory at the reported input resolution, and how does performance change under a practical inference budget?

Q2: How does geometric accuracy vary between static and moving objects?

Reviewer_PCBwOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents GeoWAM, which predicts future geometric representations to support trajectory planning for autonomous driving. The method is clearly designed and achieves meaningful improvements on NAVSIM, particularly on navhard. However, the experiments do not yet sufficiently explain the sources of these gains.

Critical strengths
  1. Clear motivation and coherent design. The paper connects future geometry prediction with trajectory planning to incorporate anticipated scene structure into driving decisions.
  2. Meaningful gains on challenging scenarios. The improvements on NAVSIM, particularly navhard, demonstrate the method’s potential in demanding planning settings.
  3. Broad evaluation. Experiments cover planning performance, future geometry prediction, and transfer to nuScenes.
  4. Informative ablations. The prediction-horizon and pretraining-data experiments help characterize the framework’s behavior, although data scale and training duration are not fully separated.
Critical weaknesses
  1. The data-scaling study confounds data volume with training computation. Appendix A.3 reports that the variants using 0, 100, 496, and 832 hours of PAI data are pretrained for 40k, 40k, 70k, and 80k steps, respectively. The scaling curve therefore varies both data volume and training computation; only the 0-to-100-hour comparison keeps the number of steps fixed. The full improvement cannot be attributed solely to greater data volume or diversity. Please provide training and validation curves against optimization steps, including geometry losses and downstream planning performance at selected checkpoints, together with comparisons at matched training budgets. These results would help distinguish data-scaling benefits from additional optimization and clarify whether the variants have converged.
  2. The contribution of environmental dynamics modeling remains unclear. The results support the effectiveness of the combined future-geometry prediction and planning framework, but do not establish how much of the gain comes from modeling independently moving traffic participants rather than static scene structure, ego motion, or additional supervision. Because future point maps are expressed in each future ego frame, even a static scene changes as the ego vehicle moves. The current ablations do not isolate these sources of improvement. Please compare against matched-budget controls using current/static geometry supervision and static-scene extrapolation based on ego motion. Separate evaluations on moving objects and static regions would further clarify whether the proposed design improves environmental dynamics modeling and whether this improvement benefits planning.
  3. The distinctive contribution needs stronger behavioral and scenario-level evidence. Showing that future geometric prediction improves aggregate planning scores is useful, but does not by itself establish a sufficiently distinctive methodological contribution beyond existing geometry-supervised and future-prediction approaches. The paper should clarify what its particular design enables and support this explanation with behavior- and scenario-level analysis. For example, does it improve responses to vehicle cut-ins, lead-vehicle braking, crossing pedestrians, or intersection traffic? Comparisons with DVGT-2 and a matched geometry-supervised baseline, reporting relevant safety, progress, and comfort metrics with sample counts, would help identify where the method offers a meaningful advantage and connect that advantage to its design.
Questions for authors
  1. Prior work has improved spatial representations by adding geometric prediction objectives, including depth prediction as auxiliary supervision. GeoWAM’s proposed contribution appears to be the use of future dense point-map prediction as the primary world-modeling objective, followed by planning conditioned on the predicted geometry. Could the authors compare this design directly with a relatively lightweight depth-supervision variant under the same architecture, data, and training budget? Such a comparison would clarify whether dense future geometry provides enough additional planning benefit to justify its computational and memory costs.

  2. The geometry-pretraining variants use different numbers of optimization steps: 40k steps without PAI data, 40k with 100 hours, 70k with 496 hours, and 80k with 832 hours. Thus, the reported scaling trend does not cleanly separate the benefit of additional data from the benefit of longer training. Could the authors provide training and validation curves, checkpoint-level planning results, and matched-budget comparisons to separate PAI data scaling from longer training?

  3. How much improvement remains over current/static geometry supervision and ego-motion-based static-scene extrapolation, and how does performance differ between moving objects and static regions?

4.The aggregate planning scores show an overall improvement, but offer limited insight into when the proposed design helps. Could the authors break down results by driving behavior or scenario type, such as turns, intersections, lane changes, or interactions with moving agents? Comparisons with DVGT-2 and a geometry-supervised baseline trained under a matched setup would help identify whether the gains are specific to particular scenarios or reflect a more general improvement.

  1. Can the authors report inference time for the proposed method and the baselines under the same hardware setting, and discuss whether the inference speed meets the real-time requirements of real-vehicle testing?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.