HGWM: Hierarchical 3D Gaussian World Model with 4D Occupancy Forecasting for End-to-End Autonomous Driving
Abstract
End-to-end autonomous driving requires a world state that preserves explicit three-dimensional geometry, evolves coherently over time, and remains directly accessible to the planner. Existing driving world models commonly rely on dense grids or latent representations, making future geometry and its connection to planning less explicit. We propose the Hierarchical Gaussian World Model(HGWM), which formulates semantic 3D Gaussian primitives as an explicit recurrent world state shared by 4D occupancy forecasting and end-to-end trajectory planning. Rather than evolving all primitives uniformly, HGWM decomposes the scene into comparatively stable background and actor-centric foreground fields and models their distinct intra- and cross-field interactions through geometry-aware hierarchical communication and sparse local refinement. A residual transition operator jointly evolves Gaussian geometric, feature, opacity, and semantic attributes, while sparse expert routing accommodates heterogeneous foreground dynamics. Crucially, the same autoregressively evolved Gaussian state is rendered into future semantic occupancy and directly queried to generate multi-modal ego trajectories, coupling scene forecasting and planning through a shared geometric state. Under the reported camera-only nuScenes protocol, HGWM achieves 25.49% semantic mIoU and 49.86% binary IoU at 3s, together with an average planning L2 error of 0.28m and a 0.11% collision rate. It further reaches a PDMS of 92.1 on NAVSIM and a Driving Score of 82.6 with a 61.8% Success Rate on Bench2Drive. These results support hierarchical Gaussian evolution as an explicit shared world-state formulation for future scene prediction and trajectory planning.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.