acceptodds
Under review as a conference paper at ICLR 2027

Spatiotemporally Consistent Sparse World Model for 4D Occupancy Prediction and Planning in Autonomous Driving

Abstract

4D semantic occupancy forecasting provides a scene-level representation for anticipating future geometry and semantics in autonomous driving. Sparse occupancy world models enable efficient long-horizon forecasting by compressing a scene into compact latent queries and rolling them forward in time. However, existing sparse world models suffer from **spatiotemporally inconsistent semantic rollouts**. This problem arises because semantic information evolved in latent query space is not explicitly aligned with the rendered 4D occupancy structure formed by refined points and their temporal evolution. To address this limitation, we present **STC-World**, a spatiotemporally consistent sparse world model that explicitly aligns latent semantics with the rendered 4D occupancy structure. For spatial semantic alignment, we propose a **coordinate-conditioned semantic decoder** that incorporates normalized point coordinates and query-relative offsets into semantic decoding, producing location-specific corrections to point-level logits. For temporal semantic consistency, we introduce **adjacent-horizon temporal self-distillation** to constrain semantic evolution during future rollout. The distillation objective uses corrected semantics at each horizon as a stop-gradient teacher for the next horizon over inherited query indices, with class-aware consistency terms for static and dynamic categories. STC-World therefore maintains semantic correspondence throughout spatial refinement and temporal rollout while retaining a sparse latent representation. On Occ3D-nuScenes, STC-World achieves a state-of-the-art average mIoU of 13.47% over the 1–3 s forecasting horizons. It attains a 0.20 m average planning L2 error and reduces the average collision rate from 0.29% to 0.09%, corresponding to a 69.0% relative reduction, while running at 2.56 FPS with 391.2 ms mean latency. Project page: [https://stc-world.github.io/](https://stc-world.github.io/).

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.