acceptodds
Under review as a conference paper at ICLR 2027

InfraWorld: Motion-Aware World Modeling for Roadside 4D Occupancy Forecasting

Abstract

Roadside 4D occupancy forecasting predicts future semantic occupancy in a fixed infrastructure coordinate system. Roadside scenes are globally structured yet locally evolving, requiring comprehensive scene-level context while precisely modeling sparse localized motion. However, existing forecasting paradigms struggle to tackle both requirements. To address this limitation, we propose InfraWorld, a world model formulating roadside occupancy forecasting as selective state evolution. InfraWorld maintains a dense BEV state for the complete scene and persistent motion queries for moving regions, assigning complementary states to global context and local dynamics. Specifically, we propose Dynamic Query Initiation (DQI) to initialize queries in regions supported by motion in the observed history. During rollout, the dense state is updated through Long–Short Temporal Fusion (LST), which combines short-term adjacent-frame continuity with long-term motion trends to preserve stable context while capturing scene evolution. Motion-Aware Query Injection (MAQI) then refines the queries and injects confidence-weighted query information into corresponding BEV tokens, enabling targeted dynamic updates without perturbing global context. Together, InfraWorld concentrates predictive capacity on evolving traffic regions. Extensive experiments on the InfraOcc and Occ3D–nus datasets demonstrate that InfraWorld achieves state-of-the-art performance for roadside 4D occupancy forecasting while retaining its effectiveness under ego-centric settings. Code will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
4D Occupancy Forecasting, Roadside Perception, World Model

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_ux9VOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses roadside 4D occupancy forecasting, which predicts future semantic occupancy in the coordinate frame of a fixed roadside sensor. Based on the statistic that 97.55% of occupied voxels in InfraOcc are static and only 1.19% belong to moving agents, the authors argue that the global scene state and local motion should be handled separately. InfraWorld rolls out a dense BEV latent from a VQ tokenizer autoregressively while carrying 20 persistent queries. It consists of three modules: DQI, which places queries using a moving-mask head; LST (long-short temporal fusion), which combines short-term ConvGRU fusion with a long-term trend summarized from history residuals; and MAQI, which writes confidence-weighted query information back into the BEV.

Critical strengths
  1. The idea of separating and handling features of dynamic objects, which are the main focus in 4D occupancy forecasting, is sound, and the ablation study supports it.
  2. The analysis of the voxel distribution is a useful addition.
Critical weaknesses
  • Separating static and dynamic elements has been proposed for 3D occupancy prediction in StreamOcc (ECCV 2026), and the long-short-term temporal fusion is similar to the design proposed in SOLOFusion (ICLR 2023). I am not convinced that the proposed components make a distinct contribution beyond these works, and a comparison or discussion would help.

  • Since the setting is fixed-view and the goal is to separate static from moving objects, one could detect changed regions by directly comparing BEV representation values over time, without a moving-mask loss to filter moving regions. A comparison with such a simpler alternative would be helpful.

  • Occ3D-nuScenes assumes a moving ego vehicle, which differs from the fixed-view setting. I would like to know the exact experimental setup used on this dataset.

  • A comparison of accuracy and cost with methods based on sparse representations, such as GaussianWorld (CVPR 2025), would strengthen the paper.

Questions for authors

Please see the questions raised in the Weaknesses. Answers to Weaknesses 1 and 3 (clarification) would most likely change my assessment; Weaknesses 2 and 4 are suggestions that would strengthen the paper but are not required for the rebuttal.

Reviewer_NvueOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes InfraWorld, the first framework for roadside 4D semantic occupancy forecasting in a fixed infrastructure coordinate system. The key observation is that roadside scenes are “globally structured yet locally evolving”: static background dominates while future changes concentrate in sparse moving regions. InfraWorld maintains a dense BEV latent state for the full scene and a small set of persistent motion queries for dynamic regions, with three components: (1) Dynamic Query Initiation (DQI), which uses a moving-mask head to allocate queries to motion-supported tiles with soft localization; (2) Long–Short Temporal Fusion (LST), combining a ConvGRU-style short-term branch with a residual-based long-term trend branch; and (3) Motion-Aware Query Injection (MAQI), which writes confidence-weighted query features into nearby BEV tokens via distance-biased cross-attention. Experiments on InfraOcc and Occ3D-nus show state-of-the-art results, e.g., +9.02 mIoU_dyn over I²-World-O on InfraOcc, with component-wise, hyperparameter, and efficiency ablations.

Critical strengths

S1. Well-motivated, underexplored problem. The static–dynamic asymmetry of roadside scenes (97.55% static voxels vs. ~2.5% moving/stationary participants) is convincingly quantified (Fig. 1a, Fig. 2), and the paper is, to my knowledge, the first to address 4D occupancy forecasting (as opposed to single-frame prediction) in fixed roadside coordinates. The problem statement is clear and the “selective state evolution” formulation is a natural fit.

S2. Sound and coherent architecture design. The division of labor—dense BEV state for global context, sparse persistent queries for localized dynamics—is a principled response to the identified asymmetry, and each component maps to a specific failure mode (DQI: query budget allocation; LST: adjacent-frame continuity vs. accumulated trends; MAQI: unreliable writeback during rollout). Zero-initialized projections and residual latent updates suggest careful engineering for training stability.

Critical weaknesses

W1. Limited technical novelty; components are largely adaptations of known techniques. The VQ tokenizer + autoregressive latent rollout follows OccWorld; the query refinement pipeline is explicitly borrowed from SparseWorld (Sec. 3.2); the short-term branch is a standard ConvGRU; confidence weighting resembles quality/reliability estimation in detection. The main novelty lies in the composition and the new problem setting, and the paper does not clearly demonstrate which design choices would not transfer from ego-centric methods. A more discriminating analysis (e.g., what specifically breaks when SparseWorld or DFIT-OccWorld is applied to roadside data) would substantially strengthen the contribution claim.

W2. Heavy reliance on a brand-new benchmark with author-retrained baselines. All InfraOcc comparisons depend on the authors retraining official implementations of OccWorld and I²-World on a small dataset (215 training / 75 validation sequences). No test split is used and no variance/significance information is provided. The 9.02-point headline margin rests on this unvalidated setup. Cross-validation or at least error bars over multiple seeds would make the claims more robust.

Questions for authors

Q1. How is the moving-mask (L_moving) supervision constructed on Occ3D-nus in the absence of flow annotations? What is DQI's recall on genuinely moving regions at initialization, and how does performance degrade when DQI queries are initialized uniformly?

Q2. How does InfraWorld handle objects that enter the sensor range during the forecast horizon? Could you stratify Table 8 by "participants present in history" vs. "newly entering"?

Q3. What are the moving-mask head's precision/recall on InfraOcc, and how sensitive are the results to the flow-generation heuristics (τ_m = 0.8, 1.5× box enlargement, zero-flow voting)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.