acceptodds
Under review as a conference paper at ICLR 2027

EvoWorld: Evolving Gaussians for 4D Occupancy Forecasting and Motion Planning

Abstract

Future 3D semantic occupancy forecasting and motion planning are central to autonomous driving, as they require models to reason about how surrounding scenes evolve and how the ego vehicle should act. Vision-centric world models must infer this evolution from camera observations, yet many existing approaches discretize scenes into latent embeddings, volumetric features, or quantized tokens and forecast future states through fixed-step autoregressive generation. This limits temporal flexibility and makes the predicted scene evolution difficult to interpret. We propose EvoWorld, a vision-centric, non-autoregressive occupancy world model that predicts a compact set of 4D semantic Gaussian primitives from multi-view camera histories to represent the entire forecast horizon. For each primitive, EvoWorld directly predicts conditional spatial geometry, temporal support, and planar velocity, specifying its shape, motion, and temporal contribution at any queried time. Conditioning and splatting these primitives yields semantic occupancy at arbitrary timestamps, with independent temporal queries that can be decoded in parallel. Primitive velocity is further decomposed into shared ego-induced scene motion and object-level dynamic motion, and the same Gaussian representation supports open-loop motion planning. Extensive experiments on Occ3D-nuScenes and Occ3D-Waymo demonstrate improved vision-centric future occupancy forecasting and strong motion planning performance, with further analyses showing accurate primitive motion and consistent object association.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Autonomous driving, occupancy forecasting, motion planning, gaussian splatting

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_z2abOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

EvoWorld is a vision-centric, non-autoregressive world model that uses continuous 4D semantic Gaussian primitives for occupancy forecasting at arbitrary timestamps and open-loop motion planning. It decomposes primitive motion into ego-induced and object-level components, improving motion interpretability and displacement accuracy. Experiments on Occ3D-nuScenes and Occ3D-Waymo demonstrate improved forecasting, competitive planning, and consistent object associations, while halving the Gaussian count preserves forecasting performance with lower memory usage and inference time.

Critical strengths

The method uses continuous 4D semantic Gaussians to explicitly represent scene geometry and motion, making future scene evolution more intuitive and interpretable. The shared representation supports independent queries at arbitrary timestamps within the forecast horizon and parallel decoding, while serving both occupancy forecasting and motion planning with temporal flexibility and computational efficiency.

Critical weaknesses
  1. The paper does not compare accuracy or efficiency with I²-World [1] or GenieDrive [2]. These recent methods may perform better or run faster, so leaving them out makes it hard to judge where EvoWorld stands and raises concerns about how the baselines were chosen. I would expect these comparisons to be included.

  2. A world model should be able to answer the question: “What would happen if the agent took a different action?” The paper does not test this ability or discuss whether EvoWorld can accurately predict different futures under different actions.

  3. I am still not clear on how EvoWorld differs from existing Gaussian-based methods, especially GaussianAD, which already predicts Gaussian displacements for future scene prediction and planning. What does the shared continuous 4D representation add beyond 3D Gaussians with time-dependent displacements? Comparing these approaches under the same setup, including a baseline that interpolates discrete displacement predictions over time, would help clarify the contribution.

[1] I²-World: Intra-Inter Tokenization for Efficient Dynamic 4D Scene Forecasting. ICCV, 2025.

[2] GenieDrive: Towards Physics-Aware Driving World Model with 4D Occupancy Guided Video Generation. CVPR, 2026.

Questions for authors

see Weakness

5 comments

We thank the reviewer for highlighting the interpretability and temporal flexibility of our explicit 4D Gaussian representation. We address the requested baselines, alternative future actions, and differences from GaussianAD.

W1. Comparisons with I²-World and GenieDrive

Evaluation setting. The original protocols differ in whether current occupancy and the future ego trajectory are provided as ground truth or predicted:

Original settingCurrent occupancyFuture ego trajectory
I²-World / GenieDriveGT (I²-World-STC: predicted)GT
EvoWorldPredictedPredicted

Adapted baselines. We feed both models STCOcc-predicted current occupancy. We replace I²-World-STC's ground-truth future ego motion with a predicted trajectory and use GenieDrive's released motion predictor. Their forecasting weights remain unchanged.

Occ3D-nuScenes results:

MetricTimeI²-World-STCGenieDriveEvoWorld-12.8kEvoWorld-25.6k
mIoU ↑1s13.8617.1915.6815.71
2s7.9611.9013.6513.62
3s5.498.9911.5211.46
Avg.9.1012.6913.6213.60
IoU ↑1s22.0125.6625.7425.56
2s16.1120.6824.1924.03
3s13.0217.4022.0521.86
Avg.17.0521.2523.9923.82
L2 (m) ↓1s0.340.350.200.19
2s0.760.740.270.26
3s1.411.260.380.35
Avg.0.840.780.280.27
Collision (%) ↓1s0.300.290.140.09
2s0.710.710.120.14
3s2.111.270.280.26
Avg.1.040.760.180.16

End-to-end efficiency on one A100:

MethodSTCOcc (ms)Forecasting (ms)Mean (ms)Median (ms)Peak (GiB)Params (M)
EvoWorld-12.8k—234.9234.9220.74.3859.2
EvoWorld-25.6k—401.6401.6389.76.3662.5
STCOcc + I²-World287.5156.0443.5411.63.2282.8
STCOcc + GenieDrive287.5150.5438.0413.73.1163.6

Both EvoWorld variants outperform the adapted baselines across all four average metrics and have lower end-to-end latency, with higher peak memory. These results support direct forecasting from a continuous 4D Gaussian world as an effective alternative to autoregressive generation from compressed scene representations.

W2. Controllable action-conditioned generation

We agree that predicting scene responses to alternative actions is valuable. EvoWorld follows the predictive forecasting setting of OccWorld, PreWorld, and SparseWorld, jointly predicting future occupancy and ego trajectories from historical observations. In EvoWorld, the ego trajectory is a prediction output, whereas I²-World and GenieDrive condition future scene generation on ego motion or driving controls. We clarify this distinction and leave action-conditioned generation for future work.

W3. Difference from GaussianAD; interpolation baseline

Two main distinctions from GaussianAD.

  1. Future-region coverage. GaussianAD propagates current-frame 3D Gaussians and notes the lack of completion for newly observed regions. EvoWorld learns Gaussians over the full future horizon and can represent newly observed regions.
  2. Explicit temporal support. GaussianAD integrates temporal features and predicts Gaussian flow. EvoWorld additionally learns a temporal center and scale for each Gaussian, allowing its contribution to change with time.

Controlled interpolation baseline. Each Gaussian predicts XY displacements at 1, 2, and 3s, with zero displacement at t=0 and linear interpolation between timestamps. All other components and supervision remain unchanged.

MethodmIoU 1s2s3sAvg.IoU 1s2s3sAvg.
Interpolated displacements14.4312.4410.4412.4422.7321.1819.1721.03
EvoWorld-12.8k15.6813.6511.5213.6225.7424.1922.0523.99

EvoWorld improves mIoU and IoU at every evaluated horizon. The improvements support the effectiveness of our continuous velocity parameterization over directly predicting and interpolating future displacements.

Thank you for the response. However, the differences cited by the authors do not justify omitting comparisons in the main manuscript, as the occupancy tasks are essentially the same. These studies are also absent from the main manuscript’s related work section (with only I²World mentioned in the appendix). Furthermore, the main manuscript was not revised during the rebuttal period to address these omissions. This raises strong suspicions that the authors have deliberately avoided discussing and comparing against these studies.

Thank you for raising this concern. We agree that the shared occupancy task warrants a direct comparison in the main manuscript. Given the limited rebuttal period, we prioritized completing the requested experiments and provided the comparison in our rebuttal response. We understand why leaving the manuscript unchanged raised concerns. We are now revising the manuscript to include the comparison in the main results and discuss each study in Related Work. We will share the updated manuscript as soon as possible.

Reviewer_jhDpOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvoWorld represents future driving scenes with a shared set of 4D semantic Gaussian primitives for occupancy forecasting and motion planning. The method separately parameterizes spatial geometry, temporal support, and planar velocity, and decomposes motion into a shared scene component and an object-level residual. Occupancy can be queried at arbitrary timestamps, while a planning head predicts trajectories from Gaussian features and anchors. Experiments evaluate occupancy forecasting on Occ3D-nuScenes and Occ3D-Waymo, together with open-loop planning.

Critical strengths

(+) The Gaussian representation supports direct occupancy queries at arbitrary timestamps without sequential rollout. Separating conditional geometry from velocity makes the motion parameters explicit, and the parameterization and motion-decomposition ablations in Table 3 support the proposed design.

(+) The method improves average occupancy forecasting across two datasets. On nuScenes, camera-only mIoU increases from 13.20 for SparseWorld to 13.60, while the average planning collision rate decreases from 0.29% to 0.16%. The object-displacement and primitive-association experiments provide further quantitative evidence of motion accuracy and temporal consistency.

Critical weaknesses

(-) Table 5 reports latency and memory for occupancy forecasting, whereas the planning head operates directly on Gaussian features and anchors without decoding future occupancy. Could the authors separately report end-to-end latency and peak GPU memory for planning-only inference, including the encoder, Gaussian refinement, and planning head? Comparisons across primitive counts would clarify the computational cost of using the Gaussian representation for planning.

(-) Inspired by future-prior distillation approaches such as AMap, could the authors investigate transferring the planning-relevant knowledge learned by EvoWorld into a lightweight student planner that does not construct and refine the full set of 4D Gaussian primitives at inference? Comparing planning performance, latency, and peak memory under matched observation inputs and training data would help determine how much of the benefit requires retaining the explicit Gaussian representation at deployment.

[1] AMap: Distilling Future Priors for Ahead-Aware Online HD Map Construction, CVPR 2026

Questions for authors

Q1: How does planning performance change when the Gaussian representation is retrained using only current-frame occupancy supervision, with the encoder and planning-head capacity held fixed?

2 comments

We thank the reviewer for recognizing EvoWorld's direct arbitrary-time queries and geometry–motion separation. We provide planning-only profiles, a distillation experiment, and a current-frame-supervision control.

W1. Planning-only latency and memory

We profile planning-only end-to-end inference on one H100, covering image encoding, Gaussian-state construction and refinement, and the planning head. Future occupancy decoding is excluded.

ModelTotal (ms) ↓Hz ↑Peak (GiB) ↓Image encoding (ms)Gaussian world state (ms)Planning head (ms)
EvoWorld-6.4k67.114.93.4119.547.01.0
EvoWorld-12.8k95.210.54.4019.675.11.0
EvoWorld-25.6k155.76.46.3819.7135.21.3

The planning head adds about 1ms; most latency comes from Gaussian-state construction and refinement. The 12.8k model reaches 10.5Hz and preserves forecasting accuracy. The faster 6.4k model incurs a loss in occupancy accuracy (Appendix C.2).

W2. Distilling EvoWorld into a lightweight planner

We distill a frozen EvoWorld-25.6k planner into a 15M-parameter ResNet-18/Transformer student. At inference, the student uses six current-frame images and the same ego state.

We train over 24 epochs and three seeds, using exclusively teacher trajectory L1 and planning-feature MSE losses.

Planning on Occ3D-nuScenes (inference on one H100):

ModelParams (M)Latency (ms) ↓Peak (GiB) ↓L2 1s / 2s / 3s (m)L2 avg. ↓Col. 1s / 2s / 3s (%)Col. avg. ↓
EvoWorld-25.6k62.5155.76.380.188 / 0.256 / 0.3480.2640.095 / 0.142 / 0.2610.166
Student15.02.840.120.187 / 0.251 / 0.3470.262 ± 0.0000.119 / 0.142 / 0.2290.163 ± 0.010

The student matches the teacher's planning L2 (0.262 vs. 0.264m) and collision rate (0.163% vs. 0.166%), with 2.84ms latency and 0.12GiB peak memory. The result shows that a compact planner can learn from the Gaussian world's planning outputs.

Q1. Planning with current-frame-only occupancy supervision

We keep the 12.8k architecture, image history, planning head, and planning loss fixed, and change occupancy supervision to the current frame.

MetricCurrent-only (1s / 2s / 3s / Avg.)Full EvoWorld (1s / 2s / 3s / Avg.)
mIoU ↑3.87 / 1.76 / 1.13 / 2.2515.68 / 13.65 / 11.52 / 13.62
IoU ↑13.56 / 9.38 / 7.74 / 10.2325.74 / 24.19 / 22.05 / 23.99
L2 (m) ↓0.22 / 0.31 / 0.41 / 0.320.20 / 0.27 / 0.38 / 0.28
Collision (%) ↓0.21 / 0.21 / 0.31 / 0.240.14 / 0.12 / 0.28 / 0.18

Compared with current-only supervision, future occupancy supervision reduces planning L2 from 0.32 to 0.28m and collision rate from 0.24% to 0.18%, while improving average forecasting mIoU from 2.25 to 13.62. This shows that learning future scene evolution provides a more useful Gaussian state for planning.

Thank you for your detailed responses and analyses, which have addressed my concerns. Since my initial rating was already positive, I will maintain my score.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response

We thank the reviewers for recognizing EvoWorld's explicit 4D Gaussian modeling, direct arbitrary-time queries, and principled geometry–motion parameterization. We clarified the scope of our claims and provided targeted evidence on comparisons, robustness, planning, and efficiency.

Clarifications and additional evidence

1. Scope and comparisons with related works (Reviewer z2ab). We clarified that EvoWorld forecasts future occupancy and ego trajectories from historical observations, under the established setting of OccWorld, PreWorld, and SparseWorld. We provided adapted I²-World and GenieDrive comparisons, showing improvements across all four average forecasting and planning metrics and end-to-end latency, with higher memory use. We clarified future-region coverage and explicit temporal support relative to GaussianAD, and reported a controlled displacement-interpolation baseline (+1.18 mIoU, +2.96 IoU).

2. Robustness and additional experiments (Reviewer Mk5Q). We reported three independent nuScenes runs, each exceeding SparseWorld's reported average mIoU and IoU, and analyzed class-specific differences. We provided quantitative Waymo occupancy evaluation at intermediate 0.1s timestamps and EPE results under four association and aggregation protocols, supporting the forecasting and planning findings.

3. Planning and lightweight deployment (Reviewers Mk5Q and jhDp). We provided an image-feature planning control and a current-frame-only occupancy-supervision ablation, showing the planning benefit of a predictive Gaussian state. We reported planning-only latency and memory, identifying Gaussian refinement as the main cost. We also distilled the Gaussian planner into a 15M-parameter student that matches teacher accuracy at 2.84ms and 0.12GiB.

4. Efficiency and Waymo fairness (Reviewer Mk5Q). We clarified that reducing Gaussian count preserves forecasting accuracy and brings EvoWorld-12.8k to comparable inference latency with SparseWorld, although peak memory remains higher. We provided 25.6k Waymo results alongside 12.8k, confirming that both improve average forecasting accuracy and collision rate over SparseWorld. We verified SparseWorld's official 64-epoch Waymo schedule and clarified our claim of strong, competitive open-loop planning.

We thank the reviewer for recognizing the principled geometry–motion parameterization and flexible temporal queries. We address the accuracy margin, efficiency and planning, and the requested evaluation controls.

W1. Robustness of the mIoU margin

We report two additional nuScenes camera-only runs (seeds 42 and 99) alongside the paper's EvoWorld-12.8k run (seed 17):

SeedmIoU 1s / 2s / 3smIoU avg.IoU avg.L2 avg. (m)Collision avg. (%)
17 (paper setting)15.68 / 13.65 / 11.5213.6223.990.2820.182
4215.54 / 13.46 / 11.3513.4523.360.2800.200
9915.59 / 13.56 / 11.4613.5423.820.2800.182
Mean ± sample std. (n=3)15.60 ± 0.07 / 13.56 ± 0.10 / 11.44 ± 0.0913.54 ± 0.0923.72 ± 0.330.281 ± 0.0010.188 ± 0.010
SparseWorld (reported run)14.93 / 13.15 / 11.5113.2022.030.270.29

Across three seeds, EvoWorld exceeds SparseWorld's reported average mIoU and IoU, with a 0.09-point standard deviation in average mIoU versus a 0.34-point mean gap. At 3s, semantic mIoU is near parity and IoU for cars, buses, and trucks is lower, while bicycles, motorcycles, and pedestrians improve. A plausible cause is the constant-velocity, time-invariant-shape assumption: turns, acceleration, and changing visible surfaces are harder to represent over long horizons, particularly for extended vehicles.

EvoWorld improves occupied-region IoU at every evaluated horizon on nuScenes and Waymo, providing complementary evidence of stronger geometric occupancy forecasting.

W2. Efficiency, planning, and the Waymo primitive count

Our efficiency contribution is the ability to reduce Gaussian count while preserving forecasting accuracy. EvoWorld-12.8k achieves comparable inference latency to SparseWorld (154.0 vs. 159.0ms), with higher peak memory (4.40 vs. 2.67GiB). This accuracy–cost tradeoff motivates the 12.8k Waymo configuration. To address the reviewer's question about selective reporting, we also evaluate 25.6k; both configurations improve average forecasting accuracy and collision rate over SparseWorld.

Our paper claims strong, competitive open-loop planning. EvoWorld-25.6k matches SparseWorld's average nuScenes L2 (0.27m) and reduces collision rate (0.16% vs. 0.29%). On Waymo, its L2 is slightly higher (0.54 vs. 0.52m) and collision rate is lower (0.14% vs. 0.22%).

Occ3D-WaymomIoU 1/2/3/Avg ↑IoU 1/2/3/Avg ↑L2 1/2/3/Avg ↓Col. 1/2/3/Avg ↓
SparseWorld17.38 / 14.96 / 13.15 / 15.1638.82 / 36.17 / 33.83 / 36.270.12 / 0.41 / 1.02 / 0.520.00 / 0.21 / 0.45 / 0.22
EvoWorld-12.8k17.71 / 15.83 / 13.98 / 15.8442.39 / 41.08 / 39.77 / 41.080.14 / 0.45 / 1.07 / 0.550.02 / 0.12 / 0.41 / 0.18
EvoWorld-25.6k18.02 / 15.91 / 13.97 / 15.9742.70 / 41.20 / 40.82 / 41.570.13 / 0.43 / 1.05 / 0.540.00 / 0.10 / 0.31 / 0.14

W3. Planning directly from image features

We compare image features with Gaussian features as planner input, keeping the planning head and training protocol fixed. This isolates the effect of the learned representation.

Planner inputL2 1s2s3sAvg.Collision 1s2s3sAvg.
Image features0.220.340.480.350.170.200.360.24
EvoWorld-12.8k0.200.270.380.280.140.120.280.18
EvoWorld-25.6k0.190.260.350.270.090.140.260.16

Both Gaussian configurations reduce trajectory L2 error and collision rate over the image-feature control, showing that the 4D Gaussian state provides more effective features for planning.

Q1. SparseWorld Waymo reproduction and training schedules

SparseWorld does not report results on Waymo in its original paper. We independently reproduced SparseWorld's Waymo results using its official 64-epoch training recipe, including 5 epochs of pretraining and 6 epochs of timestamp scheduling. EvoWorld follows its own 20-epoch training recipe.

Q2. Multi-seed variance

As shown in W1, EvoWorld's three runs yield 13.54 ± 0.09 average mIoU, compared with 13.20 for SparseWorld's reported run. The 0.34-point margin exceeds the observed EvoWorld run-to-run variation.

Q3. Quantitative accuracy at off-grid timestamps

Occupancy is supervised at 0.5s timestamps. We additionally evaluate intermediate 0.1s queries against UniOcc's 10Hz Waymo occupancy labels. These intermediate timestamps receive no occupancy supervision during training. Each entry reports mIoU / IoU (%), grouped by starting second.

Interval+0.1s+0.2s+0.3s+0.4s+0.5s
0–1s19.55/43.4919.47/43.3019.31/43.1619.08/43.1518.90/43.11
1–2s17.59/42.2217.44/41.9617.25/41.8017.00/41.7916.80/41.71
2–3s15.57/40.8615.36/40.6515.19/40.5215.02/40.4714.87/40.39
Interval+0.6s+0.7s+0.8s+0.9s+1.0s
0–1s18.70/42.9818.42/42.7218.17/42.5317.92/42.4917.71/42.39
1–2s16.60/41.5716.43/41.3316.23/41.1916.03/41.1615.83/41.08
2–3s14.71/40.3014.52/40.1114.34/39.9714.16/39.8813.98/39.77

The intermediate queries retain meaningful forecasting accuracy between supervised timestamps. These results provide the requested quantitative evaluation of arbitrary-time predictions beyond the training grid.

Q4. Sensitivity of object-level EPE to aggregation and association

We evaluate EPE using mean or median velocity and either initial GT-box assignment or the propagated, class-matched assignment used in Table 4.

Object displacement EPE, meters ↓ (1s / 2s / 3s):

AssignmentAggregationEvoWorldSingle planar velocityFull 4D covariance
GT box at t=0Median1.448 / 2.463 / 3.4471.537 / 2.828 / 4.2003.824 / 7.110 / 9.896
GT box at t=0Mean1.561 / 2.796 / 4.0301.762 / 3.298 / 4.8473.399 / 6.611 / 9.579
Table 4 associationMedian1.343 / 2.285 / 3.1201.480 / 2.753 / 3.7783.765 / 7.611 / 10.428
Table 4 associationMean1.474 / 2.618 / 3.6991.661 / 2.938 / 4.0303.587 / 7.061 / 9.617

EvoWorld achieves the lowest EPE at every horizon under all four protocols, supporting the benefit of separating ego-induced scene motion from object-level motion.

For clarity, Table 4's propagated association continues to use class-matched ground-truth boxes at each queried timestamp to identify objects.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.