- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies 4D semantic occupancy forecasting with sparse occupancy world models for autonomous driving. The authors argue that existing sparse query-based models produce spatiotemporally inconsistent semantic rollouts because latent query semantics are not explicitly aligned with the refined points that form the rendered occupancy. To address this, they propose STC-World with two components: a coordinate-conditioned semantic decoder that uses point coordinates and query-relative offsets to correct point-level semantic logits, and adjacent-horizon temporal self-distillation that uses corrected semantics at one horizon as a teacher for the next under inherited query indices, with separate consistency terms for static and dynamic classes. On Occ3D-nuScenes, STC-World reports 13.47% average mIoU, 0.20 m planning L2, and an average collision rate of 0.09%, while running at 2.56 FPS.
Critical strengths
-
The proposed changes are lightweight and the experimental disclosure is comparatively transparent. The method adds little inference cost, and the ablations have a comprehensive coverage multi-seed variability, failure cases, and mechanism diagnostics.
-
The diagnosis of sparse-query semantic misalignment is reasonable. Treating query-level semantics as insufficiently tied to refined point geometry is a plausible limitation of sparse occupancy world models, and Table 8 provides preliminary evidence that coordinate conditioning helps.
Critical weaknesses
-
The occupancy gain is small and cannot be attributed to the proposed modules. Table 1 improves on the published SparseWorld numbers by 0.27 mIoU and 0.13 IoU, comparing that paper’s reported results with the authors’ own implementation. The final system is trained for 66 epochs (48+8+10), and there is no SparseWorld control continued under the same extra budget. Table 1 also omits stronger camera-based results on the same benchmark. SparseWorld-TC-Small [1] already reports 25.60 mIoU and 49.02 IoU (Du et al., 2026), well above STC-World’s 13.47 and 22.16. These numbers use future-trajectory conditioning, which Table 1 neither includes nor discusses, so the camera-based state-of-the-art claim is not supported.
-
The ablation pattern is strongly suspicious and is left unanalyzed. In Table 4, spatial alignment and temporal distillation change average mIoU by +0.03 and +0.06, while the combination jumps by +0.29. Table 5 repeats the same pattern. The authors should analyze or draw inferences from the ablation result data. Moreover, the paper reports these numbers without explaining why nearly every partial variant returns to about 13.2 while one configuration is an outlier.
-
The claim that spatiotemporal inconsistency is the central problem, and that STC-World solves it, is not supported by the paper’s own evidence. Fig. 1 and Fig. 3 illustrate the failure on PreWorld, so the motivating phenomenon is never measured on SparseWorld, the model this method actually modifies.
[1] Du et al. SparseWorld-TC: Trajectory-Conditioned Sparse Occupancy World Model. CVPR, 2026.
Questions for authors
The proposed method appears to be more of an occupancy forecasting algorithm than a world model, since action-related modeling is absent. The model is trained to predict future occupancy from historical image observations and the current ego state, and future trajectories serve as training targets rather than rollout inputs. If the authors insist on framing the task as world modeling, they should provide clear definitions and further clarify the task formulation.
7 comments
Response to Reviewer qaKz (2/3)
W2. The ablation pattern is strongly suspicious and is ...
A2. We thank the reviewer for pointing this out. We have revised the ablation discussion and included further analysis of the results. Although the partial variants follow different initialization paths, they retain largely the same learned sparse-world representation and modify only a small part of the final prediction process. Their performance therefore remains close to the common baseline level of approximately 13.2 mIoU. In Table 6, the row labeled "no alignment" means that neither coordinate-conditioned semantic alignment nor adjacent-horizon temporal self-distillation is newly enabled during the final adaptation stage. The "spatial only" row enables coordinate-conditioned semantic alignment during the final adaptation but disables adjacent-horizon temporal self-distillation. The "temporal only" row corresponds to the temporal checkpoint before the final adaptation stage. STC-World starts from the same temporal checkpoint, adds coordinate-conditioned semantic alignment, and enables adjacent-horizon temporal self-distillation during the final adaptation.
Table 7 reuses the spatial-only configuration from Table 6 as its reference. Specifically, the first row of Table 7, where neither the static KL term nor the dynamic-logit term is enabled, is the same coordinate-conditioned semantic alignment setting as the ``spatial only'' row in Table 6: coordinate-conditioned semantic alignment is enabled, while adjacent-horizon temporal self-distillation is disabled. Thus, Table 7 does not introduce a new baseline or initialization. It further adds the static KL term, the dynamic-logit term, or both to this spatial-only configuration, while keeping the query pairing and validity masks fixed. The final row therefore corresponds to the complete STC-World configuration and reproduces the 13.47 mIoU result reported in Table 6.
The similar performance of the partial variants around 13.2 mIoU indicates that each mechanism produces a conservative change when introduced alone. Coordinate-conditioned semantic alignment supplies location-specific corrections to point-level logits, while adjacent-horizon temporal self-distillation regularizes how these corrected semantics evolve across future horizons. Because the two mechanisms act on the same corrected point-level predictions, their joint optimization can produce a non-additive change in the final solution.
The additional consistency analysis shows complementary behavior. Coordinate-conditioned semantic alignment alone gives the highest boundary recall and geometry--semantic attachment, whereas the complete model produces the lowest dense semantic drift across all evaluated horizons. These results support complementary effects across different measurements.
W3. The claim that spatiotemporal inconsistency is...
A3. In the revision, we add visualizations of the ground truth, SparseWorld, and STC-World on the same scenes and forecast horizons. As show in Figure 7and Figure 8 (both in Supplementary Material), in these examples, SparseWorld produces less spatially localized occupancy and less consistent semantic layouts across future horizons, while STC-World better preserves the geometry and semantic structure of the predicted scene. These comparisons provide direct qualitative evidence on the baseline that our method modifies.
In addition, we report the dynamic- and static-class results of STC-World and the comparison methods in Table R3 (Table 5 in manuscript). The dynamic group contains bicycle, bus, car, construction vehicle, motorcycle, pedestrian, traffic cone, trailer, and truck. The static group contains drivable surface, other flat, sidewalk, terrain, manmade, and vegetation. The groups are computed as unweighted means of per-class IoUs, excluding others and barrier. STC-World improves the dynamic-class average from 10.33% to 10.76% over SparseWorld, a gain of 0.43 percentage points. The gains at 1, 2, and 3 s are 0.40, 0.44, and 0.42 points, respectively. The static-class average increases from 18.52% to 18.58%. The larger improvement for dynamic categories is consistent with the intended role of coordinate-conditioned semantic correction and temporal regularization, since these categories undergo larger changes in position and spatial support during rollout.
Table R3. Macro-averaged IoU (%) for dynamic and static class groups on Occ3D-nuScenes.
| Method | Dynamic 1s | Dynamic 2s | Dynamic 3s | Dynamic Avg. | Static 1s | Static 2s | Static 3s | Static Avg. |
|---|---|---|---|---|---|---|---|---|
| PreWorld | 9.06 | 5.56 | 3.56 | 6.06 | 18.36 | 15.96 | 13.76 | 16.03 |
| SparseWorld | 12.68 | 10.14 | 8.18 | 10.33 | 19.63 | 18.66 | 17.28 | 18.52 |
| STC-World | 13.08 | 10.58 | 8.60 | 10.76 | 19.69 | 18.72 | 17.32 | 18.58 |
Response to Reviewer qaKz (3/3)
Q1. The proposed method appears to be more of an occupancy...
A4. We agree that STC-World does not model how a scene responds to a prescribed sequence of future ego actions. It takes historical camera observations and the observed ego state as context, then predicts future scene states through a learned latent transition. Ground-truth future occupancy and ego trajectories provide supervision for the occupancy and trajectory-related objectives during training. At inference, the occupancy rollout does not receive a ground-truth future trajectory or a candidate control sequence; future states are generated by the model's learned transition.
We use occupancy world model in the forecasting-oriented sense adopted by prior work such as PreWorld and SparseWorld. In STC-World, sparse queries represent the latent scene state, and a learned transition rolls this state forward across forecast horizons. A decoder renders each predicted state as semantic occupancy, while a planning head uses the predicted scene representation and the current ego state to generate an ego trajectory. Thus, 4D semantic occupancy forecasting is the prediction task, latent scene-state rollout is the modeling mechanism, and planning is a downstream prediction based on the forecast scene.
Response to Reviewer qaKz (1/3)
Thank you for the valuable feedback, which helped us improve the paper. We have revised the manuscript accordingly. All revisions to the manuscript are shown in blue. Below we summarize the changes, and we would appreciate it if you could let us know whether they resolve your concerns.
W1. The occupancy gain is small and cannot be attributed...
A1. We thank the reviewer for pointing out this important distinction. In this revision, we conducted an additional experiment starting from a SparseWorld checkpoint and training only the two proposed moudles. This setting still achieves a result comparable to the complete training procedure, suggesting that the improvement does not depend on extensive fine-tuning but arises from the proposed core modules.
We now cite SparseWorld-TC-Small and report additional comparisons with SparseWorld-TC-Small and I$^2$-World in Tables R1 and R2 (Tables 10 and 11 in the Supplementary Material). To make the comparison as controlled as possible, we evaluate the reproduced baselines on the same Occ3D-nuScenes validation split. All methods are evaluated on the same 4,219 samples for occupancy forecasting and the same 4,819 samples for planning. For SparseWorld-TC-Small, we use the camera-only variant without future-trajectory conditioning and evaluate both occupancy forecasting and planning. For I$^2$-World, we likewise do not provide a future trajectory as a control input and use the same evaluation samples. We retain its native precomputed LiDAR/occupancy tokens and dataset-provided transformation matrices for temporal alignment.
Both SparseWorld-TC-Small and I$^2$-World achieve lower occupancy performance under our controlled protocol than in their originally reported settings. Two changes may contribute to this difference: removing future-trajectory conditioning and adding a jointly optimized planning head with its associated losses. Despite these changes, SparseWorld-TC-Small still outperforms STC-World in occupancy forecasting, achieving an average mIoU of 21.39% and an average IoU of 43.46%. STC-World, in contrast, achieves lower planning errors, with an average L2 error of 0.20 m and an average collision rate of 0.09%, compared with 0.29 m and 0.41% for SparseWorld-TC-Small. Our clarification is that STC-World and SparseWorld-TC target different operating points: SparseWorld-TC focuses on trajectory-conditioned occupancy forecasting, whereas STC-World jointly supports camera-based occupancy prediction and downstream planning without future-trajectory control inputs.
I$^2$-World achieves an average mIoU of 10.71% and an average IoU of 21.02% under the same evaluation protocol. One possible explanation is that removing future-trajectory conditioning and jointly optimizing the planning branch affect how the forecasting model uses its precomputed LiDAR/occupancy tokens, potentially weakening their geometric and temporal alignment with future occupancy. Because this setup differs from the original I$^2$-World protocol, we report these numbers as a reference under the modified setting, not as a direct reproduction of its published results.
Table R1. Controlled occupancy comparison with additional baselines on Occ3D-nuScenes. All methods use the same occupancy samples.
Table R2. Controlled planning comparison with additional baselines on Occ3D-nuScenes. All methods use the same planning samples and evaluation protocol.