acceptodds
Under review as a conference paper at ICLR 2027

KIO-World: A Kinematics-Informed Sparse Occupancy World Model

Abstract

Occupancy world models reconstruct 3D scene geometry and semantics from past observations and predict their future evolution. Recent sparse models decode multiple future horizons from query anchors, but do not explicitly couple historical evidence alignment and future prediction through a shared motion state. Consequently, independently moving scene content can cause misaligned image feature sampling, while horizon-specific predictions lack a common motion reference. To address these issues, we present KIO-World, a kinematics-informed sparse occupancy world model that jointly models world states across horizons within a unified kinematic framework. Specifically, we equip each query anchor with a horizon-shared kinematic state that compensates for scene dynamics during historical feature sampling and provides world-state references for future query anchor evolution. To preserve spatial structure during scene evolution, we further introduce a hierarchical residual point refinement module, in which query points iteratively inherit parent geometry, and anchor-wise pose updates are decoupled from point-wise shape refinement. These mechanisms introduce complementary structural priors that support temporal reasoning and geometric refinement while retaining the flexibility of direct occupancy decoding. Extensive experiments demonstrate that KIO-World achieves state-of-the-art performance, outperforming existing camera-centric methods by 11.4% and 6.3% in average semantic mIoU and geometric IoU, respectively. Code will be released upon publication.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Occupancy World Model, 4D Occupancy Forecasting, 3D Occupancy Prediction, Autonomous Driving

Reviews

Among:
Shape:
1234this paper, Top 17%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.75 over 4 reviews, above 83% of the others.
Reviewer_ygc8OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes KIO-World, a sparse occupancy forecasting model conditioned on historical multi-view images and future ego trajectories. It introduces a shared kinematic state for each query anchor, which supports motion-compensated historical feature sampling and soft regularization of future anchor centers. It also proposes Hierarchical Residual Point Refinement (HRPR), combining parent-point inheritance, anchor-level transformations, and point-wise residual updates. The paper evaluates two model sizes on Occ3D-nuScenes and reports improvements over corresponding SparseWorld-TC baselines. Additional experiments cover RayIoU, per-class performance, kinematic-state sharing, HRPR transfer, and longer prediction horizons.

Critical strengths

S1. The main architectural choices have direct experimental support. I find Table 10 particularly valuable because it tests the proposed sharing scheme against direction-specific and horizon-specific states. The shared variant achieves 28.2 mIoU and 51.8 IoU, compared with 27.8/50.7 and 27.2/49.9, respectively. This supports the usefulness of sharing a common motion reference across temporal directions and prediction horizons. Table 4 provides complementary evidence for HRPR through improvements on both OPUS-T and SparseWorld-TC. Together, these experiments establish more than an improvement from combining several unexamined components.

S2. The reported improvements are consistent across model sizes and complementary metrics. Both model sizes improve over their corresponding ResNet-50 SparseWorld-TC baselines. For the Large model, average mIoU increases from 26.4 to 29.4 and IoU from 49.2 to 52.3. The RayIoU evaluation strengthens this result: for the Small model, average RayIoU increases from 28.5 to 31.1. The per-class analysis also reports improvements on small movable categories. I consider this combination of evidence meaningful because it supports the method across multiple evaluation views and reduces the concern that the voxel-IoU gains arise primarily from overly thick occupancy predictions.

Critical weaknesses

W1. The attribution of the future-regularization gain remains incomplete. Table 11 shows that adding the future-center regularizer, L_reg, alone changes mIoU/IoU from 27.9/50.4 to 27.9/50.6, while combining it with the static-scene prior, L_static, yields 28.2/51.8. The authors appropriately acknowledge the importance of the static-scene prior. However, without a variant that enables L_static and disables L_reg, the additional benefit of the future-center regularizer when the static prior is already present remains unclear. This leaves the attribution of the combined regularization gain incomplete, while preserving the evidence for state sharing in Table 10. The description of the final “Reg.” step in Table 3 should explicitly identify the combined losses and distinguish their joint effect from the incremental contribution of L_reg.

W2. Clarity of baseline provenance and evaluation settings. Table 8 explicitly reports a reevaluation of the released SparseWorld-TC-Small checkpoint, but the checkpoint or evaluation differences underlying its results relative to Table 1 are not fully explained. Table 3 is described as a reproduced direct-forecasting baseline, and its architectural and training relationship to SparseWorld-TC could be specified more explicitly. For the 8-second comparison in Table 2, the common test-time extension procedure is stated, but the training horizons of the SparseWorld-TC checkpoint are not specified. These clarifications would help establish the precise scope and comparability of the reported results.

Questions for authors

1.Contribution of the future-center regularizer. Appendix D.3 and Table 11 already discuss the importance of the static-scene prior to the combined regularization gain. Could you additionally report the configuration with (L_{\mathrm{static}}) enabled and (L_{\mathrm{reg}}) disabled, keeping the remaining settings fixed? This would clarify the additional contribution of (L_{\mathrm{reg}}) once the static-scene prior is present. 2.Baseline definitions and evaluation protocol. Table 8 explicitly identifies its SparseWorld-TC-Small results as a reevaluation of the released checkpoint, while Table 3 uses a reproduced direct-forecasting baseline. Could you clarify the checkpoint or evaluation differences underlying the SparseWorld-TC-Small results in Tables 1 and 8, and specify how the Table 3 baseline differs from SparseWorld-TC? For the 8-second comparison in Table 2, please also specify the future horizons supervised during training of the SparseWorld-TC checkpoint. The common test-time extension procedure is already stated, but the checkpoint’s training horizons remain unclear to me. 3.Numerical configuration details. Appendix C provides several useful implementation details, including pretraining information and loss weights. However, I could not locate the numerical values of (\mathbf{b}) in Eq. (12), (v_{\max}) and (\omega_{\max}) in Eq. (14), the static-anchor matching distance, or the occupancy output threshold. Could you provide these values, together with the decoder depth and complete per-layer point schedules for both model sizes, in a compact configuration table or configuration file? For settings inherited from a baseline, identifying the exact configuration would be sufficient. 4.Dependence of current-frame predictions on future trajectory conditions. Appendix E.2 describes identical current-frame forecasts under different future trajectory conditions, while Appendix C.2 describes temporal attention across the (H+1) horizon tokens. Is this invariance guaranteed by the architecture, or is it an observation specific to the illustrated examples? Please clarify the relevant attention mask or inference procedure, if any.

3 comments
AuthorsOpenReview ↗

Response to Reviewer ygc8

We thank Reviewer ygc8 for the detailed review and positive assessment of our experiments. We address all questions below. All revisions are marked in blue in the updated manuscript.

Q1 & W1. Contribution of Lreg\mathcal{L}_ {\mathrm{reg}} Given Lstatic\mathcal{L}_ {\mathrm{static}}, and Clarity in Table 3

We trained KIO-World-Small for 70 epochs (keeping HRPR, motion-compensated historical alignment, and Lfeas\mathcal{L}_ {\mathrm{feas}} fixed) with Lstatic\mathcal{L}_ {\mathrm{static}} enabled and Lreg\mathcal{L}_ {\mathrm{reg}} disabled (Appendix D.3, Table 11):

Lreg\mathcal{L}_ {\mathrm{reg}}Lstatic\mathcal{L}_ {\mathrm{static}}mIoU (%)IoU (%)
27.950.4
✓27.950.6
✓28.050.8
✓✓28.251.8

This comparison shows why neither term works alone and both are required together:

  • Lstatic\mathcal{L}_ {\mathrm{static}} calibrates the state at h=0h=0, while Lreg\mathcal{L}_ {\mathrm{reg}} propagates it to future centers (h≥1h \ge 1).
  • Lreg\mathcal{L}_ {\mathrm{reg}} only: Static anchors absorb future center drift into fake velocities vqL\boldsymbol{v}_ q^L instead of correcting cq,hL\boldsymbol{c}_ {q,h}^L.
  • Lstatic\mathcal{L}_ {\mathrm{static}} only: Suppressing background motion slightly helps historical sampling, but the calibrated state is never passed to future centers.
  • Both Lstatic\mathcal{L}_ {\mathrm{static}} and Lreg\mathcal{L}_ {\mathrm{reg}}: Once Lstatic\mathcal{L}_ {\mathrm{static}} keeps static velocities near zero (while leaving movable anchors free), Lreg\mathcal{L}_ {\mathrm{reg}} enforces cross-horizon center consistency and brings the main gain (+0.2 mIoU and +1.0 IoU over Lstatic\mathcal{L}_ {\mathrm{static}} alone). We also updated Section 4.3 and Table 3 to state explicitly that the final "Reg." step includes the combined kinematic losses (Lreg,Lstatic,Lfeas\mathcal{L}_ {\mathrm{reg}}, \mathcal{L}_ {\mathrm{static}}, \mathcal{L}_ {\mathrm{feas}}) and refers to Table 11 for their separate effects.

Q2 & W2. Baseline Definitions, Checkpoint Provenance, and Evaluation Protocols

  1. SparseWorld-TC-Small in Table 1 (25.6 / 49.0) vs. Table 8 (25.3 / 48.3):

    • Table 1 (25.6 / 49.0) directly quotes the numbers reported in Du et al. (2026).
    • Because Du et al. (2026) do not report per-class forecasting IoU or 0 to 3 s RayIoU for Small, we evaluated their officially released SparseWorld-TC-Small checkpoint with its official evaluation script, which gives 25.3 mIoU and 48.3 IoU. We kept the higher paper-reported 25.6 / 49.0 in Table 1 (favorable to the baseline) and reported the released checkpoint's evaluation in Tables 7-9 (Appendix D.2).
  2. Table 3 Baseline vs. SparseWorld-TC-Small:

    • For a controlled ablation in Table 3 where only HRPR, historical alignment, and kinematic regularization change across rows, Row 1 (25.4 / 48.7) is trained in the KIO-World codebase with center_offset refinement (as in OPUS and SparseWorld-TC), rigid ego-pose sampling, and direct decoding without kinematic losses (Section 4.3).
    • All rows in Table 3 share the base setup in Appendix C, trained for 70 epochs.
  3. SparseWorld-TC in Table 2 (8-Second Comparison):

    • Since Du et al. (2026) did not release a Large checkpoint, Table 2 (22.3 / 45.4) directly quotes the official SparseWorld-TC-Large numbers reported in Table 2 of Du et al. (2026).
    • Both SparseWorld-TC-Large and KIO-World-Large are supervised strictly on 0-3s during training and directly extended to 0-8s at test time without retraining (jointly decoding all 99 horizons in a single forward pass), as now explicitly stated in Section 4.2 and Table 2.

Q3. Numerical Configuration Details

Below we list all requested hyperparameters and schedules (shared by both model sizes unless noted). We have also added inline values in Appendices C.2-C.4 and this full summary table in Appendix F (Table 14) of the revised manuscript:

Parameter / ComponentReferenceKIO-World-SmallKIO-World-Large
Number of query anchors QQEq. (2), Sec. 4.16006004,8004{,}800
Decoder depth LL (active motion layers)Sec. 3.2, App. C.255 (ℓ∈{2,…,5}\ell \in \{2,\dots,5\}, ℓ=1\ell=1 rigid)55 (ℓ∈{2,…,5}\ell \in \{2,\dots,5\}, ℓ=1\ell=1 rigid)
Per-layer point schedule (N0,N1,…,N5)(N_0, N_1, \dots, N_5)Sec. 3.4, App. C.2(1,8,16,32,64,128)(1, 8, 16, 32, 64, 128)(1,2,4,8,12,16)(1, 2, 4, 8, 12, 16)
Final total points per horizon (Q×N5Q \times N_5)Sec. 4.176,80076{,}80076,80076{,}800
Feature dim DD / Point descriptor dimTab. 6, App. C.2256256 / 3232256256 / 3232
Sampling groups GG (rigid / motion) ×\times points PPApp. C.244 (22 / 22) ×4\times 444 (22 / 22) ×4\times 4
State bound b=(bvx,bvy,bvz,bω)\boldsymbol{b} = (b_ {v_x}, b_ {v_y}, b_ {v_z}, b_\omega)Eq. (12)(30.0,30.0,3.0) m/s, 1.0 rad/s(30.0, 30.0, 3.0)\,\text{m/s},\, 1.0\,\text{rad/s}(30.0,30.0,3.0) m/s, 1.0 rad/s(30.0, 30.0, 3.0)\,\text{m/s},\, 1.0\,\text{rad/s}
Huber threshold δ\delta in Lreg\mathcal{L}_ {\mathrm{reg}}Eq. (5), App. C.41.0 m1.0\,\text{m}1.0 m1.0\,\text{m}
Static-anchor matching distanceEq. (13), App. C.42.0 m2.0\,\text{m} (8 static classes)2.0 m2.0\,\text{m} (8 static classes)
Feasibility bounds (vmax⁡,ωmax⁡)(v_ {\max}, \omega_ {\max})Eq. (14), App. C.430.0 m/s, 1.0 rad/s30.0\,\text{m/s},\, 1.0\,\text{rad/s}30.0 m/s, 1.0 rad/s30.0\,\text{m/s},\, 1.0\,\text{rad/s}
Inference occupancy score thresholdApp. C.30.250.25 (sigmoid score)0.250.25 (sigmoid score)
Training epochs / batch size / initial LRSec. 4.17070 / 88 / 2×10−42\times 10^{-4}7070 / 88 / 2×10−42\times 10^{-4}

Q4. Dependence of Current-Frame (h=0h=0) Predictions on Future Trajectory Conditions

Thank you for pointing out this distinction between Appendix C.2 and Appendix E.2. Exact invariance of the h=0h=0 prediction is not guaranteed by the architecture. Instead, the nearly identical 0 s0\,\text{s} output in Fig. 6 is an empirical behavior. Specifically:

  1. Direct conditioning at h=0h=0 is invariant: At h=0h=0, the ego-pose transform is fixed to the identity T0→0=I4\boldsymbol{T}_ {0\rightarrow 0} = \boldsymbol{I}_ 4 with t0=0t_0 = 0. As a result, both the AdaLN-Zero condition g0\boldsymbol{g}_ 0 (Eq. 10) and the h=0h=0 camera projection in 4D feature sampling do not depend on future ego poses {Th→0}h=1H\{\boldsymbol{T}_ {h\rightarrow 0}\}_ {h=1}^H.
  2. Indirect coupling via unmasked temporal attention: In Appendix C.2, per-query temporal attention across the H+1H+1 horizon tokens (h=0,…,H)(h = 0, \dots, H) is unmasked (bidirectional), with no causal mask blocking h=0h=0 from attending to h≥1h \ge 1. In Fig. 6, all H+1H+1 horizons are decoded together in one forward pass, so future trajectory conditions can in principle affect h=0h=0 features through tempo_attn.
  3. Why h=0h=0 stays nearly unchanged in practice: Because h=0h=0 is directly grounded in current camera observations (T0→0=I4\boldsymbol{T}_ {0\rightarrow 0} = \boldsymbol{I}_4) and supervised by the same current-frame occupancy ground truth under any future trajectory, tempo_attn mainly passes context from h=0h=0 to future horizons rather than changing h=0h=0 geometry based on future maneuvers. Thus, the 0 s0\,\text{s} predictions appear visually indistinguishable in Fig. 6.

We have clarified the unmasked temporal attention setup in Appendix C.2 and changed "identical" to "visually indistinguishable" in Appendix E.2 with a brief explanation.

Before the discussion concludes, we would welcome your thoughts on our response and revised manuscript. In particular, please let us know whether any concerns still require clarification. We remain available to respond during the remaining discussion period. Thank you for your time.

Reviewer_CB6DOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

KIO-World introduces a shared kinematic state for historical feature alignment and future query-center regularization, together with hierarchical residual point refinement (HRPR). Experiments on Occ3D-nuScenes show improvements over matched SparseWorld-TC models.

Critical strengths

Sharing a soft motion prior between historical alignment and future prediction is well motivated. The state-sharing and soft-versus-hard rollout ablations provide useful support for this design.

Critical weaknesses
  1. The contribution of future motion coupling remains unclear. In Table 11, center fitting alone improves IoU from 50.4 to 50.6, whereas adding the static prior raises it to 51.8. Without a static-only center constraint, the results do not establish how much dynamic coupling contributes beyond stabilizing static geometry. This weakens the evidence for the main contribution.
  2. HRPR combines parent-point inheritance with anchor-level pose updates, but Tables 3–4 only evaluate the complete module. The missing parent-plus-residual baseline leaves unclear whether the pose head adds value beyond simple geometric inheritance, limiting the evidence for the second contribution.
Questions for authors
  1. With historical alignment and HRPR fixed, how does the full model compare with future center regularization applied only to static anchors using ego motion?
  2. Under the same point schedule, how does complete HRPR compare with inheriting parent points and predicting child residuals without the pose head?
2 comments
AuthorsOpenReview ↗

Response to Reviewer CB6D

We thank Reviewer CB6D for recognizing the motivation of our soft motion prior and for suggesting two targeted ablations to examine (1) dynamic future motion coupling vs. static-only center stabilization and (2) the anchor-level pose head vs. parent-plus-residual inheritance in HRPR. We ran both experiments under matched settings and added them to the revised manuscript.

Q1 & W1. Full Model vs. Static-Only Future Center Regularization Using Ego Motion

With HRPR, motion-compensated historical alignment, Lstatic\mathcal{L}_ {\mathrm{static}}, and Lfeas\mathcal{L}_ {\mathrm{feas}} fixed, we evaluated applying Lreg\mathcal{L}_ {\mathrm{reg}} only to static anchors using pure ego motion Th→0−1cq,0L\boldsymbol{T}_ {h\rightarrow 0}^{-1}\boldsymbol{c}_ {q,0}^{L} (setting velocity to zero in future references and placing no center constraint on non-static anchors). Together with Lstatic\mathcal{L}_ {\mathrm{static}}-only, the full comparison (Appendix D.3, Table 11) is:

ConfigurationLstatic\mathcal{L}_ {\mathrm{static}}Future Center Regularization (Lreg\mathcal{L}_ {\mathrm{reg}})mIoU (%)IoU (%)
Align. OnlyNone27.950.4
Lreg\mathcal{L}_ {\mathrm{reg}} OnlyAll anchors w/ vqL\boldsymbol{v}_q^L27.950.6
Lstatic\mathcal{L}_ {\mathrm{static}} Only✓None28.050.8
Static-Only Ego-Motion Reg.✓Static anchors w/ ego motion (v=0\boldsymbol{v}=\boldsymbol{0})28.051.3
Full KIO-World✓All anchors w/ shared vqL\boldsymbol{v}_q^L (Eq. 5)28.251.8

Key Takeaways:

  1. Why both Lstatic\mathcal{L}_ {\mathrm{static}} and Lreg\mathcal{L}_ {\mathrm{reg}} are required: Lstatic\mathcal{L}_ {\mathrm{static}} calibrates the motion state at h=0h=0 (28.0 / 50.8 alone, since future centers are unconstrained), while Lreg\mathcal{L}_ {\mathrm{reg}} propagates this state to future centers (27.9 / 50.6 alone, since static anchors otherwise absorb center drift into fake velocities). Used together, they reach 28.2 / 51.8 (+0.2 mIoU and +1.0 IoU over Lstatic\mathcal{L}_ {\mathrm{static}} alone).
  2. Static stabilization vs. dynamic coupling: Regularizing only static anchors with ego motion raises IoU from 50.8 to 51.3 (+0.5 IoU, as static structure dominates voxel volume), while mIoU stays at 28.0 because movable classes have no cross-horizon center constraint. Extending Lreg\mathcal{L}_ {\mathrm{reg}} to all anchors via shared velocity vqL\boldsymbol{v}_q^L adds another +0.2 mIoU and +0.5 IoU (28.0 / 51.3 to 28.2 / 51.8), providing half of the IoU gain over Lstatic\mathcal{L}_ {\mathrm{static}} alone and the entire mIoU gain by regularizing moving objects and feeding future supervision back into the shared state.
  3. Agreement with Table 10: In Table 10, Direction-Specific (27.8 / 50.7) already applies Lstatic\mathcal{L}_ {\mathrm{static}} to both past and future states and regularizes future centers, yet it still loses 0.4 mIoU and 1.1 IoU when historical alignment and future regularization do not share kq\boldsymbol{k}_q.

Q2 & W2. Complete HRPR vs. Parent Inheritance + Child Residuals Without Pose Head

Under the same point schedule (1,8,16,32,64,128)(1, 8, 16, 32, 64, 128) and controlled setting of Table 3 (Rows 1 and 2, without kinematic alignment or regularization), we evaluated parent inheritance and child residuals without the pose head (piℓ+1=pκℓ(i)ℓ+Δriℓ\boldsymbol{p}_ {i}^{\ell+1} = \boldsymbol{p}_ {\kappa_ {\ell}(i)}^{\ell} + \Delta\boldsymbol{r}_ {i}^{\ell}, added in Appendix D.4, Table 13):

Point Refinement VariantmIoU (%)IoU (%)
Centroid + Child Offset (cℓ+Δriℓ\boldsymbol{c}^\ell + \Delta\boldsymbol{r}_i^\ell, Table 3 Row 1)25.448.7
Parent Inheritance + Child Residual (w/o Pose Head, pκℓ(i)ℓ+Δriℓ\boldsymbol{p}_ {\kappa_\ell(i)}^\ell + \Delta\boldsymbol{r}_i^\ell)26.149.1
Complete HRPR (Parent + Pose Head + Child Residual, Eq. 7, Table 3 Row 2)26.849.6
  • Parent inheritance without the pose head improves over centroid offsets from 25.4 / 48.7 to 26.1 / 49.1 (+0.7 mIoU / +0.4 IoU).
  • Adding the zero-initialized anchor-level pose head (Δtℓ,Δψℓ,sℓ)(\Delta\boldsymbol{t}^\ell, \Delta\psi^\ell, \boldsymbol{s}^\ell) brings an additional +0.7 mIoU and +0.5 IoU (26.1 / 49.1 to 26.8 / 49.6), accounting for half of HRPR's total gain so that local child residuals Δriℓ\Delta\boldsymbol{r}_i^\ell do not have to absorb whole-anchor translation, yaw rotation, and scaling across horizons.

We hope these two controlled comparisons address both concerns.

Before the discussion concludes, we would welcome your thoughts on our response and revised manuscript. In particular, please let us know whether any concerns still require clarification. We remain available to respond during the remaining discussion period. Thank you for your time.

Reviewer_iBu4OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes KIO-World, a sparse occupancy world model conditioned on historical camera observations and a future ego trajectory. A shared velocity and yaw-rate state guides historical feature sampling and regularizes future anchor centers. Hierarchical residual point refinement preserves parent geometry while separating anchor-level transformations from local shape updates. Experiments on Occ3D-nuScenes compare against matched SparseWorld-TC variants and include voxel-level and ray-level metrics, component ablations, and test-time extension from three-second training horizons to eight-second forecasts.

Critical strengths
  1. The shared motion state connects historical alignment with future prediction in a useful way. It provides motion-compensated sampling locations and a common reference for future anchor centers, while direct occupancy decoding accommodates changes beyond the simple kinematic model. The comparisons with separate motion states and hard rollout support these design choices.

  2. The accuracy gains are consistent across model sizes and evaluation metrics. The Large model improves average mIoU from 26.4 to 29.4 and geometric IoU from 49.2 to 52.3 over the matched SparseWorld-TC baseline. The Small model also improves average RayIoU from 28.5 to 31.1, with gains on movable classes including pedestrians and bicycles. HRPR additionally improves both OPUS-T and SparseWorld-TC when inserted into those models, supporting its usefulness beyond the complete architecture.

Critical weaknesses

The additional benefit of future-center regularization remains unclear. Table 10 supports sharing the motion state, but Table 11 shows only a 0.2-point IoU gain from LregL_{\mathrm{reg}} alone, with the larger improvement appearing after LstaticL_{\mathrm{static}} is added. The authors acknowledge this dependency. Since the static prior can also improve historical sampling by suppressing spurious background motion, a variant using LstaticL_{\mathrm{static}} without LregL_{\mathrm{reg}} would clarify what future-center regularization adds. This limits the attribution of the regularization gain, although the full-model improvements are supported.

Questions for authors
  1. What is the performance with LstaticL_{\mathrm{static}} enabled and LregL_{\mathrm{reg}} disabled? In the direction-specific variant of Table 10, is the static prior applied to the alignment state, the future-regularization state, or both?

  2. For Table 2, please identify the SparseWorld-TC checkpoint and its supervised training horizons, and specify which horizon queries are decoded jointly during test-time extension.

3 comments
AuthorsOpenReview ↗

Response to Reviewer iBu4

We thank Reviewer iBu4 for recognizing our shared kinematic state, the consistent gains across model sizes and metrics, and the transferability of HRPR. All updates below are marked in blue in the revised manuscript.

Q1 & W1. Lstatic\mathcal{L}_ {\mathrm{static}} Enabled with Lreg\mathcal{L}_ {\mathrm{reg}} Disabled, and Static Prior in Table 10

1. New Lstatic\mathcal{L}_ {\mathrm{static}}-only ablation in Table 11: We trained KIO-World-Small for 70 epochs under the same setup (with HRPR, historical alignment, and Lfeas\mathcal{L}_ {\mathrm{feas}} fixed) with Lstatic\mathcal{L}_ {\mathrm{static}} enabled and Lreg\mathcal{L}_ {\mathrm{reg}} disabled (Appendix D.3, Table 11):

Lreg\mathcal{L}_ {\mathrm{reg}}Lstatic\mathcal{L}_ {\mathrm{static}}mIoU (%)IoU (%)
27.950.4
✓27.950.6
✓28.050.8
✓✓28.251.8

This comparison shows why both losses are needed together:

  • Lstatic\mathcal{L}_ {\mathrm{static}} calibrates the state at h=0h=0, while Lreg\mathcal{L}_ {\mathrm{reg}} propagates it to future centers (h≥1h \ge 1).
  • Lreg\mathcal{L}_ {\mathrm{reg}} only: Static anchors absorb future center drift into fake velocities vqL\boldsymbol{v}_ q^L instead of correcting cq,hL\boldsymbol{c}_ {q,h}^L.
  • Lstatic\mathcal{L}_ {\mathrm{static}} only: Suppressing background motion slightly helps historical sampling, but the calibrated state is never passed to future centers.
  • Both Lstatic\mathcal{L}_ {\mathrm{static}} and Lreg\mathcal{L}_ {\mathrm{reg}}: Once Lstatic\mathcal{L}_ {\mathrm{static}} keeps static velocities near zero (while leaving movable anchors free), Lreg\mathcal{L}_ {\mathrm{reg}} enforces cross-horizon center consistency and brings the main gain (+0.2 mIoU and +1.0 IoU over Lstatic\mathcal{L}_ {\mathrm{static}} alone).

2. Application of Lstatic\mathcal{L}_ {\mathrm{static}} in Direction-Specific (Table 10): In Direction-Specific (27.8 mIoU / 50.7 IoU), Lstatic\mathcal{L}_ {\mathrm{static}} is applied to both the alignment state kqpast\boldsymbol{k}_ {q}^{\mathrm{past}} and the future-regularization state kqfut\boldsymbol{k}_ {q}^{\mathrm{fut}} (now stated in Appendix D.3).

Q2. SparseWorld-TC Provenance, Training Horizons, and Joint Decoding in Table 2

  1. Provenance and supervised training horizons of SparseWorld-TC in Table 2:

    Because Du et al. (2026) did not release a SparseWorld-TC-Large checkpoint, the SparseWorld-TC row in Table 2 (22.3 mIoU / 45.4 IoU) directly quotes the official SparseWorld-TC-Large numbers reported in Table 2 of Du et al. (2026). Both KIO-World-Large and SparseWorld-TC-Large are trained under the same horizon: both models are trained strictly on 0-3s, and directly extended to 0-8s at test time without retraining (clarified in Section 4.2 and Table 2).

  2. Which horizon queries are decoded jointly during test-time extension:

    At test time, we expand the target horizons from 0-3s to 99 timestamps (0-8s). All 99 horizon query sets are decoded jointly in one forward pass. Each horizon receives its ego pose Th→0\boldsymbol{T}_ {h\rightarrow 0} and timestamp embedding e(th)\boldsymbol{e}(t_h), shares the canonical state kqℓ\boldsymbol{k}_q^\ell from h=0h=0, exchanges features across all 99 tokens in per-query temporal self-attention, and decodes occupancy in parallel without retraining.

Thank you for the detailed response and for running the additional ablation. Your response has addressed all of my concerns, and I will maintain my positive score.

Thank you for the follow-up and for considering our clarifications and additional ablation. We sincerely appreciate your constructive comments.

Reviewer_iKhUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

extends sparse, trajectory-conditioned occupancy forecasting with a velocity and yaw-rate state shared across the prediction horizons of each query anchor. This state compensates historical image-feature sampling and provides soft references for future anchor centers, while occupancy at each horizon remains directly decoded. A hierarchical residual point refinement module is then proposed to preserve parent-point geometry and separate anchor-level pose updates from local shape residuals. The paper reports test-time extension to 8 seconds with bi-directional temporal reasoning.

Critical strengths

The paper presents a coherent connection between historical perception and future forecasting. Sharing a motion state between historical sampling and future center constraint is a sensible inductive bias, and validated through ablation studies.

Critical weaknesses

The major issue is the limited methodology novelty and insufficient positioning against related designs. Velocity-based compensation of historical sampling locations is presented in sparseBEV, and parent-to-child point generation has precedents such as SnowflakeNet. These works do not reproduce the full design of the proposed method, but narrow its contributions to the particular coupling of motion-aware sampling with future regularization and the additional anchor-level transformation in HRPR.

Meanwhile, the contribution of future center regularization remains confounded with the static-scene prior. In Tab.11, adding the static-scene prior produces 28.2/51.8. The missing static-prior-only setting prevents determining how much of the gain requires future center regularization, rather than improved stabilization of static anchors during historical alignment.

Questions for authors

The issues are addressed in the Weakness section.

2 comments
AuthorsOpenReview ↗

Response to Reviewer iKhU

We sincerely thank Reviewer iKhU for the positive rating and helpful comments. We have marked all corresponding updates in blue in the revised manuscript.

W1. Positioning Relative to SparseBEV and SnowflakeNet

In the revised manuscript, we have cited SnowflakeNet (Xiang et al., ICCV 2021) and expanded the discussion of SparseBEV (Liu et al., ICCV 2023) in Sec. 2, Sec. 3.3, and Sec. 3.4:

  1. Comparison with SparseBEV: We agree with the reviewer that velocity-based compensation of historical sampling locations is also used in SparseBEV, and we clarify two main differences in our method. First, in terms of supervision, SparseBEV addresses single-frame 3D detection and supervises instance queries directly with ground-truth 3D boxes and velocities. In contrast, camera-only 4D occupancy forecasting provides neither box nor velocity labels, so kinematic states of sparse anchors (covering static structure and moving agents) must be learned purely from occupancy supervision and physical priors. Second, in terms of bidirectional cross-horizon coupling, SparseBEV only warps t0t_0 points backward with planar constant velocity. KIO-World instead estimates a canonical CV-Yaw state kqℓ=(vqℓ,ωqℓ)\boldsymbol{k}_ {q}^{\ell}=(\boldsymbol{v}_ q^{\ell}, \omega_ {q}^{\ell}) at h=0h=0 (including 3D translation and yaw rotation, outperforming the constant-velocity model in Table 5) and shares it bidirectionally across all H+1H+1 horizons, backward from Th→0\boldsymbol{T}_{h\rightarrow 0} to historical images and forward to regularize decoded future centers. Removing directional or cross-horizon sharing drops accuracy from 28.2 / 51.8 to 27.8 / 50.7 and 27.2 / 49.9 in Table 10.

  2. Comparison with SnowflakeNet: We also agree that parent-to-child point generation has precedents such as SnowflakeNet, and we highlight two differences in HRPR. First, in terms of task dynamics, SnowflakeNet completes single objects in a canonical frame using point-wise child displacements alone, whereas 4D driving anchors undergo coherent anchor-level pose changes (translation, yaw rotation, and scaling) across layers and horizons alongside per-layer image sampling. Second, in terms of refinement decomposition, HRPR decouples refinement around inherited parents into a zero-initialized anchor-level pose update (Δtℓ,Rz(Δψℓ),sℓ)(\Delta\boldsymbol{t}^\ell, \boldsymbol{R}_z(\Delta\psi^\ell), \boldsymbol{s}^\ell) and point-wise shape residuals Δriℓ\Delta\boldsymbol{r}_i^\ell (Eq. 7):

Point Refinement Variant (Table 3 Setting)mIoU (%)IoU (%)
Centroid + Child Offset (Table 3, Row 1)25.448.7
Parent Inheritance + Child Residual (w/o Pose Head)26.149.1
Complete HRPR (Table 3, Row 2)26.849.6

In this ablation (Appendix D.4, Table 13), parent-plus-residual splitting improves over centroid offsets by +0.7 mIoU and +0.4 IoU (25.4 / 48.7 to 26.1 / 49.1). Adding the anchor-level pose head brings a further +0.7 mIoU and +0.5 IoU (26.1 / 49.1 to 26.8 / 49.6), as child residuals no longer absorb whole-anchor pose shifts across horizons.

W2. Isolating Lreg\mathcal{L}_ {\mathrm{reg}} from Lstatic\mathcal{L}_ {\mathrm{static}} in Table 11

We trained KIO-World-Small (70 epochs, with HRPR, historical alignment, and Lfeas\mathcal{L}_ {\mathrm{feas}} fixed) with Lstatic\mathcal{L}_ {\mathrm{static}} enabled and Lreg\mathcal{L}_ {\mathrm{reg}} disabled (Appendix D.3, Table 11):

Lreg\mathcal{L}_{\mathrm{reg}}Lstatic\mathcal{L}_{\mathrm{static}}mIoU (%)IoU (%)
27.950.4
✓27.950.6
✓28.050.8
✓✓28.251.8

This comparison shows why both losses are needed together:

  • Lstatic\mathcal{L}_ {\mathrm{static}} calibrates the state at h=0h=0, while Lreg\mathcal{L}_ {\mathrm{reg}} propagates it to future centers (h≥1h \ge 1).
  • Lreg\mathcal{L}_ {\mathrm{reg}} only: Static anchors absorb future center drift into fake velocities vqL\boldsymbol{v}_ q^L instead of correcting cq,hL\boldsymbol{c}_ {q,h}^L.
  • Lstatic\mathcal{L}_ {\mathrm{static}} only: Suppressing background motion slightly helps historical sampling, but the calibrated state is never passed to future centers.
  • Both Lstatic\mathcal{L}_ {\mathrm{static}} and Lreg\mathcal{L}_ {\mathrm{reg}}: Once Lstatic\mathcal{L}_ {\mathrm{static}} keeps static velocities near zero (while leaving movable anchors free), Lreg\mathcal{L}_ {\mathrm{reg}} enforces cross-horizon center consistency and brings the main gain (+0.2 mIoU and +1.0 IoU over Lstatic\mathcal{L}_ {\mathrm{static}} alone).

Before the discussion concludes, we would welcome your thoughts on our response and revised manuscript. In particular, please let us know whether any concerns still require clarification. We remain available to respond during the remaining discussion period. Thank you for your time.

Comments on the paper

2 comments

Dear Area Chair and Reviewers,

Thank you again for your time and effort in reviewing our submission. As the discussion period is going to be closed, we would like to briefly summarize our new experiments and manuscript revisions (marked in blue in the updated PDF) for your convenience:

  1. Disentangling Lstatic\mathcal{L}_ {\mathrm{static}}, Static-Only Center Regularization, and Dynamic Coupling (Appendix D.3, Table 11, for Reviewers iKhU, iBu4, CB6D, ygc8):

    • Lstatic\mathcal{L}_ {\mathrm{static}} only (with Lreg\mathcal{L}_ {\mathrm{reg}} disabled) reaches 28.0 mIoU / 50.8 IoU (compared with 27.9 / 50.4 without both and 27.9 / 50.6 with Lreg\mathcal{L}_ {\mathrm{reg}} only). Combining both terms reaches 28.2 mIoU / 51.8 IoU, confirming that both losses are needed together.
    • Static-only ego-motion center regularization yields 28.0 mIoU / 51.3 IoU, whereas coupling all anchors via the shared velocity vqL\boldsymbol{v}_ q^L adds another +0.2 mIoU and +0.5 IoU.
  2. Internal Ablation of HRPR Without the Pose Head (Appendix D.4, Table 13, for Reviewers CB6D, iKhU):

    • Under the controlled setting of Table 3, inheriting parent points and predicting child residuals without the pose head improves over centroid offsets from 25.4 / 48.7 to 26.1 / 49.1 (+0.7 mIoU / +0.4 IoU). Adding the zero-initialized anchor-level pose head brings a further +0.7 mIoU and +0.5 IoU to reach 26.8 / 49.6.
  3. Methodological Positioning and Protocol Clarifications (for Reviewers iKhU, iBu4, ygc8):

    • Expanded comparisons with SparseBEV and SnowflakeNet in Secs. 2, 3.3, and 3.4.
    • Clarified baseline provenance across Tables 1, 2, 3, and 8, joint 9-horizon decoding during 0-8 s test-time extension (Sec. 4.2), current-frame behavior under unmasked temporal attention (Apps. C.2 and E.2), and full numerical configurations in Appendix F (Table 14).

We understand this is a busy period and truly appreciate your service. Please let us know if our responses and revisions have addressed your concerns, or if there are any remaining points we can further clarify during the discussion phase.

AuthorsOpenReview ↗

Final Rebuttal Summary

Dear Area Chair and Reviewers,

Thank you for your constructive feedback throughout the review process. During the discussion period, one reviewer explicitly stated, "Your response has addressed all of my concerns, and I will maintain my positive score."

Regarding the ablation questions raised across the reviews, we have provided point-by-point experimental responses, isolating Lstatic\mathcal{L}_ {\mathrm{static}} and Lreg\mathcal{L}_ {\mathrm{reg}}, comparing full dynamic coupling against static-only ego-motion regularization (Appendix D.3, Table 11), and evaluating HRPR without the anchor-level pose head (Appendix D.4, Table 13). We also clarified our methodological distinctions from SparseBEV and SnowflakeNet (Secs. 2, 3.3, and 3.4), the baseline provenance and evaluation protocols across Tables 1, 2, 3, 8, and 10 (including joint 9-horizon decoding during 0 to 8 s test-time extension), the current-frame behavior under unmasked temporal attention (Apps. C.2 and E.2), and the full hyperparameter settings (Appendix F, Table 14). Detailed responses and supporting results are provided in the individual discussion threads and marked in blue in the revised manuscript.

We sincerely thank the Area Chair and all reviewers again for their time and valuable efforts.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.