acceptodds
Under review as a conference paper at ICLR 2027

LighTROcc: Lightweight 4D Occupancy Forecasting via Instance-Centric 3D Gaussians

Abstract

Forecasting future 3D occupancy from surround-view cameras is essential for autonomous driving, yet existing approaches rely on dense voxel or bird's-eye-view representations whose cost grows rapidly with spatial resolution and prediction horizon. Because these representations do not explicitly maintain object identities, they also struggle to preserve instance consistency over time. We present LighTROcc, a lightweight instance-centric framework that represents movable objects with a compact set of learned queries and predicts present and future occupancy in a single forward pass. LighTROcc localizes each query through attention-guided forward lifting, combining image-space cross-attention, query-specific depth, and camera geometry to estimate its 3D center. Each instance is modeled as a mixture of anisotropic 3D Gaussians and propagated across future steps using predicted displacements, producing continuous, temporally consistent occupancy forecasts. Experiments on nuScenes and supplemented nuScenes-Occupancy show that LighTROcc outperforms the evaluated dense and instance-wise baselines in instance-level forecasting accuracy while maintaining strong voxel-level occupancy quality. Across different model configurations, LighTROcc achieves a favorable balance between forecasting accuracy and computational efficiency, demonstrating the potential of compact instance-centric modeling for camera-based 4D occupancy forecasting.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Autonomous Driving, Occupancy Prediction, Occupancy Forecasting

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DQpNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a camera-based framework for forecasting the occupancy of movable objects using instance-centric Gaussian representations. A compact set of object queries extracts multi-view temporal features, and attention-guided lifting estimates their 3D centers. Each query predicts a Gaussian mixture representing an object’s current geometry, together with future displacements. Future occupancy is obtained by translating this mixture while retaining its component offsets, covariances, and occupancy weights. Experiments on nuScenes report favorable occupancy accuracy, instance-level AP, and inference efficiency under the authors’ evaluation protocol.

Critical strengths
  1. The instance-centric formulation provides an efficient alternative to dense voxel forecasting. The reported accuracy efficiency trade-off is promising, and the lower-resolution variant helps demonstrate that the gains are not solely attributable to higher image resolution.

  2. The paper includes useful supporting analyses. The lifting and query-count ablations examine important design choices, while the constant-velocity comparison demonstrates a benefit from learned trajectory prediction. The annotation audit, original sparse-label evaluation, and supplementary visualizations also improve the transparency of the experimental study.

Critical weaknesses

W1. The contribution beyond existing Gaussian instance representations is not sufficiently isolated. GUIDE already represents object instances using Gaussian mixtures [1]. LighTROcc adds attention-guided localization and future displacement prediction, and the localization ablation supports the former. However, a particularly informative comparison would extend GUIDE with the same trajectory head and observed-motion inputs under a matched training and evaluation setup. The constant-velocity ablation establishes that the learned head improves this implementation, but does not determine how much of the overall gain requires the proposed formulation. Related work should also position the method more clearly against S2GO’s temporal Gaussian representation [2], while acknowledging that its semantic occupancy setting differs from instance-level GMO forecasting.

W2. The forecasting model imposes a restrictive geometry assumption whose consequences are insufficiently characterized. Future prediction translates each Gaussian mixture without updating its relative component positions, covariances, or occupancy weights. This cannot explicitly model future object rotation or articulated shape changes. The learned trajectory head clearly improves over constant velocity, but aggregate results do not reveal how the method behaves during vehicle turns, pedestrian articulation, or other departures from translation. A motion-dependent breakdown and a comparison with an otherwise matched model that predicts orientation would clarify when the lightweight approximation is adequate and when it fails.

W3. The temporal consistency results do not independently establish accurate temporal prediction. The reported TSC metric compares consecutive predicted shapes after centroid subtraction and PCA alignment. Since LighTROcc retains the same Gaussian geometry across future steps, high shape consistency is largely encouraged by construction; an inaccurate but unchanged shape can also receive a high score. TSC therefore supports geometric stability, but does not independently demonstrate accurate future pose or identity association. Framewise AP provides complementary evidence, although comparisons with dense methods also reflect their flow-based instance grouping. A ground-truth-referenced trajectory or instance-tube evaluation would better distinguish stable representation, accurate motion, and correct temporal association.

Questions for authors
  1. How does LighTROcc compare with GUIDE augmented by the same learned trajectory head, motion inputs, and training losses?

  2. Do the main performance rankings persist under the unmodified original-label evaluation and alternative choices in the annotation completion procedure?

  3. How does performance vary between approximately translating objects and objects undergoing substantial rotation or articulation? What is the cost and benefit of predicting future orientation?

3 comments

Thank you for your careful review and constructive suggestions. To address your concerns, we have conducted additional experiments and revised the manuscript accordingly. We provide detailed responses below, and please also refer to our newly uploaded PDF for further results and analyses.

W1/Q1. The contribution beyond existing Gaussian instance representations is not sufficiently isolated. 

To address your concern, we have added a comparison with GUIDE equipped with an adapted trajectory head in Table 12. Please note that we fine-tuned GUIDE rather than training it from scratch, as its official training configuration is incomplete. We additionally evaluate extrapolation using the velocity and heading predicted by GUIDE at the current frame.

Adding the trajectory head clearly improves forecasting performance over future horizons. However, the remaining performance gap appears to be influenced by differences in object shape accuracy and spurious or missed instance predictions, rather than trajectory prediction alone.

We have also strengthened the Related Work section to better clarify the distinctions between LighTROcc and S2GO.

W2/Q3. The forecasting model imposes a restrictive geometry assumption whose consequences are insufficiently characterized.

To address your concern, we provide a rotation-stratified evaluation in Table 13, evaluating turning objects across different rotation ranges. LighTROcc consistently achieves the highest accuracy across these ranges. However, we note that shape reconstruction errors in competing methods may outweigh rotation-induced errors, making it difficult to isolate the effect of orientation alone.

Following your suggestion, we also retrained an otherwise matched variant with an additional yaw prediction output in the trajectory head. The results below compare translation-only forecasting with yaw-aware forecasting on rotation-specific subsets.

Yaw change0.5s1.0s1.5s2.0s
<5°35.17 → 35.1731.98 → 31.9829.45 → 29.4527.27 → 27.27
5–15°32.32 → 32.4122.17 → 22.3214.41 → 14.529.50 → 9.59
≥15°24.19 → 24.5211.93 → 12.476.09 → 6.473.59 → 3.83

Explicit yaw prediction provides modest improvements, particularly for larger rotations, but has negligible impact on aggregate performance. One key limitation is the scarcity of rotating objects, accounting for only 2.04% (5–15°) and 2.16% (≥15°) of instances. We suspect this sparsity is further amplified by the overlapping 7-frame window protocol adopted by existing forecasting methods. This may also partly explain their use of inflated GMO representations with AABBs rather than OBBs.

To further examine the articulated motion, we additionally report GT-matched pedestrian instance IoU across future horizons.

Method0.5s1.0s1.5s2.0s
OccProphet0.1950.0920.0700.063
LighTROcc0.2250.0890.0740.062

Both methods exhibit decreasing IoU over longer horizons. Since all baselines operate at a voxel resolution of 0.2 m, accurately forecasting fine-grained limb articulation remains challenging. Moreover, small-volume objects such as pedestrians are particularly sensitive to minor localization and orientation errors, naturally resulting in substantially lower IoU than larger objects such as vehicles [1, 2].

[1] Caesar, Holger, et al. "nuscenes: A multimodal dataset for autonomous driving." 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2020.

[2] Cheng, Gong, et al. "Towards large-scale small object detection: Survey and benchmarks." IEEE transactions on pattern analysis and machine intelligence 45.11 (2023): 13467-13488.

W3. The temporal consistency results do not independently establish accurate temporal prediction. 

We acknowledge your point that TSC evaluates shape stability in object-local coordinates but does not directly measure future localization accuracy or consistency with GT identities. To address this issue, we introduce a GT-linked occupancy evaluation in Table 11. Predicted instances are associated with GT instances through one-to-one oracle matching at the current frame, and the resulting correspondences are kept fixed across all future horizons. Please note that oracle matching is performed only at the initial frame.

LighTROcc consistently outperforms all baselines under this fixed-assignment evaluation, providing additional evidence that our method accurately forecasts future object trajectories beyond merely preserving shape consistency. Furthermore, during training, LighTROcc establishes a single Hungarian assignment per instance and supervises its entire temporal trajectory under that assignment. This avoids frame-wise reassociation and encourages consistent instance identities throughout forecasting.

Q2. Performance Robustness under Original GT and Annotation Completion

Table 7 compares our method with existing baselines using the original sparse annotations, where the method rankings are not fully preserved. However, our synthetic probe experiments and distance- and visibility-based analyses suggest that this degradation cannot be attributed solely to detection errors. Sparse annotations can favor methods that suppress predictions for challenging objects or predict only partially observed regions, potentially yielding higher scores. Our supplemented annotations aim to address this bias by more accurately rewarding correct occupancy and penalizing missed regions. Thus, changes in method rankings can naturally arise under a more complete evaluation protocol.

Additional evaluations using two alternative 3D asset sets (A1 and A2) from Objaverse are presented below.

MethodMain IoUc\mathrm{IoU}_cMain IoUf\mathrm{IoU}_fMain APc\mathrm{AP}_cMain APf\mathrm{AP}_fA1 IoUc\mathrm{IoU}_cA1 IoUf\mathrm{IoU}_fA1 APc\mathrm{AP}_cA1 APf\mathrm{AP}_fA2 IoUc\mathrm{IoU}_cA2 IoUf\mathrm{IoU}_fA2 APc\mathrm{AP}_cA2 APf\mathrm{AP}_f
Cam4DOcc13.4211.8010.718.0213.3811.7410.667.9713.3311.8410.647.95
EfficientOCF8.678.5914.2514.138.708.6214.2014.068.628.5414.1814.04
OccProphet17.5714.2320.147.2517.6414.3320.377.3217.5014.1520.047.31
SparseOcc18.4213.6720.0513.3618.3913.7119.9913.4018.3413.5820.1113.28
GUIDE15.2112.4034.7324.5115.1812.3634.8324.4415.1312.3334.5924.39
LighTROcc19.7615.3042.6233.3419.9115.3842.5433.2819.6615.2342.3133.12

The results show minimal changes in performance and method rankings across the alternative asset sets. This suggests that our annotation completion remains robust when using reasonably representative 3D assets, rather than highly unrealistic geometries.

Dear Reviewer DQpN,

Thank you once again for your valuable review and constructive feedback. As the discussion period is approaching its end on November 18 (AoE), we would like to kindly ask whether our rebuttal has adequately addressed your concerns.

If you have any remaining questions or comments, we would be grateful to hear your thoughts.

Best regards,

The Authors

Reviewer_Qf2POpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LighTROcc forecasts current and future GMO occupancy from three frames of surround-view images. Its main contribution is attention-guided 3D localization within a compact instance-query framework, with objects represented by Gaussians and propagated through predicted displacements.

Experiments on nuScenes with supplemented occupancy labels report a favorable accuracy–efficiency trade-off. Ablations assess localization, query count, and motion prediction, with additional analyses of annotation quality and temporal shape consistency.

Critical strengths
  1. Clear and well-motivated design. Compact instance-centric modeling is well motivated by the computational cost of dense spatiotemporal representations and the need to preserve object-level consistency during forecasting.
  2. Useful ablation evidence. The experiments support the proposed localization and motion prediction designs: attention-guided soft lifting improves over direct 3D regression, while the learned trajectory head consistently outperforms constant-velocity extrapolation with current instance predictions and Gaussian geometry held fixed.
Critical weaknesses

Limited coverage of the driving environment. The framework models and evaluates only GMOs (Secs. 3.1 and 4.1), leaving static occupancy, such as buildings, vegetation, and road infrastructure, unaddressed. This limits its applicability to the planning and traversability motivation: the absence of a GMO does not establish that a region is free. Although this scope is explicitly stated, the paper does not demonstrate how the compact instance-centric representation could support joint GMO/GSO prediction, or whether its accuracy–efficiency advantages would persist after incorporating static-scene perception.

Questions for authors

Q1. Efficiency measurement and runtime breakdown. Do the latency and FPS in Table 2 include rasterization for the current frame and all four future steps, as well as post-processing? Please provide a module-level latency breakdown covering image encoding, query decoding, prediction heads, rasterization, and post-processing, and clarify historical-feature caching and timing consistency across baselines.

For the query-scaling results in Table 4, is confidence filtering ((s_q>0.9), Sec. 3.5) performed before rasterization? Please report the average number of retained queries and corresponding Gaussians, together with rasterization latency, for the 100-, 200-, and 900-query configurations. Although Figure 6 analyzes selected-query composition, its relationship to runtime remains unclear. These measurements would help distinguish decoder overhead from rasterization cost and identify the sources of the efficiency gains.

Q2. Effect of prediction scope on efficiency. LighTROcc predicts only GMOs, whereas baselines such as Cam4DOcc and OccProphet also support static-scene occupancy prediction. Were these baselines configured for GMO-only prediction in the reported comparisons, or did they retain joint GMO/GSO prediction? Please clarify their training and output configurations. Comparing latency and peak GPU memory under matched GMO-only settings would help determine whether the efficiency gains arise from the proposed architecture or, potentially, from a narrower prediction scope.

Q3. Applicability to real-world deployment. Does restricting prediction to GMOs limit the framework’s practical applicability? Safe planning also requires modeling static structures and free space, since the absence of a GMO does not imply that a region is traversable. Would deployment require an additional model, prediction branch, or map to represent the static background? Please discuss how these components could be integrated and their implications for system complexity, computational cost, and end-to-end latency.

Q4. Translation-only forecasting. The Gaussian mixtures are translated into the future without changing their internal offsets, covariances, or weights (Sec. 3.4). How much does this affect accuracy for turning vehicles and pedestrians whose orientation or shape changes? Results on these subsets, or a comparison with future rotation prediction, would help quantify this limitation.

Q5. Annotation protocol. Were all baselines trained with the same supplemented labels? The sparse-label experiment in Appendix A.4 still ignores false positives inside ground-truth boxes. Could the authors also report results without this modification? It would also be useful to check whether using different 3D assets changes the results or method rankings.

7 comments

Thank you for your thoughtful feedback and valuable suggestions. To address your concerns, we have conducted additional experiments and provided further analyses and clarifications below. Please also refer to our newly uploaded PDF for more details.

W1/Q3. Limited representation of the full driving environment.

For practical deployment, LighTROcc can be integrated with semantic HD maps at negligible overhead when pre-existing maps are available. However, since HD maps primarily represent 2D geometry, a separate GSO prediction branch would be needed for complete 3D occupancy modeling.

Although Cam4DOcc and OccProphet include GSO prediction, they lack semantic classification, making it difficult to distinguish traversable roads from sidewalks using geometry alone. A practical solution is to incorporate a lightweight semantic occupancy branch, such as FlashOcc. Since LighTROcc predicts all future frames in a single forward pass, GSO inference is needed only once. We measured the FlashOcc BEV encoder and Channel-to-Height module at approximately 5.5 ms on an RTX PRO 6000 GPU, keeping the combined latency substantially below existing 4D forecasting methods.

We emphasize that our primary focus is GMO-only forecasting, while the efficiency of LighTROcc leaves sufficient computational headroom for GSO integration in practical deployment. We have added this discussion to Appendix F to clarify the applicability of our framework.

Q1. Inference efficiency measurements and runtime breakdown.

We clarify that the original inference measurements for both Gaussian-based methods, LighTROcc and GUIDE, excluded rasterization. Unlike dense voxel-wise methods, which require explicit voxel grids at each timestep to represent metric-space geometry and spatial extent, Gaussian-based methods can directly estimate 3D shapes and spatial distances in continuous space without rasterizing the full voxel grid. Thus, our original measurements reflected a computationally efficient deployment setting. Nevertheless, we agree that end-to-end measurements with identical output formats provide a clearer comparison. Accordingly, we have updated all latency measurements in the main paper to include rasterization.

As you suggested, we provide a detailed runtime breakdown, including rasterization and post-processing, in Appendix E Table 14. These components contribute relatively little to the overall latency, which is dominated by feature encoding, decoding, and prediction. The revised measurements cover the entire pipeline from RGB inputs to grid-format occupancy outputs for all baselines, using the same GPU, batch size of 1, FP32 precision, and CUDA synchronization. Importantly, LighTROcc does not cache historical features and encodes all 18 images (3 timestamps ×\times 6 views) at each inference.

LighTROcc applies confidence filtering before rasterization. The retained query counts are reported in Figure 6, with an additional average of 10.31 for the 100-query setting. As described in Appendix B.1, each retained query contains 48 Gaussian ellipsoids. Consequently, rasterization latency depends on the number of retained queries, as summarized below.

# Queries# Retained QueriesRasterization (ms)
10010.310.26
20011.900.27
90031.820.47

Q2. Effect of prediction scope on efficiency.

We confirm that both Cam4DOcc and OccProphet were evaluated using their GMO-only configurations, and all latency measurements were conducted under the same GMO-only setting. All baselines were evaluated using their officially released implementations.

The GMO-only inference latencies are reported in Table 2 of the main paper. We additionally measured peak GPU memory usage, as summarized below.

MethodInput Res.Peak VRAM (GB)
Cam4DOcc1600 × 8964.852
OccProphet800 × 4483.218
LighTROcc1600 × 8961.593

Notably, LighTROcc achieves the lowest memory usage despite using a higher input resolution than OccProphet. OccProphet reaches its peak memory usage during U-Net processing and feature fusion in its Observer and Refiner modules, while Cam4DOcc peaks during LSS view transformation. In contrast, LighTROcc peaks during image encoding and transformer processing. Its lower memory footprint highlights the benefit of compact instance queries, which substantially reduce the computational and memory overhead associated with attention-based processing.

Q4. Translation-only forecasting.

To address your concern, we provide a rotation-stratified evaluation in Table 13, using oracle-based instance matching for all methods to evaluate objects within each rotation range. LighTROcc consistently achieves the highest accuracy across both small and large rotations. However, these results also depend on shape reconstruction quality, as shape errors in the baselines may outweigh errors caused by unmodeled rotation.

We additionally extend the trajectory head to predict object yaw and retrain the model. The following results show the accuracy before and after adding yaw prediction, evaluated only on the corresponding rotation subsets. The improvements are nearly negligible in the overall benchmark.

Yaw change0.5s1.0s1.5s2.0s
<5°35.17 → 35.1731.98 → 31.9829.45 → 29.4527.27 → 27.27
5–15°32.32 → 32.4122.17 → 22.3214.41 → 14.529.50 → 9.59
≥15°24.19 → 24.5211.93 → 12.476.09 → 6.473.59 → 3.83

Notably, objects with substantial rotation are extremely rare (2.04% for 5–15° and 2.16% for ≥15°), providing limited supervision for rotation prediction. This sparsity is likely exacerbated by the overlapping 7-frame window protocol adopted by prior forecasting methods, which may also explain their use of inflated axis-aligned bounding boxes (AABBs) rather than oriented bounding boxes (OBBs) for GMO evaluation.

We further evaluate GT-matched pedestrian instance IoU to examine the impact of articulation.

Method0.5s1.0s1.5s2.0s
OccProphet0.1950.0920.0700.063
LighTROcc0.2250.0890.0740.062

Both methods exhibit decreasing IoU over longer horizons. Since all baselines operate at a voxel resolution of 0.2 m, accurately forecasting fine-grained limb articulation remains challenging. Moreover, small-volume objects such as pedestrians are particularly sensitive to minor localization and orientation errors, naturally resulting in substantially lower IoU than larger objects such as vehicles [1, 2].

[1] Caesar, Holger, et al. "nuscenes: A multimodal dataset for autonomous driving." 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2020.

[2] Cheng, Gong, et al. "Towards large-scale small object detection: Survey and benchmarks." IEEE transactions on pattern analysis and machine intelligence 45.11 (2023): 13467-13488.

Q5. Annotation protocol.

We evaluated all baselines using their officially released implementations and checkpoints wherever available. However, GUIDE could not be retrained due to its incomplete training code and configuration, and its released checkpoint was trained on sparse annotations. To ensure a fair comparison, we consistently used the original sparse annotations for training all methods. For EfficientOCF, we followed its official training protocol, using bounding-box supervision for BEV prediction and sparse-label supervision for height prediction.

We report results on unmodified original annotations in Appendix A.4. Additional evaluations using two alternative 3D asset sets (A1 and A2) from Objaverse are presented below.

MethodMain IoUc\mathrm{IoU}_cMain IoUf\mathrm{IoU}_fMain APc\mathrm{AP}_cMain APf\mathrm{AP}_fA1 IoUc\mathrm{IoU}_cA1 IoUf\mathrm{IoU}_fA1 APc\mathrm{AP}_cA1 APf\mathrm{AP}_fA2 IoUc\mathrm{IoU}_cA2 IoUf\mathrm{IoU}_fA2 APc\mathrm{AP}_cA2 APf\mathrm{AP}_f
Cam4DOcc13.4211.8010.718.0213.3811.7410.667.9713.3311.8410.647.95
EfficientOCF8.678.5914.2514.138.708.6214.2014.068.628.5414.1814.04
OccProphet17.5714.2320.147.2517.6414.3320.377.3217.5014.1520.047.31
SparseOcc18.4213.6720.0513.3618.3913.7119.9913.4018.3413.5820.1113.28
GUIDE15.2112.4034.7324.5115.1812.3634.8324.4415.1312.3334.5924.39
LighTROcc19.7615.3042.6233.3419.9115.3842.5433.2819.6615.2342.3133.12

The results show minimal changes in performance and method rankings across the alternative asset sets. This suggests that our annotation completion remains robust when using reasonably representative 3D assets, rather than highly unrealistic geometries.

Reviewer_iVXLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

LighTROcc forecasts camera-only 4D occupancy using instance-centric 3D Gaussians. It reuses attention for localization and translates Gaussian mixtures into future frames. Results on supplemented nuScenes labels show a promising accuracy–efficiency trade-off, with a lightweight variant running at 25.10 FPS.

Critical strengths
  • The compact instance representation is well motivated for efficient occupancy forecasting.
  • Useful localization, query-count, and motion ablations are provided, including per-horizon results and a constant-velocity variant in Table 9.
  • The label supplementation addresses sparse occupancy annotations, with geometric and free-space checks provided in the appendix.
Critical weaknesses
  1. Standard benchmark comparability remains unresolved. Main results use supplemented labels, while Appendix A.4 also modifies original-label evaluation by excluding false positives inside ground-truth boxes, weakening penalties for overfilled shapes. The geometric checks do not establish that relative gains are independent of this protocol. Please report unmodified-label results with the official evaluation script and clarify how IoU_fut and IoU_avg correspond to OccProphet’s IoU_f and temporally weighted IoŨ_f, including aggregation. Table 9 already reports per-horizon results, but these do not resolve protocol differences.
  2. The closest baseline and related-work coverage are incomplete. GUIDE is kept static for forecasting despite providing velocity estimates and instance tracking. Propagating its Gaussians using predicted velocities would better isolate the forecasting contribution; LighTROcc’s own constant-velocity ablation does not replace this comparison. Relevant uncited work includes DLWM (CVPR 2026), GaussianWorld (CVPR 2025), Drive-OccWorld (AAAI 2025), and IR-WM (ICRA 2026). Please clarify the methodological differences, account for their different tasks and supervision, and provide matched comparisons where applicable. InterOCF: Spatio-Temporal 2D-3D Interaction for Camera-Only 4D Occupancy Forecasting (arXiv:2607.24431, July 2026) is also relevant and uncited. It reports fine-grained GMO results using IoU_f and temporally weighted IoŨ_f.
  3. TSC provides limited evidence for accurate temporal prediction. It aligns consecutive predicted masks using centroids and PCA, favoring the method’s fixed-shape translation even when trajectories or geometry are inaccurate. Per-frame AP also does not establish identity consistency. A ground-truth-linked temporal metric would better support the claimed consistency; rotation and deformation remain unmodeled.
  4. The localization ablation does not isolate attention reuse. Direct 3D regression and hard argmax differ in geometry and gradient flow. A differentiable 2D center and depth head with matched supervision and capacity would provide a more controlled comparison.
Questions for authors
  1. Can you report results on the unmodified benchmark labels using the official evaluation script, including IoU_f and temporally weighted IoŨ_f, and specify their relationship to IoU_fut and IoU_avg? How are frames and samples aggregated, and do the gains persist without the bounding-box false-positive exclusion?
  2. Why is GUIDE kept static during forecasting despite its velocity estimates? How does LighTROcc compare with propagating GUIDE’s Gaussians using those velocities under matched inputs, supervision, and evaluation?
  3. What evidence shows that the high TSC reflects correct temporal predictions beyond preservation of a fixed shape? Can you provide a ground-truth-linked temporal evaluation that jointly assesses occupancy accuracy and instance identity consistency?
  4. Does attention reuse outperform a conventional differentiable 2D center and depth prediction head with matched supervision and capacity? How do the existing ablations separate attention reuse from differences in geometric parameterization and gradient flow?
3 comments

Thank you for your thoughtful and constructive feedback. To address your concerns, we have conducted additional experiments and provide further clarifications below. Please also refer to our newly uploaded PDF.

W1/Q1. Standard benchmark comparability remains unresolved.

We provide additional comparisons with all baselines on the original sparse nuScenes-Occupancy annotations in Appendix A.4. As shown in Table 7, most methods exhibit lower IoU and AP, primarily due to the sparsity that persists even after LiDAR-based annotation post-processing. This is consistent with the distance- and visibility-dependent density degradation analyzed in Appendix Table 5. When the GT itself contains incomplete object volumes, predicting geometrically complete objects inevitably introduces apparent false positives.

To further investigate this issue, we provide additional false-positive analyses using several GT-aligned synthetic probes in Table 9. In particular, our asset-based probes use 3D assets independent of ShapeNet/SMPL and are aligned using GT positions, orientations, and dimensions, making them considerably more accurate than typical network predictions. Nevertheless, the original sparse GT produces several times higher false-positive rates than our supplemented GT. The same trend is observed with 3D bounding-box and Gaussian probes.

These findings suggest that the improvements under our supplemented GT are not simply favorable to our model, but rather reflect a more accurate evaluation of occupied regions missed by the inherently sparse original annotations.

Finally, we follow the same micro-aggregation protocol as OccProphet. To avoid ambiguity, we adopt the conventional metric notation IoUc\mathrm{IoU}_c, IoUf\mathrm{IoU}_f, and IoU~f\tilde{\mathrm{IoU}}_f.

W2/Q2. The closest baseline and related-work coverage are incomplete.

We provide additional forecasting comparisons with GUIDE in Appendix C.3. First, we extrapolate its predicted velocity to future frames. While this improves over static propagation, it cannot account for changes in speed or direction, resulting in relatively limited forecasting accuracy. Since GUIDE only estimates current-frame velocity, we present this as an auxiliary comparison rather than a directly matched forecasting baseline. We additionally report results from fine-tuning GUIDE with the same trajectory head architecture used in LighTROcc, as its official configuration for training from scratch is incomplete.

We also thank you for suggesting related works. We have incorporated these works into our Related Work section and clarified their distinctions. However, direct comparisons remain challenging due to unavailable code or checkpoints (DLWM, InterOCF, Drive-OccWorld) and different input protocols (GaussianWorld, Drive-OccWorld, IR-WM). Specifically, GaussianWorld uses future RGB frames in streaming mode, while Drive-OccWorld uses GT trajectories in A mode and does not release the weights or configuration for P mode. IR-WM requires future GT motion and additional sensor inputs, unlike our RGB-history-only setting. Furthermore, GaussianWorld and IR-WM do not support instance segmentation, making them less aligned with our primary instance-level evaluation.

Nevertheless, these studies are valuable references, and your suggestions have helped us strengthen the Related Work section and clarify our contributions relative to prior methods.

W3/Q3. TSC provides limited evidence for accurate temporal prediction.

We agree with your observation that TSC measures shape consistency in object-local coordinates without directly evaluating future localization accuracy or GT identity correspondence. To address this, we introduce a GT-linked occupancy evaluation in Table 11, where predicted instances are matched one-to-one with GT instances at the current frame, and these correspondences remain fixed throughout future frames. Note that oracle matching is applied only at the current frame.

Under this fixed-correspondence protocol, LighTROcc achieves the highest accuracy across all compared methods, demonstrating that our model preserves not only object geometry but also accurate future trajectories. Furthermore, LighTROcc performs Hungarian matching only once per object during training and supervises its entire trajectory under the same assignment. This design maintains consistent instance identities without frame-wise reassociation, minimizing identity switching during forecasting.

W4/Q4. The localization ablation does not isolate the effect of attention reuse.

We have added a direct 2D center prediction baseline to Table 3 of the main paper. Specifically, we introduce an MLP head that predicts the image-plane center directly from each instance query, which is then lifted into 3D and supervised using the same 3D center loss as our method. For a fair comparison, the MLP head retains identical intermediate hidden dimensions, differing only in its output dimension.

As shown in Table 3, direct 2D center prediction underperforms our attention-reuse approach. Similar to hard argmax, direct regression produces a single point estimate, making localization more susceptible to noise and outliers. In contrast, our soft-argmax approach aggregates spatial evidence across the attention map, enabling more stable and accurate 3D localization.

Dear Reviewer iVXL,

Thank you once again for your valuable review and constructive feedback. As the discussion period is approaching its end on November 18 (AoE), we would like to kindly ask whether our rebuttal has adequately addressed your concerns.

If you have any remaining questions or comments, we would be grateful to hear your thoughts.

Best regards,

The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Response to Reviewers and Summary of Revisions

We sincerely thank the reviewers and the AC for their time and careful consideration. We appreciate the reviewers' recognition of our well-motivated instance-centric formulation and promising accuracy-efficiency trade-off (iVXL, Qf2P, DQpN), as well as the effectiveness of attention-guided localization and learned trajectory prediction supported by extensive ablations (iVXL, Qf2P, DQpN). We also particularly appreciate the recognition of our rigorous GT supplementation, detailed geometric validation, and transparent experimental analyses as important strengths (iVXL, DQpN).

Following the reviewers' suggestions, we have substantially strengthened our evaluation with additional GUIDE forecasting comparisons, a direct 2D localization ablation, GT-linked temporal evaluation, and comprehensive runtime and memory analyses. We have further validated our GT supplementation through unmodified-label evaluations, synthetic-probe analyses, and alternative 3D asset experiments, alongside rotation-aware forecasting studies. Importantly, Reviewer Qf2P acknowledged that our responses had largely addressed their concerns and strengthened their positive assessment of our work, while maintaining their positive rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.