- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a camera-based framework for forecasting the occupancy of movable objects using instance-centric Gaussian representations. A compact set of object queries extracts multi-view temporal features, and attention-guided lifting estimates their 3D centers. Each query predicts a Gaussian mixture representing an object’s current geometry, together with future displacements. Future occupancy is obtained by translating this mixture while retaining its component offsets, covariances, and occupancy weights. Experiments on nuScenes report favorable occupancy accuracy, instance-level AP, and inference efficiency under the authors’ evaluation protocol.
Critical strengths
-
The instance-centric formulation provides an efficient alternative to dense voxel forecasting. The reported accuracy efficiency trade-off is promising, and the lower-resolution variant helps demonstrate that the gains are not solely attributable to higher image resolution.
-
The paper includes useful supporting analyses. The lifting and query-count ablations examine important design choices, while the constant-velocity comparison demonstrates a benefit from learned trajectory prediction. The annotation audit, original sparse-label evaluation, and supplementary visualizations also improve the transparency of the experimental study.
Critical weaknesses
W1. The contribution beyond existing Gaussian instance representations is not sufficiently isolated. GUIDE already represents object instances using Gaussian mixtures [1]. LighTROcc adds attention-guided localization and future displacement prediction, and the localization ablation supports the former. However, a particularly informative comparison would extend GUIDE with the same trajectory head and observed-motion inputs under a matched training and evaluation setup. The constant-velocity ablation establishes that the learned head improves this implementation, but does not determine how much of the overall gain requires the proposed formulation. Related work should also position the method more clearly against S2GO’s temporal Gaussian representation [2], while acknowledging that its semantic occupancy setting differs from instance-level GMO forecasting.
W2. The forecasting model imposes a restrictive geometry assumption whose consequences are insufficiently characterized. Future prediction translates each Gaussian mixture without updating its relative component positions, covariances, or occupancy weights. This cannot explicitly model future object rotation or articulated shape changes. The learned trajectory head clearly improves over constant velocity, but aggregate results do not reveal how the method behaves during vehicle turns, pedestrian articulation, or other departures from translation. A motion-dependent breakdown and a comparison with an otherwise matched model that predicts orientation would clarify when the lightweight approximation is adequate and when it fails.
W3. The temporal consistency results do not independently establish accurate temporal prediction. The reported TSC metric compares consecutive predicted shapes after centroid subtraction and PCA alignment. Since LighTROcc retains the same Gaussian geometry across future steps, high shape consistency is largely encouraged by construction; an inaccurate but unchanged shape can also receive a high score. TSC therefore supports geometric stability, but does not independently demonstrate accurate future pose or identity association. Framewise AP provides complementary evidence, although comparisons with dense methods also reflect their flow-based instance grouping. A ground-truth-referenced trajectory or instance-tube evaluation would better distinguish stable representation, accurate motion, and correct temporal association.
Questions for authors
-
How does LighTROcc compare with GUIDE augmented by the same learned trajectory head, motion inputs, and training losses?
-
Do the main performance rankings persist under the unmodified original-label evaluation and alternative choices in the annotation completion procedure?
-
How does performance vary between approximately translating objects and objects undergoing substantial rotation or articulation? What is the cost and benefit of predicting future orientation?
3 comments
W3. The temporal consistency results do not independently establish accurate temporal prediction.
We acknowledge your point that TSC evaluates shape stability in object-local coordinates but does not directly measure future localization accuracy or consistency with GT identities. To address this issue, we introduce a GT-linked occupancy evaluation in Table 11. Predicted instances are associated with GT instances through one-to-one oracle matching at the current frame, and the resulting correspondences are kept fixed across all future horizons. Please note that oracle matching is performed only at the initial frame.
LighTROcc consistently outperforms all baselines under this fixed-assignment evaluation, providing additional evidence that our method accurately forecasts future object trajectories beyond merely preserving shape consistency. Furthermore, during training, LighTROcc establishes a single Hungarian assignment per instance and supervises its entire temporal trajectory under that assignment. This avoids frame-wise reassociation and encourages consistent instance identities throughout forecasting.
Q2. Performance Robustness under Original GT and Annotation Completion
Table 7 compares our method with existing baselines using the original sparse annotations, where the method rankings are not fully preserved. However, our synthetic probe experiments and distance- and visibility-based analyses suggest that this degradation cannot be attributed solely to detection errors. Sparse annotations can favor methods that suppress predictions for challenging objects or predict only partially observed regions, potentially yielding higher scores. Our supplemented annotations aim to address this bias by more accurately rewarding correct occupancy and penalizing missed regions. Thus, changes in method rankings can naturally arise under a more complete evaluation protocol.
Additional evaluations using two alternative 3D asset sets (A1 and A2) from Objaverse are presented below.
| Method | Main | Main | Main | Main | A1 | A1 | A1 | A1 | A2 | A2 | A2 | A2 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Cam4DOcc | 13.42 | 11.80 | 10.71 | 8.02 | 13.38 | 11.74 | 10.66 | 7.97 | 13.33 | 11.84 | 10.64 | 7.95 |
| EfficientOCF | 8.67 | 8.59 | 14.25 | 14.13 | 8.70 | 8.62 | 14.20 | 14.06 | 8.62 | 8.54 | 14.18 | 14.04 |
| OccProphet | 17.57 | 14.23 | 20.14 | 7.25 | 17.64 | 14.33 | 20.37 | 7.32 | 17.50 | 14.15 | 20.04 | 7.31 |
| SparseOcc | 18.42 | 13.67 | 20.05 | 13.36 | 18.39 | 13.71 | 19.99 | 13.40 | 18.34 | 13.58 | 20.11 | 13.28 |
| GUIDE | 15.21 | 12.40 | 34.73 | 24.51 | 15.18 | 12.36 | 34.83 | 24.44 | 15.13 | 12.33 | 34.59 | 24.39 |
| LighTROcc | 19.76 | 15.30 | 42.62 | 33.34 | 19.91 | 15.38 | 42.54 | 33.28 | 19.66 | 15.23 | 42.31 | 33.12 |
The results show minimal changes in performance and method rankings across the alternative asset sets. This suggests that our annotation completion remains robust when using reasonably representative 3D assets, rather than highly unrealistic geometries.
Dear Reviewer DQpN,
Thank you once again for your valuable review and constructive feedback. As the discussion period is approaching its end on November 18 (AoE), we would like to kindly ask whether our rebuttal has adequately addressed your concerns.
If you have any remaining questions or comments, we would be grateful to hear your thoughts.
Best regards,
The Authors
Thank you for your careful review and constructive suggestions. To address your concerns, we have conducted additional experiments and revised the manuscript accordingly. We provide detailed responses below, and please also refer to our newly uploaded PDF for further results and analyses.
W1/Q1. The contribution beyond existing Gaussian instance representations is not sufficiently isolated.
To address your concern, we have added a comparison with GUIDE equipped with an adapted trajectory head in Table 12. Please note that we fine-tuned GUIDE rather than training it from scratch, as its official training configuration is incomplete. We additionally evaluate extrapolation using the velocity and heading predicted by GUIDE at the current frame.
Adding the trajectory head clearly improves forecasting performance over future horizons. However, the remaining performance gap appears to be influenced by differences in object shape accuracy and spurious or missed instance predictions, rather than trajectory prediction alone.
We have also strengthened the Related Work section to better clarify the distinctions between LighTROcc and S2GO.
W2/Q3. The forecasting model imposes a restrictive geometry assumption whose consequences are insufficiently characterized.
To address your concern, we provide a rotation-stratified evaluation in Table 13, evaluating turning objects across different rotation ranges. LighTROcc consistently achieves the highest accuracy across these ranges. However, we note that shape reconstruction errors in competing methods may outweigh rotation-induced errors, making it difficult to isolate the effect of orientation alone.
Following your suggestion, we also retrained an otherwise matched variant with an additional yaw prediction output in the trajectory head. The results below compare translation-only forecasting with yaw-aware forecasting on rotation-specific subsets.
Explicit yaw prediction provides modest improvements, particularly for larger rotations, but has negligible impact on aggregate performance. One key limitation is the scarcity of rotating objects, accounting for only 2.04% (5–15°) and 2.16% (≥15°) of instances. We suspect this sparsity is further amplified by the overlapping 7-frame window protocol adopted by existing forecasting methods. This may also partly explain their use of inflated GMO representations with AABBs rather than OBBs.
To further examine the articulated motion, we additionally report GT-matched pedestrian instance IoU across future horizons.
Both methods exhibit decreasing IoU over longer horizons. Since all baselines operate at a voxel resolution of 0.2 m, accurately forecasting fine-grained limb articulation remains challenging. Moreover, small-volume objects such as pedestrians are particularly sensitive to minor localization and orientation errors, naturally resulting in substantially lower IoU than larger objects such as vehicles [1, 2].
[1] Caesar, Holger, et al. "nuscenes: A multimodal dataset for autonomous driving." 2020 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2020.
[2] Cheng, Gong, et al. "Towards large-scale small object detection: Survey and benchmarks." IEEE transactions on pattern analysis and machine intelligence 45.11 (2023): 13467-13488.