- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Sim2Occ achieves this by constructing a data generation pipeline that incorporates entity-consistent voxel annotations, scene content enrichment, and adaptive viewpoint sampling to generate high-quality occupancy datasets from simulations. A dataset constructed in this way can be trained and tested for a variety of occupancy prediction tasks, including semantic, open-vocabulary, panoramic, and embodied occupancy tasks. Through extensive experiments using multiple baseline models, the author demonstrated that increasing the scale of synthetic data can effectively improve the performance of occupancy prediction.
The article is clear and easy to understand. This paper addresses the challenge of scaling occupancy prediction using supervised learning methods when training data is scarce. Although the overall approach tends to focus on solving existing problems through engineering methods, this issue is of great importance to the field of occupancy prediction. The conclusions of the experimental section generally support the authors’ view that simulated data can improve the accuracy of occupancy rate predictions. However, I have identified some shortcomings in the existing experiments that prevent them from fully supporting the details presented in the paper. By including these experiments, the author can better demonstrate the true value of Sim2Occ.
Critical strengths
This paper's greatest strength is that it provides a workflow for obtaining low-cost occupancy simulation data and shows that scaling up with this data can improve the performance of supervised occupancy prediction tasks. Since most previous occupancy prediction tasks relied primarily on a single real-world dataset—OccScanNet—for training and validation, this has limited the generalization ability of supervised occupancy prediction.
Second, the unified data annotation framework is innovative. The authors unify geometric occupancy, semantic categories, and instance identities into a consistent 3D representation. This property offers a natural advantage in occupancy prediction tasks. Hence, a single 3D annotation workflow can be effectively applied to a variety of occupancy tasks.
Critical weaknesses
In my view, the main shortcoming is the gap between the author’s arguments and the experimental evidence; I hope the author can include quantitative experiments to better support the paper's claims.
-
The experiments demonstrate the effectiveness of Sim2Occ pretraining, but lack direct comparisons with existing synthetic datasets under identical training settings. Therefore, the current results cannot establish whether the improvements stem from synthetic pretraining in general or from the specific advantages of Sim2Occ. I hope the authors will include this experiment to better demonstrate that this data-generation pipeline achieves better results on the occupancy prediction task than other simulation datasets under the same training conditions and control variables.
-
Table 5 evaluates sequential occupancy prediction only on synthetic scenes. Could the authors further evaluate the pretrained models on the real-world EmbodiedOcc-ScanNet benchmark to verify whether these improvements transfer to real-world sequential occupancy prediction? Both EmbodiedOcc and GPOcc have reported results on the real-world EmbodiedOcc-ScanNet benchmark.
Questions for authors
The first question is shown in the weaknesses section. The experiments demonstrate the effectiveness of Sim2Occ pretraining, but lack direct comparisons with existing synthetic datasets under identical training settings. Could the authors provide these comparisons to better demonstrate Sim2Occ's unique advantages?
The second question is also raised in the weaknesses section. Could the authors further evaluate the pretrained models on the real-world EmbodiedOcc-ScanNet benchmark to verify whether these improvements transfer to real-world sequential occupancy prediction?
4 comments
Response to W2 & Q2
Thank you for this important suggestion. We have now evaluated Sim2Occ pretraining on the real-world EmbodiedOcc-ScanNet benchmark using EmbodiedOcc. We first reproduced the baseline, then initialized the model with the 20k, 50k, and 100k Sim2Occ pretrained checkpoints to evaluate transfer to real-world sequential occupancy prediction. These results are now reported in Appendix F.7, Table S17.
| Initialization | Synthetic source size | SC IoU ↑ | mIoU ↑ |
|---|---|---|---|
| Baseline | — | 49.33 | 41.15 |
| Sim2Occ pretraining | 20k | 50.54 | 42.20 |
| Sim2Occ pretraining | 50k | 51.24 | 42.52 |
| Sim2Occ pretraining | 100k | 51.38 | 43.33 |
Scores are percentages on EmbodiedOcc-ScanNet. All gains are computed against baseline.
Sim2Occ pretraining improves both SC IoU and mIoU at every source scale, and both metrics increase consistently from 20k to 100k. With 20k synthetic observations, SC IoU improves by 1.21 and mIoU by 1.04. At 100k, the gains reach 2.05 SC IoU and 2.18 mIoU, yielding 51.38 SC IoU and 43.33 mIoU. The improvement therefore extends to both geometric completion and semantic prediction in real-world sequences.
Together with the synthetic sequence results in Table 6, this experiment demonstrates that the benefits of scaling Sim2Occ pretraining extend to real-world sequential occupancy prediction. The learned occupancy priors support scene-level prediction from successive real observations, connecting synthetic pretraining with embodied perception in real indoor environments.
Thanks for the response, which has addressed my concerns. I will maintain my positive assessment.
We are very glad to hear that our response addressed your concerns! Thank you again for your constructive review and for maintaining your positive assessment of our work!
Response to W1 & Q1
Thank you for this insightful question.
To the best of our knowledge, Sim2Occ is the first framework to combine a systematic study of scalable synthetic pretraining for indoor occupancy with semantic, panoptic, and embodied benchmarks derived from shared scene annotations. SUNCG/SSCNet [1] is an important synthetic indoor-scene dataset and an early precedent for indoor semantic scene completion. Its supervision is centered on semantic scene-completion inputs and voxel labels, whereas Sim2Occ builds a shared scene field with aligned geometry, semantics, and instance identities, supports topology-aware completion and quality-filtered RGB observations, and reuses the same annotations across three occupancy settings. This gives our framework a broader task scope and a direct way to study scale and real-world transfer under a common construction protocol. SUNCG has been subject to a copyright and licensing dispute involving Planner 5D, Princeton, and Facebook, and related releases were withdrawn. We cannot obtain the version required for a matched RGB-based comparison.
We further examine the value of Sim2Occ through two complementary experiments: supervision ablations across three data scales and transfer using occupancy pretraining data generated by our Sim2Occ framework from HSSD synthetic indoor scenes.
1. Contribution of supervision construction. These results are now in Section 4.4, Table 5. We evaluate EmbodiedOcc after pretraining on 3D-FRONT-derived data and fine-tuning on NYUv2, using the same model, data protocol, and training schedule at each scale. Geometry-only pretraining uses binary occupied/free targets. Generic voxelization replaces specialized volumetric construction with surface voxelization and exterior flood filling of enclosed regions, while retaining the same RGB observations and downstream evaluation. The full setting retains the entity-consistent semantic and instance supervision produced by Sim2Occ.
Metrics are percentages; bold marks the best score at each scale. Sim2Occ improves mIoU over generic voxelization by 1.66, 1.55, and 1.48, and over geometry-only supervision by 2.53, 2.95, and 2.63, at 20k, 50k, and 100k, respectively. Increasing data scale improves SC IoU for all three supervision settings, while the full construction remains strongest in semantic transfer at every scale. Sim2Occ at 20k also exceeds the mIoU of generic voxelization at 100k (23.00 vs. 22.58), showing the contribution of supervision construction beyond additional observations. The geometry-only row further separates binary occupancy learning from the semantic and instance information carried by the full labels.
2. Generality across synthetic sources. HSSD [2] provides synthetic indoor scenes, which we convert into occupancy pretraining data using Sim2Occ. Through scene conversion, volumetric annotation, and observation filtering, we generate 20k RGB–occupancy pairs. We compare this source with 3D-FRONT at the same 20k scale, using EmbodiedOcc and the same pretraining steps, model initialization, and NYUv2 fine-tuning protocol. Thus, the source comparison changes the synthetic scenes while keeping the downstream transfer procedure fixed.
Pretraining on occupancy data generated by Sim2Occ from HSSD scenes improves SC IoU by 6.64 and mIoU by 2.72. The transfer gains from both HSSD and 3D-FRONT support Sim2Occ as a general framework for converting synthetic indoor scenes into occupancy pretraining data that benefits real-world prediction.
We will publicly release our code and all datasets to help fill the gap in synthetic indoor occupancy data and support further research in the occupancy community.
References
[1] S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser. Semantic Scene Completion from a Single Depth Image. CVPR, 2017.
[2] M. Khanna, Y. Mao, H. Jiang, S. Haresh, B. Shacklett, D. Batra, A. Clegg, E. Undersander, A. X. Chang, and M. Savva. Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation. CVPR, 2024.