acceptodds
Under review as a conference paper at ICLR 2027

Sim2Occ: Scaling Indoor Occupancy Pretraining with Unified Synthetic Supervision

Abstract

Indoor occupancy models benefit from large-scale supervised learning, but existing real-world datasets provide limited volumetric supervision and are costly to expand while maintaining consistent geometry, semantic labels, and instance identities. We introduce Sim2Occ, an automated framework that turns furnished virtual interiors into a unified source of synthetic occupancy supervision for pretraining and benchmarking. At its core is an entity-consistent scene field that resolves mesh ownership and completes enclosed object bodies, keeping geometry, semantic labels, and instance identities consistent within room-bounded supervision domains. Scene enrichment and room-adaptive view selection produce diverse image–occupancy pairs, with alignment checked against rendered depth. We provide nested semantic training sets of 20k, 50k, and 100k observations, alongside panoptic and embodied occupancy benchmarks. Across five baselines, increasing the synthetic training scale improves both geometric and semantic occupancy prediction, with gains of 5.32–9.37 mIoU from 20k to 100k. When fine-tuned on real-world datasets, models pretrained on our benchmark achieve semantic completion gains of up to 4.30 mIoU on NYUv2 and 3.47 on OccScanNet. Open-vocabulary occupancy also benefits, gaining 4.21 mIoU on NYUv2. Larger single-frame training sources also improve embodied occupancy prediction, while panoptic baselines demonstrate instance-aware completion from the shared annotations. Sim2Occ establishes a scalable resource for studying transferable indoor occupancy representations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D Occupancy Prediction, 3D Pretraining, 3D scene Understanding

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_qSEkOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

Sim2Occ achieves this by constructing a data generation pipeline that incorporates entity-consistent voxel annotations, scene content enrichment, and adaptive viewpoint sampling to generate high-quality occupancy datasets from simulations. A dataset constructed in this way can be trained and tested for a variety of occupancy prediction tasks, including semantic, open-vocabulary, panoramic, and embodied occupancy tasks. Through extensive experiments using multiple baseline models, the author demonstrated that increasing the scale of synthetic data can effectively improve the performance of occupancy prediction.

The article is clear and easy to understand. This paper addresses the challenge of scaling occupancy prediction using supervised learning methods when training data is scarce. Although the overall approach tends to focus on solving existing problems through engineering methods, this issue is of great importance to the field of occupancy prediction. The conclusions of the experimental section generally support the authors’ view that simulated data can improve the accuracy of occupancy rate predictions. However, I have identified some shortcomings in the existing experiments that prevent them from fully supporting the details presented in the paper. By including these experiments, the author can better demonstrate the true value of Sim2Occ.

Critical strengths

This paper's greatest strength is that it provides a workflow for obtaining low-cost occupancy simulation data and shows that scaling up with this data can improve the performance of supervised occupancy prediction tasks. Since most previous occupancy prediction tasks relied primarily on a single real-world dataset—OccScanNet—for training and validation, this has limited the generalization ability of supervised occupancy prediction.

Second, the unified data annotation framework is innovative. The authors unify geometric occupancy, semantic categories, and instance identities into a consistent 3D representation. This property offers a natural advantage in occupancy prediction tasks. Hence, a single 3D annotation workflow can be effectively applied to a variety of occupancy tasks.

Critical weaknesses

In my view, the main shortcoming is the gap between the author’s arguments and the experimental evidence; I hope the author can include quantitative experiments to better support the paper's claims.

  1. The experiments demonstrate the effectiveness of Sim2Occ pretraining, but lack direct comparisons with existing synthetic datasets under identical training settings. Therefore, the current results cannot establish whether the improvements stem from synthetic pretraining in general or from the specific advantages of Sim2Occ. I hope the authors will include this experiment to better demonstrate that this data-generation pipeline achieves better results on the occupancy prediction task than other simulation datasets under the same training conditions and control variables.

  2. Table 5 evaluates sequential occupancy prediction only on synthetic scenes. Could the authors further evaluate the pretrained models on the real-world EmbodiedOcc-ScanNet benchmark to verify whether these improvements transfer to real-world sequential occupancy prediction? Both EmbodiedOcc and GPOcc have reported results on the real-world EmbodiedOcc-ScanNet benchmark.

Questions for authors

The first question is shown in the weaknesses section. The experiments demonstrate the effectiveness of Sim2Occ pretraining, but lack direct comparisons with existing synthetic datasets under identical training settings. Could the authors provide these comparisons to better demonstrate Sim2Occ's unique advantages?

The second question is also raised in the weaknesses section. Could the authors further evaluate the pretrained models on the real-world EmbodiedOcc-ScanNet benchmark to verify whether these improvements transfer to real-world sequential occupancy prediction?

4 comments
AuthorsOpenReview ↗

Response to W1 & Q1

Thank you for this insightful question.

To the best of our knowledge, Sim2Occ is the first framework to combine a systematic study of scalable synthetic pretraining for indoor occupancy with semantic, panoptic, and embodied benchmarks derived from shared scene annotations. SUNCG/SSCNet [1] is an important synthetic indoor-scene dataset and an early precedent for indoor semantic scene completion. Its supervision is centered on semantic scene-completion inputs and voxel labels, whereas Sim2Occ builds a shared scene field with aligned geometry, semantics, and instance identities, supports topology-aware completion and quality-filtered RGB observations, and reuses the same annotations across three occupancy settings. This gives our framework a broader task scope and a direct way to study scale and real-world transfer under a common construction protocol. SUNCG has been subject to a copyright and licensing dispute involving Planner 5D, Princeton, and Facebook, and related releases were withdrawn. We cannot obtain the version required for a matched RGB-based comparison.

We further examine the value of Sim2Occ through two complementary experiments: supervision ablations across three data scales and transfer using occupancy pretraining data generated by our Sim2Occ framework from HSSD synthetic indoor scenes.

1. Contribution of supervision construction. These results are now in Section 4.4, Table 5. We evaluate EmbodiedOcc after pretraining on 3D-FRONT-derived data and fine-tuning on NYUv2, using the same model, data protocol, and training schedule at each scale. Geometry-only pretraining uses binary occupied/free targets. Generic voxelization replaces specialized volumetric construction with surface voxelization and exterior flood filling of enclosed regions, while retaining the same RGB observations and downstream evaluation. The full setting retains the entity-consistent semantic and instance supervision produced by Sim2Occ.

Source sizeSupervisionSC IoU ↑mIoU ↑
20kGeometry-only34.5520.47
Generic voxelization35.7021.34
Sim2Occ36.8123.00
50kGeometry-only36.2321.13
Generic voxelization37.1522.53
Sim2Occ38.2524.08
100kGeometry-only37.1721.43
Generic voxelization37.2322.58
Sim2Occ39.2424.06

Metrics are percentages; bold marks the best score at each scale. Sim2Occ improves mIoU over generic voxelization by 1.66, 1.55, and 1.48, and over geometry-only supervision by 2.53, 2.95, and 2.63, at 20k, 50k, and 100k, respectively. Increasing data scale improves SC IoU for all three supervision settings, while the full construction remains strongest in semantic transfer at every scale. Sim2Occ at 20k also exceeds the mIoU of generic voxelization at 100k (23.00 vs. 22.58), showing the contribution of supervision construction beyond additional observations. The geometry-only row further separates binary occupancy learning from the semantic and instance information carried by the full labels.

2. Generality across synthetic sources. HSSD [2] provides synthetic indoor scenes, which we convert into occupancy pretraining data using Sim2Occ. Through scene conversion, volumetric annotation, and observation filtering, we generate 20k RGB–occupancy pairs. We compare this source with 3D-FRONT at the same 20k scale, using EmbodiedOcc and the same pretraining steps, model initialization, and NYUv2 fine-tuning protocol. Thus, the source comparison changes the synthetic scenes while keeping the downstream transfer procedure fixed.

Scene sourceConstructionObservationsSC IoU ↑mIoU ↑
None——30.6819.78
3D-FRONTSim2Occ20k36.8123.00
HSSDSim2Occ20k37.3222.50

Pretraining on occupancy data generated by Sim2Occ from HSSD scenes improves SC IoU by 6.64 and mIoU by 2.72. The transfer gains from both HSSD and 3D-FRONT support Sim2Occ as a general framework for converting synthetic indoor scenes into occupancy pretraining data that benefits real-world prediction.

We will publicly release our code and all datasets to help fill the gap in synthetic indoor occupancy data and support further research in the occupancy community.

References

[1] S. Song, F. Yu, A. Zeng, A. X. Chang, M. Savva, and T. Funkhouser. Semantic Scene Completion from a Single Depth Image. CVPR, 2017.

[2] M. Khanna, Y. Mao, H. Jiang, S. Haresh, B. Shacklett, D. Batra, A. Clegg, E. Undersander, A. X. Chang, and M. Savva. Habitat Synthetic Scenes Dataset (HSSD-200): An Analysis of 3D Scene Scale and Realism Tradeoffs for ObjectGoal Navigation. CVPR, 2024.

AuthorsOpenReview ↗

Response to W2 & Q2

Thank you for this important suggestion. We have now evaluated Sim2Occ pretraining on the real-world EmbodiedOcc-ScanNet benchmark using EmbodiedOcc. We first reproduced the baseline, then initialized the model with the 20k, 50k, and 100k Sim2Occ pretrained checkpoints to evaluate transfer to real-world sequential occupancy prediction. These results are now reported in Appendix F.7, Table S17.

InitializationSynthetic source sizeSC IoU ↑mIoU ↑
Baseline—49.3341.15
Sim2Occ pretraining20k50.5442.20
Sim2Occ pretraining50k51.2442.52
Sim2Occ pretraining100k51.3843.33

Scores are percentages on EmbodiedOcc-ScanNet. All gains are computed against baseline.

Sim2Occ pretraining improves both SC IoU and mIoU at every source scale, and both metrics increase consistently from 20k to 100k. With 20k synthetic observations, SC IoU improves by 1.21 and mIoU by 1.04. At 100k, the gains reach 2.05 SC IoU and 2.18 mIoU, yielding 51.38 SC IoU and 43.33 mIoU. The improvement therefore extends to both geometric completion and semantic prediction in real-world sequences.

Together with the synthetic sequence results in Table 6, this experiment demonstrates that the benefits of scaling Sim2Occ pretraining extend to real-world sequential occupancy prediction. The learned occupancy priors support scene-level prediction from successive real observations, connecting synthetic pretraining with embodied perception in real indoor environments.

Thanks for the response, which has addressed my concerns. I will maintain my positive assessment.

We are very glad to hear that our response addressed your concerns! Thank you again for your constructive review and for maintaining your positive assessment of our work!

Reviewer_1swqOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper introduces Sim2Occ, an automated framework and benchmark family that converts furnished virtual interiors into indoor occupancy supervision and studies how synthetic data scale affects occupancy learning and real-world transfer.

Sim2Occ has three stages. Structured scene composition retains 3D-FRONT/3D-FUTURE layouts and geometry, enriches appearance with category-compatible materials, and inserts support-constrained assets. Entity-consistent volumetric annotation uses physical entities rather than mesh components as annotation units, so cabinet panels share one identity while independent accessories stay separate; it performs topology-conditioned completion (filling enclosed furniture but keeping open shelving empty) and room-bounded support, with each occupied voxel carrying occupancy, category, and a stable instance ID. Room-adaptive observation selection samples coverage-balanced camera poses and filters views by comparing rendered depth with occupancy first-hit depth

From the shared scene field, the paper builds semantic (20k/50k/100k nested sets), panoptic, and embodied (30-view room episodes) benchmarks with house-level train/test separation. Experiments on MonoScene, EmbodiedOcc, SplatSSC, GPOcc, and POP3D show that scaling from 20k to 100k consistently improves SC IoU and mIoU.

Critical strengths
  1. The paper frames indoor occupancy around data scaling and sim-to-real transfer, using nested 20k/50k/100k sets with matched iteration budgets to isolate source-scale effects across semantic, open-vocabulary, panoptic, and embodied settings.

  2. The data construction pipeline is carefully engineered, handling physical ownership, topology-conditioned completion, and room-bounded support beyond naive mesh voxelization, with Appendix F.3 ablations confirming these components are functional rather than cosmetic.

Critical weaknesses
  1. The writing is unnecessarily abstract and notation-heavy, obscuring an otherwise clear pipeline and forcing readers to reconstruct key operational details from the appendix.

  2. The paper lacks true zero-shot sim-to-real evaluation, relying instead on target-domain fine-tuning. While inconsistent label taxonomies across datasets may hinder direct evaluation, this should motivate label-agnostic or open-vocabulary testing (e.g., using foundation models like SAM3) rather than falling back on fine-tuning. As it stands, the transfer claim is reduced to a mere initialization effect, not direct sim-to-real generalization in the spirit of Depth Anything or VGGT.

  3. The paper misses a critical baseline: equal-budget pretraining on real data. With OccScanNet already providing 65k real observations, it remains unclear whether 100k synthetic frames are actually worth more than a smaller amount of additional real data.

  4. Most critically, the paper lacks an ablation isolating geometric pretraining from semantic/instance pretraining. If the performance gains primarily stem from dense geometry rather than the proposed entity-consistent labels, then vast amounts of unlabeled or cross-domain data (e.g., outdoor datasets like nuScenes) could serve as a far cheaper and more scalable pretraining source, undermining the necessity of the complex annotation pipeline. Notably, outdoor semantic scene completion methods (e.g., VoxFormer [1], Symphonize [2], CGFormer [3]) have shown that accuracy gains often rely more heavily on geometric improvements from depth estimation. A plausible alternative explanation is that Sim2Occ is mainly enhancing the model's generic geometric perception, rather than providing transferable semantic or instance-level knowledge. Without a controlled ablation (e.g., pretraining with only binary occupancy labels), the authors cannot rule out this explanation.

[1] CVPR2023, VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completion [2] CVPR2024, Symphonize 3D Semantic Scene Completion with Contextual Instance Queries [3] NeurIPS2025, Context and Geometry aware Voxel Transformer for Semantic Scene Completion

Questions for authors

Please see weakness

6 comments
AuthorsOpenReview ↗

Response to W1

Thank you for this helpful feedback. We have revised the Method section to make the construction process easier to follow, particularly where abstract notation obscured the underlying operations. The revision clarifies the data flow and key processing steps while reducing redundant notation.

  • Pipeline overview (Section 3). We replaced the abstract composition of stage operators with a direct description of scene enrichment, entity grouping, voxelization, and observation generation. We also explain what the shared scene field stores and how each rendered image is paired with its occupancy target.
  • Entity-consistent annotation (Section 3.2). We clarified how source placement records, parent relations, component categories, and spatial adjacency establish entity ownership. The simplified annotation equation now follows an explicit explanation of how occupied voxels inherit their owners’ categories and identities. We also clarify that source-scene geometry supplies supervision for occluded objects within the room.
  • Observation selection (Section 3.3). We distinguish coverage-based pose selection before rendering from visible-surface alignment checks after rendering. The text now explains how greedy selection updates the selected poses and coverage counts, and how quality filtering precedes the final selection of eligible observations.
  • Unified benchmarks (Section 3.4). We replaced the support-equivalence equation with a direct explanation of how semantic and panoptic targets share occupied geometry. We also clarify how embodied targets accumulate valid supervision regions and read their labels from the shared field.

The Method section now contains four displayed equations instead of six, with more explicit operational explanations around the retained equations. The changes are highlighted in blue in the revised manuscript.

AuthorsOpenReview ↗

Response to W2

Thank you for this important suggestion. We agree that direct evaluation on real images is needed to establish zero-shot sim-to-real generalization. We have now evaluated synthetic-trained MonoScene and EmbodiedOcc directly on NYUv2 and OccScanNet, without target-domain fine-tuning. These experiments further demonstrate the models' ability to transfer directly from synthetic to real scenes.

1. Direct zero-shot evaluation. These results are now reported in Appendix F.5, Table S15. Each model is trained on the 20k, 50k, or 100k Sim2Occ source and evaluated on the real target datasets. We report both semantic mIoU and binary occupied/free SC IoU. SC IoU evaluates geometry without requiring agreement between semantic category names; mIoU evaluates semantic transfer under the shared category taxonomy. All scores are percentages. Bold marks the best result across source scales for each model and metric on each dataset.

NYUv2 — zero-shot evaluation

ModelSource sizeSC IoU ↑mIoU ↑
20k28.9914.28
MonoScene50k29.6215.99
100k29.0616.25
20k28.8612.85
EmbodiedOcc50k30.4113.62
100k31.0114.04

OccScanNet — zero-shot evaluation

ModelSource sizeSC IoU ↑mIoU ↑
20k22.918.36
MonoScene50k23.348.43
100k23.359.35
20k18.928.36
EmbodiedOcc50k19.489.25
100k19.729.29

Semantic mIoU improves from 20k to 100k for both models on both real datasets. On NYUv2, MonoScene increases from 14.28 to 16.25 mIoU and EmbodiedOcc from 12.85 to 14.04. On OccScanNet, their mIoU increases from 8.36 to 9.35 and from 8.36 to 9.29, respectively. SC IoU also improves from 20k to 100k in all four settings. Thus, the effect of synthetic scale extends to direct evaluation, with particularly consistent trends in semantic prediction.

These measurements distinguish two useful outcomes of the same data source. The original transfer experiments measure how synthetic supervision benefits subsequent real-domain learning. The new experiments measure what the synthetic-trained models already predict on real scenes before that adaptation. The latter establishes direct sim-to-real transfer across two architectures and two real datasets.

2. Open-vocabulary evidence already in the submission. We evaluate POP3D, which learns occupied geometry and language-aligned voxel features without 3D semantic category labels and uses text queries for prediction. The main tables report quantitative results using the shared 11-class query set. The appendix additionally provides qualitative text-query visualizations for concepts such as pillows and curtains, illustrating predictions beyond the coarse evaluation vocabulary. These examples complement the fixed-category evaluation by showing how the same source supports language-based occupancy prediction. The reported POP3D real-domain transfer results use fine-tuning; the tables above supply the new evidence for direct zero-shot transfer.

3. Context of training scale. Depth Anything [1] uses approximately 1.5 million labeled images and 62 million unlabeled images, while VGGT [2] trains on 17 datasets with 3D annotations. These works demonstrate the value of broad geometric training data. Our experiments study an occupancy-specific synthetic source of 20k–100k observations, with dense volumetric supervision for each observation. At this scale, the new results already show direct geometric and semantic transfer to real indoor scenes. This supports the value of Sim2Occ as a scalable source for occupancy learning and provides a concrete basis for studying zero-shot transfer as the source grows.

References

[1] L. Yang et al. Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data. CVPR, 2024.

[2] J. Wang et al. VGGT: Visual Geometry Grounded Transformer. CVPR, 2025.

AuthorsOpenReview ↗

Response to W3

Thank you for identifying this missing comparison. We have added real-source pretraining on OccScanNet followed by NYUv2 fine-tuning with EmbodiedOcc, and a 45k Sim2Occ source to match the real training-set size. The two 45k runs use the same number of training observations and identical model initialization, pretraining budgets, and NYUv2 fine-tuning data and settings. Only the pretraining source changes. These results are now reported in Appendix F.6, Table S16.

Pretraining sourceSource observationsSC IoU ↑mIoU ↑
None—30.6819.78
OccScanNet (real)45k36.6722.49
Sim2Occ20k36.8123.00
Sim2Occ45k36.6323.21
Sim2Occ50k38.2524.08
Sim2Occ100k39.2424.06

Scores are percentages after NYUv2 fine-tuning. The 45k rows form the equal-budget comparison. The 65k figure in the submission describes the overall OccScanNet collection; this experiment uses its training split.

At the matched 45k source size, Sim2Occ achieves 23.21 mIoU, compared with 22.49 for real-source pretraining, with comparable SC IoU. Even the 20k synthetic source achieves 23.00 mIoU and 36.81 SC IoU, exceeding the real-source result with fewer source observations. Expanding Sim2Occ to 50k and 100k gives 24.08 and 24.06 mIoU, respectively, while SC IoU increases to 38.25 and 39.24. These results show that the synthetic source provides effective semantic transfer at a matched data volume, and that its benefits extend to larger scales.

A plausible explanation for the stronger semantic transfer is the consistency and completeness of the supervision. Real-world depth acquisition can produce missing or distorted measurements on transparent and reflective surfaces, such as glass and mirrors, while incomplete observations can leave sparse regions in reconstructed point clouds. These acquisition errors can propagate into volumetric targets. Sim2Occ constructs occupancy targets directly from scene geometry and assigns semantic categories and instance identities through a shared entity representation, avoiding the propagation of sensor acquisition errors into its labels. This provides dense supervision within the valid completion domain, including occluded regions, with consistent category and identity assignments across views. Such supervision can help models learn coherent object and scene representations, offering an explanation for the stronger semantic transfer observed with synthetic pretraining. Sim2Occ thereby connects simulation-derived supervision with pretraining for real-world occupancy prediction.

The practical value also concerns how additional supervision is obtained. Expanding a real indoor occupancy collection requires acquiring scenes, reconstructing and aligning their geometry, and producing consistent annotations. Sim2Occ generates observations and volumetric targets from virtual scenes under a common construction protocol. Scene content, viewpoints, and source size can therefore be expanded without a new round of real-world capture and manual dense annotation. The matched-size result and the scale study together support Sim2Occ as an effective, expandable source of pretraining supervision for real indoor occupancy.

Reviewer_qBqaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Sim2Occ, which transforms virtual indoor scenes into occupancy data that is consistent in terms of geometry, semantics and instance identity. Through scene enrichment, entity-level volume annotation and viewpoint filtering, it constructs a semantic, panoramic and embodied occupancy benchmark. Experiments demonstrate the benefits of scaling up synthetic data, as well as the assistance provided by synthetic pre-training for fine-tuning on real data.

Critical strengths

1 The problem is of practical significance. The framework focuses on entity attribution, occlusion region completion and cross-view annotation consistency, and is capable of providing shared supervision for multiple object detection tasks.

2 The experiments cover a variety of architectures and two real-world datasets, whilst controlling the iteration budget for different pre-training scales. The appendix also includes an analysis of the number of rooms and view density, as well as data construction ablation studies, which support the effectiveness of the approach.

Critical weaknesses

The implementation description for the key annotation stages is insufficient. Entity attribution, closed-region completion and the preservation of open gaps constitute the core contributions; however, these are primarily described through rules and examples. It remains unclear how the filling operations and permissible filling regions in Appendix Formula S1 are calculated, as well as the key thresholds for quality screening, which affects reproducibility.

There is a lack of direct verification of the core annotation quality. Deep consistency primarily checks the alignment of visible surfaces, whilst local semantic continuity is insufficient to adequately determine whether instance attribution and the completion of hidden regions are correct. Sampling verification or error statistics need to be provided for these core aspects.

The advantages over existing synthetic data sources are not yet sufficiently demonstrated. Whilst the existing ablation studies support the effectiveness of some construction steps, there is a lack of comparison with other synthetic sources using the same model, similar data scale and computational budget; consequently, it remains difficult to assess the unique value of this resource relative to existing solutions.

Questions for authors

How can the ownership of entities within mesh components and the fillable internal areas be automatically determined? How should one handle non-enclosed meshes, open furniture or ownership conflicts?

Has a separate quality check been carried out on instance ownership and the completion of invisible areas? What are the main types of errors and their incidence rates?

Given similar data volumes and pre-training budgets, how much additional transfer learning benefit can Sim2Occ provide compared to existing synthetic indoor data?

6 comments
AuthorsOpenReview ↗

Response to W1 & Q1

Thank you for highlighting the need for clearer implementation details. We have expanded Appendix B.3 with entity ownership, the filling operations and admissible regions in Eq. S1, and conflict-handling rules. Appendix C.1 now specifies screening thresholds and the quality-score configuration (Table S6). We summarize these operations below.

Key annotation operations and handling rules

Operation or caseDecision basisHandling rule
Entity ownershipSource placement identities, parent relations, and constrained geometric adjacencyAssign structural parts to their furniture entity; keep independently placed accessories separate. Repeated placements of the same CAD model remain distinct.
Interior candidatesOccupancy support along three coordinate axesA voxel enters the general filling candidates when at least two axes provide interior support.
Admissible filling regionSource categories, descriptions, and entity boundsApply body completion to eligible beds, sofas, and closed storage within their voxel bounds.
Non-watertight meshes and open furnitureVoxel support and structural typeComplete eligible bodies when support conditions hold; exclude open shelving from the closed-storage filling branch and retain table/chair geometry.
Ownership conflictsExisting labels, occupancy priority, and fixed processing orderFurniture filling writes only to free voxels; earlier fills retain ownership. Initial geometry overlaps follow occupancy priority, with traversal order breaking ties.
Observation filteringValid depth, first-hit agreement, and connected error regionsReject failed views, rank eligible observations, and select views for coverage.

Filling construction in Eq. S1. For an entity's surface voxels, we examine grid lines along each coordinate axis within its voxel bounds. A location receives interior support along an axis if it lies strictly between the first and last voxels belonging to that entity on the same grid line. Support from at least two axes defines the general candidate set. Beds additionally use the common footprint of qualifying upper and lower layers, extended between them. The admissible region restricts these candidates by eligible body type and entity bounds. Thus, completion can operate on non-watertight source meshes, while open shelving is excluded through the structural eligibility rule. Added voxels inherit the entity's category and instance identity; existing occupied and invalid voxels are preserved. Window normalization is handled separately within architectural openings, where free, wall, and window voxels may be reassigned while other occupied categories and invalid labels are protected.

Quality-screening criteria. Rendered depth is compared with occupancy first-hit depth on rays with valid, locally stable reference depth inside the target volume. Frames with valid depth covering less than 80% of the image are rejected. A first hit more than 0.24 m in front of the rendered surface defines a premature-occupancy ray; frame rejection additionally considers the fraction and spatial connectivity of such errors. Missing visible support and poor surface alignment are also checked, with window-related missing support treated separately. Eligible views are then ranked by geometric agreement and scene information, and coverage selection uses the remaining qualified observations.

Quality score. For eligible views, each measurement below is linearly normalized over its interval and clipped to [0,1]; downward terms are reversed. The score is 100 times the weighted sum. These intervals define ranking contributions, separate from rejection thresholds.

MeasurementIntervalWeight
Depth 90th–10th percentile span ↑[0.35, 2.50] m0.35
Depth-edge fraction ↑[0.001, 0.015]0.15
Dominant connected planar fraction ↓[0.25, 0.65]0.10
First-hit alignment within 0.24 m ↑[0.35, 0.85]0.10
Premature-occupancy fraction ↓[0.08, 0.50]0.15
Missing-support fraction ↓[0.15, 0.50]0.15
AuthorsOpenReview ↗

Response to W2 & Q2

Thank you for suggesting direct verification of instance ownership and completed regions. We randomly sampled rooms for manual inspection, reviewing 1,005 distinct output instance IDs across 84 rooms and 215 views, using RGB images, projected instance masks, and occupancy visualizations. IDs are deduplicated within each room across views. Source meshes, component records, and volume slices provide additional evidence for ambiguous assignments and selected completed regions.

Complementary quality checks. Depth consistency checks the agreement between rendered surfaces and occupancy first intersections. It detects surface misalignment, missing visible geometry, and excess occupancy in front of the rendered surface. Instance inspection checks whether furniture components share the appropriate identity and whether independent accessories remain distinct. Source-geometry comparisons and volume slices allow inspection of selected regions hidden in the RGB view, including completed furniture bodies and open spaces.

We verify completed regions hidden from the current viewpoint using source-scene geometry, room ownership, and volumetric slices. During development, these checks identified multilayer or hollow windows and hidden neighboring-room geometry leaking into supervision. We corrected these issues through window normalization and room-ownership constraints, then verified the results through volumetric visualization and per-frame checks.

Instance-ownership findings. Manual review confirmed 18 output IDs with ownership errors, summarized below.

Error typeAffected output IDs
Furniture assembly split across component IDs6
Major furniture part separated from its parent instance4
Mirror and frame assigned separate identities4
Incomplete pillow attribution4
Total18

The confirmed instance-ownership errors affect 18 of the 1,005 sampled IDs (1.79%). Together, the depth-consistency checks and manual inspection support the overall quality of the dataset in visible geometric alignment and instance consistency, providing a sound basis for occupancy pretraining.

AuthorsOpenReview ↗

Response to W3 & Q3

Thank you for this insightful question.

To the best of our knowledge, Sim2Occ is the first framework to combine a systematic study of scalable synthetic pretraining for indoor occupancy with semantic, panoptic, and embodied benchmarks derived from shared scene annotations. SUNCG/SSCNet [1] is an important synthetic indoor-scene dataset and an early precedent for indoor semantic scene completion. Its supervision is centered on semantic scene-completion inputs and voxel labels, whereas Sim2Occ builds a shared scene field with aligned geometry, semantics, and instance identities, supports topology-aware completion and quality-filtered RGB observations, and reuses the same annotations across three occupancy settings. This gives our framework a broader task scope and a direct way to study scale and real-world transfer under a common construction protocol. SUNCG has been subject to a copyright and licensing dispute involving Planner 5D, Princeton, and Facebook, and related releases were withdrawn. We cannot obtain the version required for a matched RGB-based comparison.

We further examine the value of Sim2Occ through two complementary experiments: supervision ablations across three data scales and transfer using occupancy pretraining data generated by our Sim2Occ framework from HSSD synthetic indoor scenes.

1. Contribution of supervision construction. These results are now in Section 4.4, Table 5. We evaluate EmbodiedOcc after pretraining on 3D-FRONT-derived data and fine-tuning on NYUv2, using the same model, data protocol, and training schedule at each scale. Geometry-only pretraining uses binary occupied/free targets. Generic voxelization replaces specialized volumetric construction with surface voxelization and exterior flood filling of enclosed regions, while retaining the same RGB observations and downstream evaluation. The full setting retains the entity-consistent semantic and instance supervision produced by Sim2Occ.

Source sizeSupervisionSC IoU ↑mIoU ↑
20kGeometry-only34.5520.47
Generic voxelization35.7021.34
Sim2Occ36.8123.00
50kGeometry-only36.2321.13
Generic voxelization37.1522.53
Sim2Occ38.2524.08
100kGeometry-only37.1721.43
Generic voxelization37.2322.58
Sim2Occ39.2424.06

Metrics are percentages; bold marks the best score at each scale. Sim2Occ improves mIoU over generic voxelization by 1.66, 1.55, and 1.48, and over geometry-only supervision by 2.53, 2.95, and 2.63, at 20k, 50k, and 100k, respectively. Increasing data scale improves SC IoU for all three supervision settings, while the full construction remains strongest in semantic transfer at every scale. Sim2Occ at 20k also exceeds the mIoU of generic voxelization at 100k (23.00 vs. 22.58), showing the contribution of supervision construction beyond additional observations. The geometry-only row further separates binary occupancy learning from the semantic and instance information carried by the full labels.

2. Generality across synthetic sources. HSSD [2] provides synthetic indoor scenes, which we convert into occupancy pretraining data using Sim2Occ. Through scene conversion, volumetric annotation, and observation filtering, we generate 20k RGB–occupancy pairs. We compare this source with 3D-FRONT at the same 20k scale, using EmbodiedOcc and the same pretraining steps, model initialization, and NYUv2 fine-tuning protocol. Thus, the source comparison changes the synthetic scenes while keeping the downstream transfer procedure fixed.

Scene sourceConstructionObservationsSC IoU ↑mIoU ↑
None——30.6819.78
3D-FRONTSim2Occ20k36.8123.00
HSSDSim2Occ20k37.3222.50

Pretraining on occupancy data generated by Sim2Occ from HSSD scenes improves SC IoU by 6.64 and mIoU by 2.72. The transfer gains from both HSSD and 3D-FRONT support Sim2Occ as a general framework for converting synthetic indoor scenes into occupancy pretraining data that benefits real-world prediction.

We will publicly release our code and all datasets to help fill the gap in synthetic indoor occupancy data and support further research in the occupancy community.

Reviewer_zNPEOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces an automated framework for generating large-scale synthetic supervision for indoor 3D occupancy prediction. The key idea is to construct a unified scene representation with consistent geometry, semantic labels, and instance identities. The framework consists of three components: (1) structured scene composition to enrich indoor environments, (2) entity-consistent volumetric annotation to generate occupancy labels, and (3) room-adaptive camera sampling to produce aligned RGB–occupancy pairs. The authors construct benchmarks for semantic, panoptic, and embodied occupancy tasks. Experiments show consistent improvements when scaling training data from 20k to 100k observations, along with improved real-world transfer.

Critical strengths
  • Scalable supervision framework
    • Provides consistent geometric, semantic, and instance-level annotations across multiple occupancy tasks.
  • Comprehensive evaluation
    • Evaluates five baselines and demonstrates consistent scaling benefits and effective real-world transfer.
Critical weaknesses
  • Insufficient disentanglement of data scaling and methodological contributions
    • While performance improves with increasing synthetic data scale, it remains unclear whether the gains come from the proposed supervision construction or simply more training data. Comparing against standard voxelization at 20k, 50k, and 100k would help isolate the method's contribution.
  • Heavy reliance on heuristics
    • The framework relies heavily on handcrafted heuristics/hyperparameters, whose robustness is not very clear despite good cross-dataset transfer results.
Questions for authors
  • Could the authors compare the scaling trends of the proposed method and standard voxelization at 20k, 50k, and 100k to better isolate the contribution of the proposed method?
  • Are the hyperparameters shared across different benchmarks, or are they tuned separately for each task? And how sensitive is the performance to key hyperparameters, such as camera sampling ratios and coverage weights?
5 comments
AuthorsOpenReview ↗

Response to W1 & Q1

Thank you for suggesting this comparison across data scales. We extended the 20k construction ablation to 50k and 100k and added a geometry-only pretraining baseline. Sim2Occ outperforms generic voxelization at all three data scales in both SC IoU and mIoU, demonstrating a contribution from supervision construction beyond increasing the number of observations.

We evaluate EmbodiedOcc after synthetic pretraining and NYUv2 fine-tuning. To examine the contribution of our supervision construction, we replace the specialized annotation operations with generic voxelization, which directly voxelizes source surfaces and fills enclosed regions through exterior flood filling. Geometry-only pretraining uses binary occupied/free targets without semantic supervision. These results and their analysis are now included in Table 5 and Section 4.4 of the revised manuscript. The results are:

Source sizeSupervisionSC IoU ↑mIoU ↑
20kGeometry-only34.5520.47
Generic voxelization35.7021.34
Sim2Occ36.8123.00
50kGeometry-only36.2321.13
Generic voxelization37.1522.53
Sim2Occ38.2524.08
100kGeometry-only37.1721.43
Generic voxelization37.2322.58
Sim2Occ39.2424.06

Scores are percentages; bold marks the best result at each source scale. Without synthetic pretraining, EmbodiedOcc achieves 30.68 SC IoU and 19.78 mIoU.

Construction and scale. At 20k, 50k, and 100k, Sim2Occ improves over generic voxelization by 1.66, 1.55, and 1.48 mIoU, respectively, with corresponding SC IoU gains of 1.11, 1.10, and 2.01. Increasing the source from 20k to 100k improves SC IoU from 35.70 to 37.23 for generic voxelization and from 36.81 to 39.24 for Sim2Occ, showing the benefit of scale for geometric occupancy prediction. The semantic comparison further highlights the value of our supervision construction: Sim2Occ with 20k observations exceeds the mIoU of generic voxelization with 100k observations (23.00 vs. 22.58). Thus, additional observations improve geometric prediction, while our supervision construction provides an additional advantage in semantic completion.

Contribution beyond binary geometry. Geometry-only pretraining improves over the baseline without synthetic pretraining at every scale. Full Sim2Occ supervision provides further gains of 2.53, 2.95, and 2.63 mIoU, together with 2.26, 2.02, and 2.07 SC IoU. This additional comparison supports the value of semantic supervision for transfer beyond learning binary occupancy.

AuthorsOpenReview ↗

Response to W2 & Q2

Thank you for raising the question of parameter sharing and robustness. The three benchmarks share the default hyperparameters for observation generation. Our semantic, panoptic, and embodied occupancy benchmarks are constructed within the same Sim2Occ framework, with shared scene composition and entity-consistent annotation. Task-specific curation then selects suitable observations: panoptic examples emphasize distinguishable instances, while embodied episodes emphasize cumulative room coverage. The same semantic pretraining sets are also used across downstream models and real-world target datasets.

To assess parameter sensitivity, we vary the camera sampling mixture and the category- and position-coverage weights. We fix the same 80 source rooms and a 20k-observation budget, changing one parameter group at a time. We evaluate EmbodiedOcc after synthetic pretraining and NYUv2 fine-tuning. Both coverage weights are tested at half and twice their default values. We have added this sensitivity analysis to Appendix F.4, Table S14 of the revised manuscript.

Hyperparameter variedTested valueSC IoU ↑mIoU ↑
Camera sampling ratios50/30/2037.0322.64
40/40/20 (default)36.8123.00
30/40/3036.7222.88
Category coverage weight (λc)1.036.5122.04
2.0 (default)36.8123.00
4.036.6022.48
Position coverage weight (λp)0.7536.5522.25
1.5 (default)36.8123.00
3.036.7322.84

The three blocks assess camera sampling ratios, category coverage weights, and position coverage weights, respectively. Within each block, all other parameters retain their defaults: mixture 40/40/20, λc = 2.0, and λp = 1.5. The same default result is repeated for comparison, giving seven distinct configurations. Metrics are in percent; bold marks the best result within each block. Camera ratios denote target proportions of central exploration, moderate-tilt context, and downward context. λc and λp encourage coverage of less-selected anchor categories and camera positions, respectively.

Downstream performance remains relatively stable over the tested parameter ranges. Across all seven configurations, mIoU ranges from 22.04 to 23.00, and SC IoU from 36.51 to 37.03. Changing the camera mixture alters mIoU by at most 0.36 relative to the default. All six parameter variants retain gains of 2.26–3.10 mIoU and 5.83–6.35 SC IoU over training without synthetic pretraining (19.78 mIoU and 30.68 SC IoU). These results show that the transfer benefits persist under changes to the sampling mixture and coverage weights.

Observations per room. Table S12 in Appendix F.3, on room diversity and observation density, also examines the number of observations allocated to each room. At a fixed total budget of 20k observations, using 250, 100, and 50 views per room across 80, 200, and 400 rooms yields 23.00, 22.77, and 22.92 mIoU, respectively. The range is 0.23 mIoU, providing further evidence of stable transfer performance across these observation-allocation settings. With room sets fixed, increasing views per room from 100 to 250 for 200 rooms and from 50 to 250 for 400 rooms brings further gains of 1.31 and 1.14 mIoU, respectively.

Dear Reviewer zNPE,

Thank you again for your thoughtful questions about the contribution of supervision construction and the robustness of our design choices. We would appreciate your further thoughts on the new evidence presented in our rebuttal and revised manuscript.

We extended the supervision comparisons to 20k, 50k, and 100k observations, including both generic voxelization and geometry-only pretraining (Section 4.4, Table 5). Sim2Occ achieves higher SC IoU and mIoU at every scale, and its 20k setting exceeds the semantic transfer performance of generic voxelization at 100k. We also clarified the shared default hyperparameters across the three benchmarks and added sensitivity experiments for camera sampling ratios and coverage weights (Appendix F.4, Table S14). Transfer gains persist across all tested configurations, complemented by the observation-allocation analysis in Appendix F.3, Table S12.

If you have an opportunity to revisit these results before the discussion closes, we would be grateful to hear whether they resolve your main concerns or whether any further clarification would be helpful.

Thank you for your time and careful consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.