acceptodds
Under review as a conference paper at ICLR 2027

Learning Geometric Corrections for Physically Plausible 3D Indoor Scene Generation

Abstract

3D indoor scene generation provides a foundation for virtual reality, interactive gaming, robot simulation, and embodied AI, requiring environments that are both diverse and physically sensible. Although diffusion models have advanced layout synthesis, generated scenes may still contain collisions, boundary violations, scale inconsistencies, and obstructed paths. Physics-guided methods often rely on predefined constraints or search-based refinement, making it difficult to adapt correction decisions to evolving scene states, while jointly correcting position, size, and orientation can cause interference among attribute updates and introduce new spatial violations when resolving local conflicts. We present a framework that learns geometric corrections for physically plausible 3D indoor scene generation. It builds a multi-relational semantic–geometric scene graph, fuses semantic and direction-aware geometric context, and predicts gated corrections for position, size, and orientation through factor-aware branches. Counterfactual perturbations provide supervision, and the predicted corrections are injected into diffusion posterior updates during denoising. Extensive experiments demonstrate that our method achieves leading layout fidelity and collision reduction through state-adaptive, factor-aware geometric correction, jointly advancing generation quality and physical plausibility. Our code is available at https://anonymous.4open.science/status/LGC-1B50.

Primary area:
generative models
Keywords:
3D Indoor Scene Generation, Diffusion Models, Adaptive Geometric Correction, Physical Plausibility

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_M6KVOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a learned geometric correction module for diffusion-based 3D indoor layout generation. At selected denoising steps, it takes the denoised layout estimate and builds three structures: a semantic scene graph encoded by a pretrained GraphTripleConvNet, a scalar–vector geometric graph encoded by GVP, and furniture–wall-segment relations. The two representations are fused, and factor-specific branches predict gated corrections for position, size and orientation. The predictor is trained on counterfactual perturbations of valid 3D-FRONT layouts. At sampling time, the corrections are clipped, rescaled and added to the clean estimate before the DDPM posterior update. Experiments on 3D-FRONT bedrooms, dining rooms and living rooms report the best FID and slightly lower collision rates than DiffuScene, MiDiffusion and PhyScene.

Critical strengths

1 Replacing hand-designed physical guidance with a learned, state-dependent correction predictor is a reasonable direction. Splitting the correction into gated position, size and orientation branches is a sensible design, and the ablation (Table 3) shows that the full encoder combined with factor-aware correction works best. 2 The paper is detailed and reproducible. The appendix specifies graph construction, features, the counterfactual dataset and the training stages, and code is released. The guidance-scheduling study (Fig. 4) also gives useful insight into when and how strongly to apply correction.

Critical weaknesses

1 The physical improvements are small, and generation remains far from physically plausible. Around 78–83% of scenes and about half of all objects still collide. Collision rates improve over PhyScene by only 1–3 points, OOB-obj is worse than PhyScene in all three room types, and no seed variance is reported, so it is unclear whether the gains are significant. 2 Key baselines and broader evaluation are missing. The paper does not compare with: simple post-hoc collision and boundary optimization; PhyScene-style guidance applied to the same backbone; 3 All experiments are on a single dataset (3D-FRONT), which is synthetic.

Questions for authors

See weakness.

4 comments

We thank the reviewer for raising these concerns about physical plausibility, baseline comparisons, and dataset scope. We address each point below.

Respond to question 1 : Improvements are small , bedroom CKL is the worst of all methods, OOB-obj is worse than PhyScene, and no seeds/variance are reported.

11 The collision metrics.

Our collision checks use cuboid bounding boxes rather than the furniture meshes themselves ; Therefore, an intersection between bounding boxes does not necessarily indicate a collision between furniture objects. To complement this analysis, **we report the mean penetration depth and normalized embedding ratio for colliding pairs, along with their distributions **(Tables 1–3) , and include an additional analysis of collision severity and boundary violations in Appendix D.

Table 1.Penetration depth

PhySceneOurs
Mean depthcmcm↓3.560±1.1812.273±1.434

Table 2.Normalized embedding

PhySceneOurs
Mean embedding ratio%↓5.495±1.1992.745±1.150

Table 3.Embedding-ratio distribution

Colliding pairs(%)PhysceneOurs
<5%81.5489.70
5-10%9.284.17
>10%9.186.13

22The CKL metric.

CKL measures the agreement between the category distribution of generated furniture and that of the real data. Our method primarily optimizes geometric relationships and does not directly optimize category-distribution matching. Because geometric corrections are integrated into the joint denoising process, CKL may be affected by their indirect influence on subsequent category predictions, as well as by fluctuations in category frequencies due to the limited sample size.

33The object-level out-of-boundsOOBOOB rate.

Our OOB-obj performance is worse than PhyScene. However, our method reduces the rate of more substantial boundary violations: at violation-area thresholds of 5% and 10%, the rates are 11.55% and 8.39%, respectively, compared with 19.67% and 13.38% for PhyScene. We have added this analysis to Appendix D, “Additional Analysis of Collision Severity and Boundary Violations.”

Objects exceeding threshold%↓PhySceneOurs
2%35.4736.31
5%19.6711.55
10%13.388.39

44The random seeds and variability.

The main text reports only the average results across random seeds. To further assess variability and stability, we provide additional results across random seeds in table below.

bedroomFID↓KID↓SCACKL↓Col-pair↓Col-obj↓Col-scene↓OOB-obj↓Overflow↓
42.527.5665.631.4913.6746.8277.9335.423.69
42.547.7065.781.5813.7547.2778.4635.493.48
43.197.6366.321.3913.7148.1878.6834.864.15
42.637.6165.811.5013.8647.5179.1836.274.76
43.077.8566.061.6414.1647.3279.2936.314.97
Mean ± SD42.790 ± 0.3167.670 ± 0.11265.920 ± 0.2721.520 ± 0.09513.830 ± 0.19847.420 ± 0.49578.670 ± 0.62335.670 ± 0.6174.210 ± 0.649
Dining roomFID↓KID↓SCACKL↓Col-pair↓Col-obj↓Col-scene↓OOB-obj↓Overflow↓
30.594.3766.320.5413.3147.6682.4934.834.16
30.744.6866.790.5914.2047.7282.6735.604.24
31.155.1267.370.6515.5448.6983.8936.254.53
32.285.9267.540.6114.1747.3183.0735.724.32
32.496.0168.180.7915.2848.8283.5336.054.60
Mean ± SD31.450 ± 0.8815.220 ± 0.73167.240± 0.7140.636 ± 0.09514.500 ± 0.90948.040 ± 0.67383.130 ± 0.58435.690 ± 0.5464.370 ± 0.188
Living RoomFID↓KID↓SCACKL↓Col-pair↓Col-obj↓Col-scene↓OOB-obj↓Overflow↓
30.5310.7376.930.4714.3147.9481.7934.433.47
31.0811.7277.360.4114.3447.8382.8334.924.35
30.8411.3577.850.5314.7348.5183.1935.184.59
31.6910.8478.120.3515.0649.1583.0735.474.58
31.7112.0178.340.4915.8149.4283.4235.755.16
Mean ± SD31.170 ± 0.52211.330 ± 0.55177.720 ± 0.5730.450 ± 0.07114.850 ± 0.61948.570 ± 0.70882.860 ± 0.63535.150 ± 0.5094.430 ± 0.614

Respond to question 2 : Key baselines and broader evaluation are missing. The paper does not compare with: simple post-hoc collision and boundary optimization; PhyScene-style guidance applied to the same backbone

11The same backbone.

DiffuScene serves as the same backbone for these comparisons. As described in Section 3.3, we add the predicted geometric corrections to the denoised layout estimate before the posterior update. Disabling the corrector recovers the original DiffuScene sampling procedure . Accordingly, DiffuScene, PhyScene, and Ours correspond to the DiffuScene backbone alone, the backbone with training-free physical guidance, and the backbone with our learned corrector, respectively. Table 2 reports results for these settings using the same backbone.

22Regarding post-hoc optimization.

Post-hoc layout optimization is an early line of work on indoor furniture layout [1]. Prior work has noted that collision and room-boundary constraints may be in tension, and that final-stage corrections may not resolve errors accumulated during generation [2,3]. We also discuss the potential limitations of such approaches in the manuscript:

“When corrections are computed using explicitly formulated rules or objectives, their effectiveness depends on the constraints and interactions captured by these formulations. If relevant interactions are not adequately accounted for, correcting one relation may introduce conflicts elsewhere.”

[1]Interactive furniture layout using interior design guidelines[J]. ACM transactions on graphics TOGTOG, 2011, 3044: 1-10.

[2]Physcene: Physically interactable 3d scene synthesis for embodied ai[C]//2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024: 16262-16272.

[3]ScenePilot: Controllable Boundary-Driven Critical Scenario Generation for Autonomous Driving[J]. arXiv preprint arXiv:2605.21168, 2026.

Respond to question 3 : All experiments are on a single dataset 3D−FRONT3D-FRONT, which is synthetic.

We use 3D-FRONT because it is widely used in this field and enables comparisons with prior methods under a common benchmark. On other public datasets, the data or code needed to run prior methods are often unavailable, making consistent comparisons difficult. The table below summarizes the datasets used in recent related work.

3D-FRONTSUNCGStructured3DScanNetcitation count
ATISS: Autoregressive Transformations for Indoor Scene SynthesisNips 2021√384
SceneFormer: Indoor Scene Generation with Transformers3DV 2021√291
Scene Synthesis via Uncertainty-Driven Attribute SynchronizationICCV 2021√√70
DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis<br>CVPR 2024√356
PhyScene: Physically Plausible 3D Scene Synthesis for Indoor ScenesCVPR 2024√203
DiffInDScene: Diffusion-based High-Quality 3D Indoor Scene GenerationCVPR 2024√✓¹40
From Programs to Poses: Factored Real-World Scene Generation via Learned Program LibrariesNeurIPS 2025✓²✓²7
Ctrl-Room: Controllable Text-to-3D Room Meshes Generation with Layout Constraints3DV 2025✓³98
Mixed diffusion for 3d indoor scene synthesisWACV 2026√46
SemLayoutDiff: Semantic Layout Generation with Diffusion Model for Indoor Scene Synthesis3DV 2026√12
Global Graph-Validated Optimization for<br>VLM-based 3D Indoor Scene GenerationECCV 2026√
InSpace: Structure-Aware 3D Indoor Scene<br>Generation from a Single 360° ImageECCV 2026√

Dear Reviewer,

Thank you again for your time and thoughtful feedback. As the discussion period is drawing to a close, we would greatly appreciate your feedback on whether our responses have adequately addressed your concerns. If any questions remain, we would be happy to provide further clarification.

If our responses have resolved your main concerns, we would be grateful if you could consider revisiting your score in light of these clarifications. Thank you for your time and consideration.

Reviewer_qL23OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a learned geometric-correction module for diffusion-based 3D indoor scene generation. During denoising, the predicted clean layout is encoded as a semantic graph and a direction-aware geometric graph, including furniture-wall relations. Factor-specific branches then predict gated per-object corrections. These corrections are added to the clean-layout estimate and used in the DDPM posterior update. The branches are trained on synthetic counterfactual layouts, which are perturbed 3D-FRONT scenes with the original layout as the target. On 3D-FRONT bedrooms, dining rooms and living rooms, the method improves FID and collision metrics over ATISS, DiffuScene, MiDiffusion, PhyScene and a GPT-based baseline. Ablations cover the encoders, the factor-aware design and the guidance schedule.

Critical strengths
  1. Well-motivated and carefully engineered approach. Replacing hand-crafted constraints/gradient guidance with a scene-conditioned, factor-aware learned correction applied during denoising is a sensible direction. The direction-aware geometric representation (scalar-vector features plus explicit furniture-wall relations) is described in enough detail in the appendix to be reproducible, and code is provided.

  2. Consistent, though modest, reduction of collision rates (Col-pair/Col-obj/Col-scene) over all baselines in all three room types (Table 2), together with an ablation indicating that the joint semantic+geometric encoding and the factor-aware heads contribute to this effect (Table 3). The limitations section also candidly acknowledges the supervision mismatch.

Critical weaknesses
  1. Key claim not isolated by controlled baselines. There is no backbone-only vs. backbone+corrector comparison, nor comparison with training-free guidance (e.g., PhyScene-style) or SPREAD on the same backbone.

  2. Improvements are small (Col-scene still ~79%, living-room FID +1.3%), bedroom CKL is the worst of all methods, OOB-obj is worse than PhyScene, and no seeds/variance are reported.

Questions for authors

I would like the authors to address the three weaknesses above.

3 comments

We thank the reviewer for raising these concerns. We address each point below.

Respond to question 1 :Controlled baseline comparisons on the same backbone.

11The same backbone.

DiffuScene serves as the same backbone for these comparisons. As described in Section 3.3, we add the predicted geometric corrections to the denoised layout estimate before the posterior update. Disabling the corrector recovers the original DiffuScene sampling procedure.

22Comparison of Backbone Only, Backbone + Corrector, and Training-Free Guidance.

DiffuScene, PhyScene, and Ours in Table 2 all use the DiffuScene backbone, corresponding to the backbone alone, the backbone with training-free physical guidance, and the backbone with our learned corrector, respectively .

Our method improves generation quality and the physical-plausibility metrics across all three room types. These results support our central claim: with the same DiffuScene backbone, learning scene-state-dependent, attribute-factorized geometric corrections can reduce collisions more effectively than using no correction or existing training-free physical guidance, while also improving the distributional fidelity of generated layouts.To clarify the baseline configurations and our central claim, we have added further experimental details to the revised manuscript (Lines 320–323).

Respond to question 2 : Improvements are small , bedroom CKL is the worst of all methods, OOB-obj is worse than PhyScene, and no seeds/variance are reported.

11 The collision metrics.

Our collision checks use cuboid bounding boxes rather than the furniture meshes themselves; therefore, an intersection between bounding boxes does not necessarily indicate a collision between furniture objects. To complement this analysis, we report the mean penetration depth and normalized embedding ratio for colliding pairs, along with their distributions(Tables 1–3) , and include an additional analysis of collision severity and boundary violations in Appendix D.

Table 1.Penetration depth

PhySceneOurs
Mean depthcmcm↓3.560±1.1812.273±1.434

Table 2.Normalized embedding

PhySceneOurs
Mean embedding ratio%↓5.495±1.1992.745±1.150

Table 3.Embedding-ratio distribution

Colliding pairs(%)PhysceneOurs
<5%81.5489.70
5-10%9.284.17
>10%9.186.13

22The CKL metric.

CKL measures the agreement between the category distribution of generated furniture and that of the real data. Our method primarily optimizes geometric relationships and does not directly optimize category-distribution matching. Because geometric corrections are integrated into the joint denoising process, CKL may be affected by their indirect influence on subsequent category predictions, as well as by fluctuations in category frequencies due to the limited sample size.

33The object-level out-of-boundsOOBOOB rate.

Our OOB-obj performance is worse than PhyScene. However, our method reduces the rate of more substantial boundary violations: at violation-area thresholds of 5% and 10%, the rates are 11.55% and 8.39%, respectively, compared with 19.67% and 13.38% for PhyScene. We have added this analysis to Appendix D, “Additional Analysis of Collision Severity and Boundary Violations.”

Objects exceeding threshold%↓PhySceneOurs
2%35.4736.31
5%19.6711.55
10%13.388.39

44The random seeds and variability.

The main text reports only the average results across random seeds. To further assess variability and stability, we provide additional results across random seeds in table below.

bedroomFID↓KID↓SCACKL↓Col-pair↓Col-obj↓Col-scene↓OOB-obj↓Overflow↓
42.527.5665.631.4913.6746.8277.9335.423.69
42.547.7065.781.5813.7547.2778.4635.493.48
43.197.6366.321.3913.7148.1878.6834.864.15
42.637.6165.811.5013.8647.5179.1836.274.76
43.077.8566.061.6414.1647.3279.2936.314.97
Mean ± SD42.790 ± 0.3167.670 ± 0.11265.920 ± 0.2721.520 ± 0.09513.830 ± 0.19847.420 ± 0.49578.670 ± 0.62335.670 ± 0.6174.210 ± 0.649
Dining roomFID↓KID↓SCACKL↓Col-pair↓Col-obj↓Col-scene↓OOB-obj↓Overflow↓
30.594.3766.320.5413.3147.6682.4934.834.16
30.744.6866.790.5914.2047.7282.6735.604.24
31.155.1267.370.6515.5448.6983.8936.254.53
32.285.9267.540.6114.1747.3183.0735.724.32
32.496.0168.180.7915.2848.8283.5336.054.60
Mean ± SD31.450 ± 0.8815.220 ± 0.73167.240± 0.7140.636 ± 0.09514.500 ± 0.909<br>48.040 ± 0.67383.130 ± 0.58435.690 ± 0.5464.370 ± 0.188
Living RoomFID↓KID↓SCACKL↓Col-pair↓Col-obj↓Col-scene↓OOB-obj↓Overflow↓
30.5310.7376.930.4714.3147.9481.7934.433.47
31.0811.7277.360.4114.3447.8382.8334.924.35
30.8411.3577.850.5314.7348.5183.1935.184.59
31.6910.8478.120.3515.0649.1583.0735.474.58
31.7112.0178.340.4915.8149.4283.4235.755.16
Mean ± SD31.170 ± 0.52211.330 ± 0.55177.720 ± 0.5730.450 ± 0.07114.850 ± 0.61948.570 ± 0.70882.860 ± 0.63535.150 ± 0.5094.430 ± 0.614

Thank you for the authors' responses. The additional experiments have addressed most of my concerns. I have no further questions. I hope these experiments can be included in the final version of the paper. I will keep my current score.

Reviewer_P4mnOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper learns geometric corrections for diffusion-based indoor scene generation. A network combines semantic and geometric graphs with separate position, size, and orientation branches. It is trained to recover filtered 3D-FRONT layouts from synthetic perturbations and applied during sampling with a frozen diffusion backbone. Experiments on three room types report lower FID and collision rates.

Critical strengths
  • The proposed representation combines semantic relations, directional geometry, and room boundaries to inform object-wise corrections.
  • Factor-specific corrections can be integrated without retraining the generator. The reported FID improvements are consistent across three room types.
Critical weaknesses
  • The perturbations may leave a scene plausible, yet the training target requires recovering the original arrangement. The paper does not establish that this recovery objective reliably improves physical validity while preserving layout diversity.
  • The gains over PhyScene are small, with only small reductions in scene collision rates and inconsistent boundary improvements. The evidence for an overall improvement in physical plausibility is limited.
Questions for authors
  • Indoor scenes can have multiple physically valid arrangements, and small furniture movements may preserve validity. However, Appendix C defines correction targets and factor labels by deviations from the original layout. Could the authors clarify the reason for using recovery of the original layout as the correction target, and how this objective accommodates valid alternative arrangements?

  • The GPT-6 Astra examples in Figure 3 appear to use furniture constructed in Blender rather than retrieved assets. Could the authors clarify how differences in asset sources are controlled in the rendering-based evaluation, so that the comparison reflects layout quality rather than asset geometry?

3 comments

We thank the reviewer for raising these important questions.

Respond to weakness 1 and question 1 : Could the authors clarify the reason for using recovery of the original layout as the correction target, and how this objective accommodates valid alternative arrangements?

Our goal is to learn transferable correction rules conditioned on the geometric relationships in the current scene. We treat the original layout as a particular valid solution, rather than a general solution. The original layout serves as a supervision reference, allowing us to construct paired examples with clearly defined correction attributes, directions, and magnitudes. For collision and out-of-bounds perturbations, we apply changes that exacerbate local conflicts and use the inverse corrections to provide directional supervision for alleviating them. Thus, recovering the reference layout is a means of obtaining correction supervision.

At inference time,the model uses the learned correction rules conditioned on the geometric relationships in the current scene without reference to the original layout to determine an adjustment direction and produce a potentially physically plausible layout.

Respond to weakness 2 : The gains over PhyScene are small, with only small reductions in scene collision rates and inconsistent boundary improvements. The evidence for an overall improvement in physical plausibility is limited.

11 The collision metrics.

Our collision checks use cuboid bounding boxes rather than the furniture meshes themselves; therefore, an intersection between bounding boxes does not necessarily indicate a collision between furniture objects. To complement this analysis, we report the mean penetration depth and normalized embedding ratio for colliding pairs, along with their distributions (Tables 1–3) , and include an additional analysis of collision severity and boundary violations in Appendix D.

Table 1.Penetration depth

PhySceneOurs
Mean depthcmcm↓3.560±1.1812.273±1.434

Table 2.Normalized embedding

PhySceneOurs
Mean embedding ratio%↓5.495±1.1992.745±1.150

Table 3.Embedding-ratio distribution

Colliding pairs(%)PhysceneOurs
<5%81.5489.70
5-10%9.284.17
>10%9.186.13

22The object-level out-of-boundsOOBOOB rate.

Our OOB-obj performance is worse than PhyScene.However, our method reduces the rate of more substantial boundary violations: at violation-area thresholds of 5% and 10%, the rates are 11.55% and 8.39%, respectively, compared with 19.67% and 13.38% for PhyScene. We have added this analysis to Appendix D, “Additional Analysis of Collision Severity and Boundary Violations.”

Objects exceeding threshold%↓PhySceneOurs
2%35.4736.31
5%19.6711.55
10%13.388.39

Respond to question 2 : Could the authors clarify how differences in asset sources are controlled in the rendering-based evaluation?

For the results in Table 2 ,to ensure that the comparison reflects layout quality rather than differences in asset geometry, we compute the metrics using oriented bounding boxes rather than collisions between furniture meshes. Thus, mesh surface details and appearance differences do not directly enter the physical-conflict metrics. This evaluation therefore reflects the quality of the generated layouts and is largely unaffected by differences in asset appearance.

For the results in Table 1 ,to minimize the impact of differences in asset sources, we use the same Blender rendering settings for all methods: 256 × 256 resolution, an orthographic top-down view, category-based furniture colors, and disabled textures. We also specify the room style in the prompts for GPT-6 Astra.

Dear Reviewer,

Thank you again for your time and thoughtful feedback. As the discussion period is drawing to a close, we would greatly appreciate your feedback on whether our responses have adequately addressed your concerns. If any questions remain, we would be happy to provide further clarification.

If our responses have resolved your main concerns, we would be grateful if you could consider revisiting your score in light of these clarifications. Thank you for your time and consideration.

Thanks for the authors' responses. My concern has been addressed, and I will increase my score accordingly.

Comments on the paper

2 comments

Dear Area Chair,

We have submitted point-by-point responses to reviewers P4mn and M6KV, but have not yet received further comments. As the rebuttal period is nearing its end, we would like to know whether our responses have adequately addressed the reviewers’ concerns.

Best regards,

The authors

Dear Area Chair,

The other two reviewers have confirmed that our responses addressed their concerns. We have not yet heard back from Reviewer M6KV, and we would like to know whether our responses have addressed their concerns as well, or whether further clarification is needed.

As the discussion period is coming to a close, we would appreciate it if you could encourage Reviewer M6KV to follow up.

Thank you again for your time and assistance.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.