acceptodds
Under review as a conference paper at ICLR 2027

SceneBloom: Agentic Indoor Environment Generation with Realistic and Diverse Layouts for Embodied Learning

Abstract

Training generalizable embodied agents requires realistic, varied, and simulation-ready indoor environments. Language-guided methods support semantic planning and object-level reasoning, but precise spatial grounding remains difficult. Visual generative models offer rich priors over plausible arrangements, yet their outputs require geometric reasoning to become physically valid 3D scenes. We introduce SceneBloom, an agentic framework that combines visual layout sampling with contract-guided 3D compilation. A hierarchical sampler uses diffusion priors to generate furniture arrangements within functional zones. A compiler then uses relations extracted from each proposal to guide geometric fitting and local pose correction. Across five room types, SceneBloom attains the highest visual quality and physical stability among the compared methods and is the only method with no detected mesh collisions or out-of-bounds objects. Its layouts are also closer to the distribution of real rooms than those of most baselines. Ablations show that visual proposals outperform textual proposals in both physical plausibility and visual quality, and that contract guidance further improves layout coherence and realism. Fine-tuning an object-navigation policy on these scenes improves its success on unseen goals in both generated and real environments, and raising scene difficulty through SceneBloom further improves success in real environments.

Primary area:
applications to robotics, autonomy, planning
Keywords:
embodied AI, 3D scene generation, indoor scene synthesis, diffusion models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_kRN8OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SceneBloom generates indoor scenes by dividing a room into functional zones, sampling an image of each zone using diffusion models conditioned on specific assets, and converting those proposals into 3D layouts. Its main contribution is a geometric compiler that resolves overlaps while preserving image-derived relations such as facing, alignment, and proximity. Validation includes comparisons across five room types, controlled component ablations, collision and dynamic-stability tests, navigation-connectivity measurements, and downstream object-navigation experiments.

Critical strengths

The strengths of the study are in two aspects. First, the paper demonstrates the usefulness of its generated scenes through object-navigation training and evaluation on held-out generated and real-scanned environments. The controlled experiments with added occluders further show how scene generation can supply task-relevant training variation. Although the gains are modest, this provides practical evidence of value beyond visual quality and scene-generation metrics. Second, the study separating visual layout proposals from geometric correction allows the system to repair placements while preserving intended spatial relationships. With inventories and assets fixed, contract guidance reduces relation violations from 37.45% to 4.25% and improves visual coherence. Together with zero detected collisions in the main evaluation and approximately 2.7 minutes per layout, these results support the approach’s effectiveness and efficiency.

Critical weaknesses

My major concens are as following aspects:

  1. The framework’s benefit beyond foundation-model capability remains incompletely established. The controlled ablations demonstrate improvements from visual proposals and relation contracts, but only under the tested model configuration. They do not establish whether these benefits persist with stronger foundation models, or whether comparable performance could be achieved through better prompting and in-context examples with equivalent information and tool access. This leaves uncertainty about the proposed components’ contribution as the underlying models improve.
  2. The reliability of the generated relations and physical arrangements is insufficiently validated. Contract satisfaction is measured against the VLM’s own extracted relations, without independently verifying whether those relations are correct or complete. Furthermore, despite zero detected collisions, only 75.3% of movable objects pass the stability test. Containment and under-object arrangements are not explicitly addressed, while navigation connectivity does not establish functional clearances for opening cabinets or using furniture. These gaps limit confidence in the broader claim of producing physically valid, simulation-ready environments.
  3. Limited scene categories are tested. I would love to see the performance on more scene cases beyoud of living room.
Questions for authors

I have the following questions:

  1. Table 11 provides useful evidence for visual proposals and contract guidance under the tested configuration. Does their advantage persist when the textual planner is replaced with a stronger foundation model? A controlled comparison would help establish whether the proposed design provides benefits beyond improved prompting or in-context learning. Can you distinguish the contribution of the pretrained visual prior from that of the contract-guided compiler. 2.The reduction in contract violations from 37.45% to 4.25% demonstrates agreement with the extracted contracts, but does not establish whether those contracts correctly describe the intended arrangement. Have the authors independently evaluated incorrect or omitted relations, especially for occluded objects and ambiguous facing directions? How was the confidence threshold of 0.75 selected, and does confidence correspond to actual extraction accuracy?
  2. Table 4 contains five relations but no explicit “inside” or “under” relation. Can the current compiler represent valid arrangements with overlapping ground-plane footprints, such as a chair tucked under a table or an object inside a cabinet? Please distinguish unsupported arrangements from those handled through separate placement rules. Similarly, beyond navigation-mesh connectivity, does the method enforce clearances needed to open cabinets, pull out chairs, or approach furniture for use?
  3. Why use GPT-4O as the evaluator instead of using more advanced agent such as GPT-6?
  4. How does commonly failure case look like? can you also report the runtime vs. number of objects / funtional zones in the scene?
10 comments
AuthorsOpenReview ↗

Response to Reviewer kRN8 (Part 1/5)

We thank the reviewer for the careful reading and detailed feedback. We have revised the paper accordingly and addressed each of the raised weaknesses and questions in detail below. All revisions are marked in blue in the revised manuscript.

W1 & Q1a Component contributions with stronger foundation models

W1: The framework’s benefit beyond foundation-model capability remains incompletely established. The controlled ablations demonstrate improvements from visual proposals and relation contracts, but only under the tested model configuration. They do not establish whether these benefits persist with stronger foundation models, or whether comparable performance could be achieved through better prompting and in-context examples with equivalent information and tool access. This leaves uncertainty about the proposed components’ contribution as the underlying models improve.

Q1a: Table 11 provides useful evidence for visual proposals and contract guidance under the tested configuration. Does their advantage persist when the textual planner is replaced with a stronger foundation model? A controlled comparison would help establish whether the proposed design provides benefits beyond improved prompting or in-context learning. Can you distinguish the contribution of the pretrained visual prior from that of the contract-guided compiler.

Response

Thank you for suggesting this comparison. We extended the Table 14 ablation, which uses Qwen3-VL-8B-Instruct, to GPT-6 Sol, comparing textual and visual proposals with and without contract guidance. Although GPT-6 Sol textual proposals combined with our contract guidance achieve higher functionality and coherence scores (Table R2), both proposed components retain their advantages with the stronger model. Visual proposals retain their geometric-diversity advantage, and contract guidance gives the closest alignment with real-room layouts by preserving their spatial organization during compilation (Table R1). We have added this comparison to Appendix D.2 (Figure 11).

1. Visual proposals provide geometric diversity through the pretrained diffusion prior. The visual stage draws on this prior to generate varied spatial arrangements. Visual proposals achieve higher geometric diversity than textual proposals under both Qwen3-VL-8B and GPT-6 Sol, with and without contract guidance. The relative gains range from 5% to 13%, with all four paired 95% confidence intervals above zero. Across the evaluated configurations, GPT-6 Sol produces lower geometric diversity than Qwen3-VL-8B, illustrating that stronger textual planning does not automatically yield more diverse layouts.

2. Contract guidance preserves and recovers the proposal’s spatial relations during compilation. These constraints help retain the proposed arrangement as the compiler resolves geometric conflicts. Within the visual branch, enabling contract guidance reduces MP3D LDD by 0.024 under Qwen3-VL-8B and 0.021 under GPT-6 Sol, with both paired confidence intervals below zero, while geometric diversity remains nearly unchanged. This supports its role in preserving spatial organization that better aligns with real-room layouts.

Table R1: Geometric diversity and layout-distribution alignment across foundation-model configurations. Geometric diversity measures pairwise layout variation within each generation condition, averaged over five room types. MP3D LDD measures distance to real-room layout distributions. Bold indicates the best value within each backbone.

BackboneProposalContractGeometric diversity ↑MP3D LDD ↓
Qwen3-VL-8BTextOff.351.1863
Qwen3-VL-8BTextOn.370.1796
Qwen3-VL-8BVisualOff.388.1963
Qwen3-VL-8BVisualOn.389.1727
GPT-6 SolTextOff.294.1894
GPT-6 SolTextOn.290.1899
GPT-6 SolVisualOff.327.1955
GPT-6 SolVisualOn.329.1746

Table R2: Judged scene quality across foundation-model configurations. All configurations within each backbone use the same GPT-4o evaluation rubric. Bold indicates the best value within each backbone.

BackboneProposalContractRealism ↑Functionality ↑Coherence ↑Completeness ↑
Qwen3-VL-8BTextOff5.445.474.293.80
Qwen3-VL-8BTextOn5.505.594.373.93
Qwen3-VL-8BVisualOff5.665.654.314.05
Qwen3-VL-8BVisualOn5.835.854.654.07
GPT-6 SolTextOff6.426.805.524.26
GPT-6 SolTextOn6.567.206.184.30
GPT-6 SolVisualOff6.306.625.284.10
GPT-6 SolVisualOn6.566.825.544.32
AuthorsOpenReview ↗

Response to Reviewer kRN8 (Part 2/5)

W2a & Q1b Relation contract accuracy

W2a: The reliability of the generated relations and physical arrangements is insufficiently validated. Contract satisfaction is measured against the VLM’s own extracted relations, without independently verifying whether those relations are correct or complete. ...

Q1b: The reduction in contract violations from 37.45% to 4.25% demonstrates agreement with the extracted contracts, but does not establish whether those contracts correctly describe the intended arrangement. Have the authors independently evaluated incorrect or omitted relations, especially for occluded objects and ambiguous facing directions? How was the confidence threshold of 0.75 selected, and does confidence correspond to actual extraction accuracy?

Response

We verify the extracted contracts in two approaches. Manual inspection shows that the relations extracted by Qwen3-VL-8B-Instruct, the model used by our method, correctly describe the proposals with few omissions. A second model, GPT-6 Sol, independently extracts relations from the same proposals and confirms that the compiled layouts satisfy the proposed arrangement. We have added the manual verification and the GPT-6 Sol evaluation to Appendix C.2 (Table 8).

1. Manual verification confirms accurate extraction with few omitted relations.

  • Qwen3-VL-8B and GPT-6 Sol extract relations with high precision. We sampled 100 distinct relations covering different relation types, including relations extracted by either model alone and by both models. We manually checked them against the proposal images without seeing which model extracted them.

Table R1: Manual verification of sampled relations. The two rows share 35 relations, giving 100 distinct relations in total. Precision is weighted to account for the sampling proportions.

Extracting modelRelations checkedPrecision on judgeable relations ↑
Qwen3-VL-8B, used by our method6795.8%
GPT-6 Sol6899.0%
  • Qwen3-VL-8B also extracts accurate relations under challenging visual conditions. We checked 30 cases selected for occlusion or oblique viewing directions. The two incorrect relations concern side-view chairs facing desks.

Table R2: Precision under occlusion and oblique viewing directions.

Targeted conditionCasesRelationsPrecision on judgeable relations ↑
Occlusion1521100%
Oblique viewing direction152190.0%
Total304294.9%
  • Few clearly visible relations are omitted. In 20 randomly sampled zones, we identified 46 correct Qwen3-VL-8B relations and five omissions, giving 90.2% recall. All five omissions are beside relations. Three already have a move_near constraint on the same object pair, preserving proximity information that guides their placement.

2. An independent relation set from GPT-6 Sol confirms that compilation realizes the proposed arrangement.

For each proposal, GPT-6 Sol extracts its own relation set from the same image. Compilation uses only the Qwen3-VL-8B contract, and the GPT-6 Sol relations serve only to evaluate the final layouts.

With the contract, violations of the Qwen3-VL-8B relations fall from 37.45% to 4.25%, and violations of the independent GPT-6 Sol relations also fall from 25.89% to 4.09%. The manual verification and this independent evaluation therefore show that the contracts capture valid spatial relationships and help realize them in the compiled layouts.

Table R3: Relation violations in the compiled layouts.

Compilation settingViolations of Qwen3-VL-8B relations ↓Violations of GPT-6 Sol relations ↓
Without a relation contract37.45%25.89%
With the Qwen3-VL-8B contract4.25%4.09%

3. The measured violation rates are insensitive to the confidence threshold within the tested range.

A sensitivity analysis over γ ∈ [0.60, 0.85] gives unchanged violation rates, so the results do not depend on the choice of 0.75. The manual checks above directly confirm that the extracted relations are correct.

AuthorsOpenReview ↗

Response to Reviewer kRN8 (Part 3/5)

W2b Object stability and motion

W2b: ...despite zero detected collisions, only 75.3% of movable objects pass the stability test. ...

Response

Thank you for pointing this out. Because the overall 75.3% score mixes large and small objects, we report stability by object size, which shows where the remaining instability occurs. SceneBloom’s large objects remain nearly stationary, and its small objects also achieve higher stability and less motion than all baselines. We have added the results to Appendix C.1 (Table 6).

1. Large objects achieve 96.8% stability with minimal movement.

This exceeds the 69.2–74.4% stability of the baselines. Large objects move just 1.1 mm at the median, and only 0.3% move more than 0.2 m, compared with 15.9–20.3% for the baselines.

Table R1: Stability and motion of large objects.

MethodStability ↑Mean displacement (m) ↓Median displacement (m) ↓Moved >0.2 m ↓
Holodeck69.2%0.15830.010618.0%
SceneWeaver73.5%0.20120.007020.3%
SAGE74.4%0.85000.011615.9%
SceneBloom96.8%0.01140.00110.3%

2. Small objects also show substantially higher stability and less motion.

SceneBloom achieves the highest small-object stability and the lowest motion on every reported metric. The remaining failures are concentrated among small objects placed on furniture. Our diagnostic analysis associates instability with thin or slender shapes, suggesting that susceptibility to tipping contributes to these failures even when initial placements are collision-free.

Table R2: Stability and motion of small objects.

MethodStability ↑Mean displacement (m) ↓Median displacement (m) ↓Moved >0.2 m ↓
Holodeck18.0%1.10940.673472.6%
SceneWeaver33.0%0.44370.104643.0%
SAGE38.2%1.00500.110142.1%
SceneBloom66.6%0.13510.015513.4%

W2c & Q2a Support for under-object and containment arrangements

W2c: ...Containment and under-object arrangements are not explicitly addressed, ...

Q2a: ...Please distinguish unsupported arrangements from those handled through separate placement rules. ...

Response

Thank you for pointing out these two common arrangements. We extended the compiler with under relations and placement inside articulated storage objects. We have added both extensions to Appendix A.5 (Figure 7 illustrates both arrangements).

1. Under relations specify which objects should be placed beneath others. We add the under relation to the relation contract alongside the five relations in Table 4. The VLM extracts object pairs from the visual proposal, and the compiler adjusts their placement while checking mesh collisions. For example, a chair can be moved beneath its associated table. GPT-6 Sol independently evaluated whether the executed chair–table placements satisfied the intended relation, confirming 90.9%.

2. Inside placement extends the existing small-object pipeline. The VLM selects contents for articulated storage objects, while the compiler identifies internal compartments and supporting surfaces. Objects are placed within these compartments with the doors closed. Per-link convex decomposition preserves the interior free space during physics simulation.

3. Both extensions preserve physical quality. With the same proposals and original relation contracts, existing furniture stability remained unchanged in both comparisons. Neither extension introduced detected mesh collisions, out-of-bounds objects, or additional violations of the original relation contracts.

These extensions demonstrate how the compiler can incorporate additional spatial arrangements through relation extraction and placement procedures.

Reviewer_E6UfOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes SceneBloom, which first use a VLM for functional zoning, then use a diffusion model with asset LoRAs to generate layout proposals for each functional zone; afterward, lift the proposals to 3D and extract relational contracts, and a geometry compiler fits the layout, repairs relations, and completes objects. Overall, it is a scene generation system that combines visual layout priors with deterministic 3D scene compilation.

Critical strengths
  1. The paper connects visual layout proposals, relation extraction, geometric compilation, and object completion, and also reports stability, real layout distributions, and per-stage runtime. As a scene generation system, the pipeline and implementation information are relatively complete.

  2. The authors use the generated rooms for navigation training and also test real-environment performance.

Critical weaknesses
  1. The code shows that the compiler actively separates objects, leaving a default spacing of 0.05m, out-of-bounds objects are moved back into the room, and if space is insufficient, the room is expanded up to 12m. Therefore, zero collisions and zero out-of-bounds indicate that the final output has been repaired to be feasible, but do not by themselves prove that the initial layout is more reasonable. The paper also does not report collision/relation metrics before and after post-processing, object displacement, or room expansion.

  2. In table 3, Real-environment SR goes from 48.26 to 48.57, only 0.31 percentage points higher. In table 2, SPL even slightly decreases from 21.18 to 21.17. SPL on seen targets rises from 61.31 to 64.00, but Randomized and Agent-planned also have 63.26 and 63.39. The authors only report the average over three seeds without a fluctuation range, making it difficult to judge whether these small differences are reliable.

Questions for authors

see weakness.

4 comments
AuthorsOpenReview ↗

Response to Reviewer E6Uf (Part 1/2)

We thank the reviewer for the thoughtful and constructive feedback. We have revised the paper accordingly and addressed each of the raised weaknesses and questions in detail below. All revisions are marked in blue in the revised manuscript.

W1 The compiler’s role in realizing the visual proposal

W1: The code shows that the compiler actively separates objects, leaving a default spacing of 0.05m, out-of-bounds objects are moved back into the room, and if space is insufficient, the room is expanded up to 12m. Therefore, zero collisions and zero out-of-bounds indicate that the final output has been repaired to be feasible, but do not by themselves prove that the initial layout is more reasonable. The paper also does not report collision/relation metrics before and after post-processing, object displacement, or room expansion.

Response

The compiler preserves and recovers the spatial organization of the original visual proposal while correcting geometric errors introduced during single-image 3D lifting. We now report before/after metrics and analyze object displacement and room expansion to clarify this role. We have added these measurements to Appendix D.1.

1. Compilation preserves and recovers the proposal’s spatial organization.

Both compiler variants eliminate collisions, but their effects on the proposed arrangement differ. Without the relation contract, relation satisfaction falls from 69.3% to 54.2%. With the contract, it rises to 93.0%.

The full compiler preserves 98.1% of the relations satisfied after lifting and restores 81.4% of those broken during lifting. These results show that compilation retains and restores the proposal’s layout intent while resolving geometric conflicts. Figure 3 illustrates this distinction: geometry-only compilation leaves relations such as chairs facing their table broken, while contract guidance restores them.

Table R1: Layout fidelity before and after compilation on the 50 evaluated scenes.

LayoutColliding objects ↓Proposal relations satisfied ↑
Before compilation (lifted)54.0%69.3%
Compiled without relation contract0%54.2%
Compiled with relation contract0%93.0%

2. Object displacement primarily reflects placing whole zones into a shared room.

Each zone is lifted in its own camera frame, so assembling the planned room requires moving these furniture groups together. Whole-zone translation accounts for 94.8% of the summed squared object displacement. This translation preserves relative object positions within each zone, indicating that the displacement measure is dominated by placing groups into the room.

Room expansion accommodates the assembled zones and their required clearances. The resulting occupancy is 0.30, compared with 0.58 before expansion and 0.34 in real MP3D rooms. The additional space therefore brings the measured occupancy closer to the real-room reference.

AuthorsOpenReview ↗

Response to Reviewer E6Uf (Part 2/2)

W2 Navigation gains and reliability

W2: In table 3, Real-environment SR goes from 48.26 to 48.57, only 0.31 percentage points higher. In table 2, SPL even slightly decreases from 21.18 to 21.17. SPL on seen targets rises from 61.31 to 64.00, but Randomized and Agent-planned also have 63.26 and 63.39. The authors only report the average over three seeds without a fluctuation range, making it difficult to judge whether these small differences are reliable.

Response

To assess the training value of our scenes beyond fine-tuning an existing policy, we train SemExp [1] from scratch at three scene budgets. Specifically, SemExp builds a top-down semantic map and uses a goal policy trained with reinforcement learning to choose where to search. We also evaluate SceneBloom’s controllable search difficulty under two fine-tuning recipes.

1. Training from scratch: SceneBloom achieves the highest mean real-environment SR across all three scene budgets.

SceneBloom outperforms training on real scans, randomized scenes, and agent-planned scenes in mean SR at every budget. Its advantage over real-scan training is approximately three percentage points at both 80 and 120 houses. Two scene properties help interpret these results:

  • Coherent layouts preserve navigable space for exploration. Randomized houses place the same furniture in the same rooms at random, which often splits a room’s walkable floor and confines training searches to part of the house. In the 80-house training sets, the largest connected region covers 98.4% of walkable area in SceneBloom houses, compared with 86.7% in Randomized houses. Agent-planned houses share our functional zones and assets, but the text agent places objects less realistically than our visual proposals, so the policy learns to search among arrangements that are less common in real homes.
  • Learnable search difficulty supports training an RL policy from scratch. SceneBloom combines realistic spatial organization with more manageable search difficulty than the real-scan training houses. These houses therefore give the policy more complete searches to learn from, as 91.9% of training episodes end at the policy’s own STOP, compared with 75.0% in real scans.

We have added these results to Section 4.2.1 (Figure 6) and Appendix E.3 (Table 17).

Table R1: SemExp trained from scratch and evaluated on 1,000 episodes in real HM3D validation homes.

Training scenesSR, 40 houses ↑SR, 80 houses ↑SR, 120 houses ↑
Real scans33.70 ± 2.7032.53 ± 2.9132.87 ± 0.95
Randomized32.53 ± 2.3232.67 ± 1.1733.50 ± 1.22
Agent-planned34.00 ± 1.2334.07 ± 0.7134.40 ± 1.23
SceneBloom34.70 ± 0.6235.57 ± 1.0035.63 ± 0.99

2. Fine-tuning: SceneBloom’s controllable difficulty supports further improvement in real-environment generalization.

For an agent that already has navigation capabilities, SceneBloom’s occluder zones hide goals behind furniture, which gives such an agent more challenging target-search problems. We evaluate this capability under both the original recipe and a mask-perturbation recipe. The latter perturbs masks during training to reduce the mismatch with predicted masks at evaluation, while layouts and the evaluation procedure remain unchanged. We have added the mean ± SD of every fine-tuned configuration under both recipes to Appendix E.4 (Table 18).

SceneBloom with occluders achieves the highest mean SR and SPL among the compared sources under both recipes. With mask perturbation, it reaches 50.85 SR, exceeding the pretrained agent by 2.59 percentage points and real-scan continuation by 1.93 points.

Table R2: Fine-tuning OVSegDT under the original and mask-perturbation recipes.

Training scenesOriginal SR ↑Original SPL ↑Mask perturbation SR ↑Mask perturbation SPL ↑
Pretrained48.2621.1848.2621.18
Real-scan continuation47.57 ± 0.3120.68 ± 0.1048.92 ± 0.7721.40 ± 0.23
SceneBloom48.57 ± 0.5221.17 ± 0.1749.46 ± 0.6421.17 ± 0.15
Randomized with occluders48.34 ± 0.7120.72 ± 0.2950.08 ± 0.5221.81 ± 0.46
Agent-planned with occluders49.38 ± 1.4021.03 ± 0.5149.65 ± 0.5821.47 ± 0.22
SceneBloom with occluders49.50 ± 1.3421.35 ± 0.3750.85 ± 0.9822.19 ± 0.27

3. Seen goals: the training advantage is clearer on harder generalization tasks.

SceneBloom achieves strong seen-goal SR and the highest mean SPL. Familiar goals in generated environments place fewer demands on transfer, so small differences in this setting provide a limited picture of scene-source quality. Harder generalization tasks show the advantage more clearly, as the held-out real-environment results in parts 1 and 2 show.

[1] Chaplot et al. Object Goal Navigation using Goal-Oriented Semantic Exploration. NeurIPS, 2020.

Dear Reviewer,

Thanks for your time and constructive feedback. As the discussion period closes in two days, we want to confirm whether our responses have addressed your concerns.

We would be happy to clarify any remaining questions. Thank you again for your time and consideration.

Best regards, The Authors

Thanks for the detailed response. The added analyses and experiments have now addressed most of my concerns, particularly the more reliable evaluation. Thus, I will raise my rating from 2 to 3.

Reviewer_m1YKOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents SceneBloom, an agentic framework for generating realistic, simulation-ready and embodied-learning supportting 3D indoor environments from text descriptions. In SceneBloom, a VLM first uses retrieved room-layout references to divide a room into functional zones and specify the furniture. A diffusion model with asset-specific LoRA adapters then generates visual proposals of furniture arrangements. These proposals are lifted into 3D, while a VLM extracts spatial relation contracts, such as facing and alignment. A geometric compiler subsequently resolves collisions and layout conflicts while preserving these relations, then adds wall-mounted and surface objects.

The paper evaluates SceneBloom on 50 scenes across five room types. The comparison experiments in Table 1 and Figure 4 includes physical plausibility, visual quality, layout similarity to real rooms and generation efficiency. Ablations are also sufficient, examining the effects of visual proposals, relation contracts, and layout retrieval (Figure 5, Tables 10–11). For embodied learning, the authors fine-tune an object-goal navigation policy on generated scenes and evaluate its generalization to unseen generated scenes and HM3D-OVON scanned environments (Tables 2–3), including an experiment with additional occluders to increase navigation difficulty.

Critical strengths
  1. The paper provides thorough quantitative and qualitative evaluations along with meaningful ablation studies, which demonstrate clear improvements in scene generation quality with efficient generation time cost.

  2. Rather than stopping at the generation of simulation-ready scenes, the authors actually train an embodied navigation policy on the generated environments and evaluate its generalization to unseen scenes.

Critical weaknesses
  1. The method relies not only on text but also on retrieved structured 3D room layouts as references. The retrieval ablation suggests that these references contribute substantially to the final results.

  2. The improvements in downstream navigation performance are relatively small. While the evaluations of physical plausibility, visual quality, and layout similarity demonstrate improvements in scene generation, it remains unclear which specific scene properties contribute to more effective training environments for embodied agents.

Questions for authors
  1. How well does the method generalize to novel room layouts without suitable retrieved references?
  2. How scalable is the per-asset LoRA training requirement to new objects?
  3. Could the authors report results on HM3D-OVON Val Seen and Val Seen Synonyms to better characterize the generalization benefits across different goal-category splits?
  4. How do the seen/unseen goal categories used in SceneBloom fine-tuning correspond to the official HM3D-OVON category splits?
  5. In Table 2, the SR improvement on real scanned scenes is only 0.31 percentage points (averaged over three training seeds). Could the authors expand the evaluation to establish the statistical reliability of this gain?
7 comments
AuthorsOpenReview ↗

Response to Reviewer m1YK (Part 1/4)

We thank the reviewer for the insightful and positive evaluation of our work. We have revised the paper accordingly and addressed each of the raised weaknesses and questions in detail below. All revisions are marked in blue in the revised manuscript.

W1 Component contributions with and without retrieval

W1: The method relies not only on text but also on retrieved structured 3D room layouts as references. The retrieval ablation suggests that these references contribute substantially to the final results.

Response

Retrieval and the proposal and compilation stages do different jobs, and the following ablation separates them cleanly (Table R1). Retrieval supplies exemplars of the requested room type, which shape functional zoning. Without it, the number of zones per scene falls from 3.94 to 3.14, and the visual quality scores fall in every setting, most in functionality. Physical validity does not depend on retrieval, as collisions and stability stay almost the same without it. The references therefore contribute the functional organization of a room, which is the role they are designed for.

To assess the contributions of visual proposals and contract guidance, we evaluated all eight combinations of retrieval, proposal type, and contract guidance. We have added this additional ablation to Table 14 in Appendix D.1.

  • Visual proposals achieve higher mean layout-quality scores across all four retrieval–contract settings. The improvement appears on all four criteria, including when contract guidance is disabled. Thus, the quality advantage of visual proposals is present both with and without retrieval or contract guidance.
  • Contract guidance improves relation preservation for both proposal types, with and without retrieval. For visual proposals, enabling contract guidance reduces contract violations from 37.45% to 4.25% with retrieval and from 29.79% to 5.93% without retrieval. Textual proposals show the same pattern.

Table R1: Factorial ablation of retrieval, proposal type, and contract guidance. Bold indicates the best reported mean within each retrieval group.

RetrievalProposalContractRel. violations (%) ↓Realism ↑Functionality ↑Coherence ↑Completeness ↑
WithTextualOff39.075.445.474.293.80
WithVisualOff37.455.665.654.314.05
WithTextualOn10.755.505.594.373.93
WithVisualOn4.255.835.854.654.07
WithoutTextualOff35.554.864.544.113.62
WithoutVisualOff29.795.084.794.233.90
WithoutTextualOn13.074.904.584.173.70
WithoutVisualOn5.935.385.044.543.90

These results support contributions from both components: visual proposals improve judged layout quality, while contract guidance helps preserve proposal-specified spatial relations during compilation.

AuthorsOpenReview ↗

Response to Reviewer m1YK (Part 2/4)

W2 & Q5 Navigation benefits, scene properties, and reliability

W2: The improvements in downstream navigation performance are relatively small. ...it remains unclear which specific scene properties contribute to more effective training environments for embodied agents.

Q5: In Table 2, the SR improvement on real scanned scenes is only 0.31 percentage points (averaged over three training seeds). Could the authors expand the evaluation to establish the statistical reliability of this gain?

Response

We report training-seed variability for the navigation results and also train SemExp [1] from scratch to more broadly assess the training value of our scenes. Specifically, SemExp builds a top-down semantic map and uses a goal policy trained with reinforcement learning to choose where to search. SceneBloom achieves higher mean success on real environments than training on real scans, randomized scenes, or agent-planned scenes across the three budgets. The results support the value of coherent layouts and learnable search difficulty.

1. Training from scratch shows consistent mean advantages across scene budgets.

SceneBloom improves mean SR over real-scan training by approximately three percentage points at both 80 and 120 houses. Two scene properties help explain these results:

  • Coherent layouts preserve navigable space for exploration. Randomized houses place the same furniture in the same rooms at random, which often splits a room’s walkable floor and confines training searches to part of the house. In the 80-house training sets, the largest connected region covers 98.4% of walkable area in SceneBloom houses, compared with 86.7% in Randomized houses. Agent-planned houses share our functional zones and assets, but the text agent places objects less realistically than our visual proposals, so the policy learns to search among arrangements that are less common in real homes.
  • Learnable search difficulty supports training an RL policy from scratch. SceneBloom combines realistic spatial organization with more manageable search spaces than the real-scan training houses. These houses therefore give the policy more complete searches to learn from, as 91.9% of training episodes end at the policy’s own STOP, compared with 75.0% in real scans.

We have added these results to Section 4.2.1 (Figure 6) and Appendix E.3 (Table 17).

Table R1: SemExp trained from scratch and evaluated on 1,000 episodes in real HM3D validation homes.

Training scenesSR, 40 houses ↑SR, 80 houses ↑SR, 120 houses ↑
Real scans33.70 ± 2.7032.53 ± 2.9132.87 ± 0.95
Randomized32.53 ± 2.3232.67 ± 1.1733.50 ± 1.22
Agent-planned34.00 ± 1.2334.07 ± 0.7134.40 ± 1.23
SceneBloom34.70 ± 0.6235.57 ± 1.0035.63 ± 0.99

2. Controllable search difficulty provides further training benefits for a pretrained policy.

To examine the training benefits and their variability, we report mean ± SD for every fine-tuned configuration below. SceneBloom’s occluder zones hide goals behind furniture, so a pretrained policy faces harder target-search challenges that are closer to those in real homes. We evaluate this capability under both the original recipe and a mask-perturbation recipe designed to reduce the mismatch between ground-truth training masks and predicted evaluation masks. The latter modifies training perception inputs while layouts and the evaluation procedure remain unchanged. We have added these results under both recipes to Appendix E.4 (Table 18).

SceneBloom with occluders improves mean SR and SPL over its default configuration under both recipes and achieves the highest mean scores among the compared sources. With mask perturbation, it reaches 50.85 SR, exceeding the pretrained policy by 2.59 points and real-scan continuation by 1.93 points.

Table R2: Fine-tuning OVSegDT under the original and mask-perturbation recipes.

Training scenesOriginal SR ↑Original SPL ↑Mask perturbation SR ↑Mask perturbation SPL ↑
Pretrained48.2621.1848.2621.18
Real-scan continuation47.57 ± 0.3120.68 ± 0.1048.92 ± 0.7721.40 ± 0.23
SceneBloom48.57 ± 0.5221.17 ± 0.1749.46 ± 0.6421.17 ± 0.15
Randomized with occluders48.34 ± 0.7120.72 ± 0.2950.08 ± 0.5221.81 ± 0.46
Agent-planned with occluders49.38 ± 1.4021.03 ± 0.5149.65 ± 0.5821.47 ± 0.22
SceneBloom with occluders49.50 ± 1.3421.35 ± 0.3750.85 ± 0.9822.19 ± 0.27

These comparisons connect scene properties to training outcomes: organized layouts support effective exploration, while control over search difficulty allows the scenes to challenge both initially untrained and pretrained policies.

[1] Chaplot et al. Object Goal Navigation using Goal-Oriented Semantic Exploration. NeurIPS, 2020.

AuthorsOpenReview ↗

Response to Reviewer m1YK (Part 3/4)

Q1 Scene generation with cross-type references

Q1: How well does the method generalize to novel room layouts without suitable retrieved references?

Response

Thank you for this question. We have added the results and qualitative examples to Appendix C.5 (Table 13 and Figure 9). SceneBloom can generate room types absent from the reference library by adapting layouts retrieved from related room types. References are selected using the same embedding-similarity search as the main pipeline. We add a general instruction to the planning stage for such requests: distinguish spatial organization that can be reused from additional functions required by the target room. Room size, functional zones, and furniture counts are then planned for the target. For example, a restaurant can reuse table-and-chair groupings from dining-room references while introducing a bar and a host stand.

Table R1 reports results for hospital rooms, restaurants, and waiting rooms generated without same-type references. All evaluated scenes have zero detected mesh collisions and zero out-of-bounds objects. Figure 9 in Appendix C.5 shows the retrieved references alongside the generated scenes, illustrating how their spatial organization is adapted to the target room’s functions.

Table R1: Generation with references from different room types.

Target room typeReference typeScenesObjects per sceneColl. (%) ↓OOB ↓Stab. (%) ↑Real. ↑Func. ↑Coh. ↑Compl. ↑
Hospital roomBedroom1030.10087.65.405.323.985.90
RestaurantDining room1057.80090.06.205.764.486.28
Waiting roomLounge1058.70071.56.086.204.647.16

Q2 Scalability of per-asset LoRA training

Q2: How scalable is the per-asset LoRA training requirement to new objects?

Response

Per-asset LoRA training is a one-time, offline cost, and each adapter is reusable across scene generations. Only floor assets used in visual proposal generation require training, while surface and wall objects need no asset-specific training. It takes approximately 7.4 minutes per asset on a single H100 GPU, including model preparation and training. Training cost scales with the number of distinct new floor assets, and different assets can be trained independently on separate GPUs to accelerate library expansion.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We sincerely thank all reviewers for their thoughtful and constructive feedback. We have incorporated the suggested experiments, discussions, and analyses into the revised manuscript, with revisions highlighted in blue. Below, we summarize our core contributions and the major updates introduced during the rebuttal.

Core contributions of SceneBloom

We thank the reviewers for recognizing our work's main contributions:

  1. Agentic scene generation framework with visual sampling and contract-guided compilation (highlighted by E6Uf, kRN8): SceneBloom samples each functional zone's furniture arrangement with diffusion priors and compiles it into 3D under relation contracts, preserving qualitative relationships while repairing numerical poses.
  2. Highly realistic and physically valid scenes (highlighted by m1YK, E6Uf, kRN8): Contract-guided compilation resolves geometric conflicts while preserving the sampled layouts and their diversity. SceneBloom thus achieves the highest visual quality and physical stability among the compared methods, with no detected mesh collisions and the layouts closest to real rooms.
  3. Coherent layouts and learnable search difficulty for embodied navigation (highlighted by m1YK, E6Uf, kRN8): SceneBloom layouts keep walkable space connected and provide searches at a difficulty the policy can learn from. On held-out real scans, policies trained from scratch in SceneBloom houses outperform those trained on an equal number of real scans from the same dataset, and raising the difficulty toward that of real homes further improves a pretrained policy.
  4. Efficient scene generation (highlighted by m1YK, kRN8): Without an iterative placement loop, SceneBloom generates a layout in 2.7 minutes, about 27× faster than SAGE.

Major updates of experimental results during rebuttal

  • Section 4.2.1 & Appendix E.3: Training an additional object navigation policy from scratch with 40, 80, and 120 houses, showing that policies trained on SceneBloom houses transfer best to real homes at every house count (addressing m1YK's W2 & Q5, E6Uf's W2).
  • Appendix D.1: Full factorial ablation of retrieval, proposal type, and contract guidance, isolating the contribution of each component (addressing m1YK's W1).
  • Appendix D.1: Layout metrics before and after compilation, showing that the compiler removes all collisions while keeping 98.1% of the relations that hold after lifting (addressing E6Uf's W1).
  • Appendix C.5: Generation of four new room types, three of which have no reference rooms of the same type, with zero detected collisions (addressing m1YK's Q1, kRN8's W3).
  • Appendix C.2: Manual and independent verification of relation contracts, confirming the correctness of the extracted relations (addressing kRN8's W2a & Q1b).
  • Appendix C.1: Stability by object size, where SceneBloom is the most stable for both large and small objects (addressing kRN8's W2b).
  • Appendices A.5 & A.6: Compiler extensions for under relations, placement inside articulated storage, and functional clearance (addressing kRN8’s W2c, W2d & Q2).
  • Appendix D.2: Proposal and contract ablation with GPT-6 Sol, showing that both components keep their advantages with a stronger foundation model (addressing kRN8's W1 & Q1a).
  • Appendix B.3: GPT-6 Astra as a second judge, which again scores SceneBloom highest (addressing kRN8's Q3).
  • Section 4.1.1 & Table 1: Expanded evaluation of SceneWeaver from three room types to all five room types.

Major updates of discussions and analyses during rebuttal

  • Section 4.2 & Appendices E.2, E.3, E.5: Analysis of why SceneBloom scenes benefit navigation training, showing that coherent layouts preserve navigable space for exploration, while control over search difficulty allows the scenes to challenge both initially untrained and pretrained policies (addressing m1YK's W2).
  • Appendix C.6: Failure case analysis with examples (addressing kRN8's Q4).
  • Appendix E.4: Mapping of the evaluation goals to the official HM3D-OVON splits, clarifying that real scan goals are used neither in pretraining nor in our fine-tuning (addressing m1YK's Q4).
  • Appendix D.1: Analysis of object displacement during compilation, showing that it mostly comes from placing whole zones and keeps relative positions within each zone (addressing E6Uf's W1).

We believe these additions address the reviewers' concerns and strengthen our manuscript. The individual responses provide the full tables and details for each point.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.