acceptodds
Under review as a conference paper at ICLR 2027

ProScene: Progressive 3D Indoor Scene Generation with Spatial-Validity Guidance

Abstract

Indoor scene synthesis methods commonly generate a layout where objects are represented by oriented 3D bounding boxes, followed by mesh retrieval or generation based on box parameters to complete the scene. This often leads to output with spatial violations, including collisions and out-of-bounds errors. Because boxes are too coarse to represent complex shapes or interleaving objects, true violations cannot be reliably distinguished from valid configurations where bounding boxes overlap but the meshes do not, limiting accurate guidance toward spatial validity. We therefore propose ProScene, a retrieval-free multi-stage model that progressively generates 3D scenes through a boxcuboidmesh hierarchy. ProScene first produces a coarse box layout in stage 1, then employs a coupled layout-cuboid diffusion model in stage 2 to jointly refine the layout and generate object cuboids, with bidirectional feature exchange allowing both representations to co-evolve during denoising. We further apply cross-scale spatial-validity guidance, using boxes for efficient spatial error assessment and cuboids for more accurate evaluation. The resulting violation energies are used to guide the output toward spatial plausibility. Finally, stage 3 conditions an object generator on the predicted cuboids and semantic information to synthesize detailed meshes. This hierarchy provides an explicit structural bridge between coarse scene arrangement and detailed object geometry, enabling spatial-violation-aware scene generation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D computer vision, generative AI

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_DmEmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

ProScene generates indoor scenes through a box, cuboid, mesh hierarchy. Coupled layout and cuboid denoisers exchange features, while box- and cuboid-level violation energies guide training and late sampling. A cuboid-conditioned TRELLIS model then generates individual meshes. Experiments show reduced mesh collisions and room-boundary violations across three initial layout generators.

Critical strengths
  • Cuboids provide a useful bridge between efficient scene planning and detailed mesh synthesis, enabling finer collision reasoning than a single box while supporting retrieval-free generation.
  • Jointly evolving layout and object structure is well motivated: geometry can inform placement, and scene context can inform shape. The interaction and guidance ablations support this design across a modular pipeline.
Critical weaknesses
  • False-positive box penalties remain in the objective. Equations 11–15 directly penalize box overlap even when the corresponding cuboids do not collide. This can discourage the valid interleaving used to motivate finer geometry. Combined guidance improves results in Table 4, so the design may be a useful robustness trade-off, but its effect on valid overlaps needs explanation.
  • Existence-weighted guidance permits a structural shortcut. Collision energy can decrease by suppressing cuboids instead of moving them (Eq. 10). Appendix A.3 acknowledges this and uses smaller cuboid updates, applied in only the final two steps; denoising priors also provide safeguards. These mitigate the risk, but completeness or cuboid-coverage measurements would clarify whether it occurs. Top-down semantic FID is a limited check on missing object parts.
Questions for authors
  • How does direct box-overlap penalization affect valid interleaving, and why is it preferable to using boxes only to select cuboid candidate pairs?
  • How do occupancy, residual collisions, and cuboid coverage change from before guidance to the cuboid output and then the final meshes?
3 comments
AuthorsOpenReview ↗

Response to Reviewer DmEm

We thank the reviewer for recognizing the motivation for using cuboids as an intermediate structural representation and the benefit of bidirectional layout-cuboid evolution. We address the concerns below.

(W1 & Q1). Effect of box-overlap penalization on valid interleaving. We analyze this effect on bedroom scenes, following the setting of our ablation study. In general, box-level guidance encourages overlapping boxes to move farther apart, including valid interleaving pairs. Thus, direct box guidance can in principle discourage valid interleaving. However, object pairs with actual geometric collisions, identified by colliding cuboids, are pushed farther apart than valid interleaving pairs, with a 27.6% larger average center separation. In practice, we control its strength through the guidance hyperparameters, and under the setting used in our experiments, all valid-interleaving pairs are preserved, although their box penetration is slightly reduced by the box-collision penalty.

We keep the box-collision term because it provides a complementary coarse correction signal and leads to better results, as supported by the ablation study in Table 4 (main paper). Since we constrain each cuboid to lie within its parent box, cuboids from different objects can collide only if their parent boxes overlap. Therefore, we use box overlap to identify candidate object pairs for cuboid-level spatial violation assessment, avoiding the need to check all cuboid pairs and improving efficiency.

(W2 & Q2). Structural shortcut and geometry preservation. We evaluate whether guidance reduces spatial violations by suppressing cuboids. The table below reports the average number of active cuboids per object, occupied volume, box-occupancy ratio, and spatial violations before and after guidance, together with statistics of the final generated meshes.

Active cuboids/objectVolume/object (m3m^3)Box occupancy rateVolume/scene (m3m^3)VBL/scene (m3m^3)
Before guidance2.9721.227260.768686.687630.475
After guidance2.9721.223450.768456.667280.463
Final mesh-0.527190.462642.86790.224

With the small cuboid guidance step size ηC=0.002\eta_C=0.002 used in our experiments, the average number of active cuboids remains unchanged at 2.9722.972 per object (close to the average of 2.9562.956 cuboids per object computed from the ground-truth cuboid labels). Cuboid volume decreases by only 0.3%, whereas cuboid-level VBL decreases by approximately 2.5%. Therefore, the reduction in spatial violations is not achieved by suppressing cuboids. The generated meshes are also consistent with the final cuboid structures: 95.86% of the generated mesh volume is covered by the predicted cuboids. Together, these results show that the improvement in spatial validity is obtained while preserving the predicted object structure through the cuboid output and subsequent mesh generation.

AuthorsOpenReview ↗

Follow-up on Author Response and Further Discussion

Dear Reviewer DmEm,

Thank you for your time and constructive feedback. We have provided detailed responses and additional results addressing your questions and concerns.

As the author-reviewer discussion period is closing soon, we would be grateful if you could take a look at our clarifications. We hope they have addressed your concerns; if anything remains unclear, we would be very happy to discuss further.

Thank you again for your time.

Best regards,
The Authors

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer DmEm,

Thank you again for your thoughtful comments on our paper. We have provided detailed responses and additional experiments to address the concerns raised in your review.

As the discussion period is now nearing its end, we would be happy to clarify any remaining questions or concerns. If you find our responses satisfactory, we would be grateful if you could take them into account when finalizing your assessment.

Best regards, The Authors

Reviewer_P7reOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents PROSCENE, a retrieval-free, floor-plan-conditioned indoor scene generation pipeline that progressively represents each scene at three levels: box, cuboid, and mesh. Stage 1 uses an existing generator to produce a coarse box layout. Stage 2 jointly refines that layout and generates per-object cuboid sets through two synchronized diffusion denoisers with bidirectional feature exchange. Spatial-validity energies are evaluated first at the box level and then more precisely at the cuboid level; these energies regularize training and guide the final diffusion steps. Stage 3 adapts TRELLIS to generate an individual mesh for each object conditioned on its predicted cuboids and semantic category. Experiments on three 3D-FRONT room categories use ATISS, DiffuScene, and MiDiffusion as alternative Stage 1 models. The paper reports top-down FID and mesh-based collision/out-of-bounds metrics, together with ablations of the intermediate representation, bidirectional coupling, and spatial-validity guidance.

Critical strengths
  1. A well-motivated intermediate representation. Using cuboids between boxes and meshes directly addresses an important mismatch in prior pipelines: boxes are efficient for layout planning but too coarse for distinguishing true mesh collisions from valid interleaving. The hierarchy gives the method an explicit, differentiable structure on which spatial constraints can act before expensive mesh generation.
  2. A modular design with consistent gains across multiple layout generators. PROSCENE can be attached to ATISS, DiffuScene, and MiDiffusion, and it reduces the reported VBL for every room type and every Stage 1 model in Table 1. The ablations in Tables 2-4 are directionally consistent: cuboid conditioning outperforms box conditioning, bidirectional exchange outperforms one-way or no interaction, and each guidance component contributes to lower violations. This consistency is the strongest evidence supporting the paper.
Critical weaknesses
  1. The evaluation does not adequately establish mesh quality or the advantage over existing spatial guidance. The only realism metric is FID on 256 x 256 top-down orthographic images rendered with a shared semantic color palette (Appendix A.5). This protocol is mainly sensitive to layout and category occupancy and provides limited evidence about detailed object geometry, topology, texture, and multi-view quality. Consequently, comparable FID does not substantiate the claim that Stage 3 produces high-quality detailed meshes.

  2. PhyScene, a closely related floor-plan-conditioned method with collision and boundary guidance, has publicly released implementation code, evaluation scripts, and a pretrained checkpoint, but is not evaluated under the same final-mesh protocol. The reported ablations support the proposed components internally; however, a comparison with this available guided baseline would better establish their advantage over prior spatial guidance.

Questions for authors
  1. How are the K=12 target cuboids constructed from each training mesh? Please specify the decomposition or fitting algorithm, and how objects requiring more than 12 parts are handled.
  2. Can the authors compare against the publicly released PhyScene implementation under a matched floor-plan-conditioned setting and final-mesh VBL protocol?
1 comment
AuthorsOpenReview ↗

Response to Reviewer P7re

We thank the reviewer for recognizing the motivation for our hierarchical representation, modular design, and consistent gains across different layout generators. We address the concerns below.

(Q1). Cuboid decomposition. We use the cuboid decomposition provided by CASAGPT [R1] as the target cuboids. The object mesh is first voxelized and coarse-grained by merging voxels into larger cuboidal structures without increasing the occupied volume. Cuboid pairs are then iteratively merged using projection-based geometric criteria and a dynamic volume-ratio threshold. In 3D-FRONT training and test sets, the maximum number of resulting cuboids per object is 1212; we therefore use K=12K=12 cuboid slots, with unused slots represented by the non-existence state. We will clarify this preprocessing procedure in the revised experimental setting. Please refer to CASAGPT paper [R1] for more details on the decomposition algorithm.

[R1] Feng, Weitao, et al. "CASAGPT: cuboid arrangement and scene assembly for interior design." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

(W2 & Q2). Comparison with PhyScene. We thank the reviewer for the suggestion. PhyScene currently releases a pretrained checkpoint only for LivingRoom, while training code and pretrained checkpoints for the other room categories are not publicly available. We therefore use the released LivingRoom checkpoint and perform an experiment following the protocol of Table 1 (main paper).

MethodFIDMBLOOBVBL
PhyScene42.8870.0520.2420.294
ProScene (w/ PhyScene)42.1740.0380.1280.166

Using PhyScene as the box generator, ProScene reduces MBL from 0.0520.052 to 0.0380.038, OOB from 0.2420.242 to 0.1280.128, and VBL from 0.2940.294 to 0.1660.166. This directly demonstrates the additional benefit of our cuboid-level representation and cross-scale spatial-validity guidance over the existing box-level spatial guidance of PhyScene.

Reviewer_JwnhOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies spatial violations in indoor scene generation, particularly cases where bounding boxes give a poor approximation of the actual object geometry. ProScene starts with an existing layout generator, jointly refines the layout and generates object cuboids through two interacting diffusion branches, and then uses the cuboids to condition a TRELLIS-based mesh generator.

Critical strengths
  1. The paper targets a practical problem. A scene can look reasonable at the layout level and still contain unusable mesh intersections. Evaluating violations on the assembled meshes makes the reported improvements more meaningful than checking bounding-box overlap alone.
  2. Using cuboids to connect layout refinement with object generation is a sensible design. It gives the layout model more information about object shape and gives the mesh generator a scene-dependent structural condition. This is a useful direction for generating objects that fit their surroundings.
Critical weaknesses
  1. The main comparisons miss methods that directly address spatial validity, especially PhyScene and CASAGPT. Table 1 shows that ProScene improves the original layout generators, but it is less clear what it adds over existing collision guidance or cuboid-based methods. The internal ablations do not fully answer this question.
  2. I am not convinced that the lower collision volumes come without losing object geometry. FID is computed from semantic-colored top-down images, which can miss substantial geometric changes. The appendix also acknowledges that guidance can suppress cuboids.
  3. Some definitions make the guidance hard to assess. An absent cuboid has an existence target of −1-1, but its sigmoid weight is still about 0.2690.269. The collision screening also relies on cuboids staying inside their parent boxes, yet the paper does not explain how this is enforced. These details need clarification because they directly affect which collisions are detected and penalized.
Questions for authors
  1. Do you have results against PhyScene or CASAGPT under the same evaluation protocol? A simpler control that refines the same initial layouts using the proposed spatial energies while keeping the retrieved object meshes fixed would also help establish what the coupled generative model adds.
  2. Can you report how well the final meshes match the predicted cuboid structures, together with object geometry or occupied-volume statistics before and after guidance?
5 comments
AuthorsOpenReview ↗

Response to Reviewer Jwnh

We appreciate the reviewer's feedback and thank the reviewer for their comments. We are pleased that the reviewer found our paper targets a practical problem with a sensible design. We address the reviewer's concerns as follows.

(W1 & Q1). Comparison with existing collision guidance or cuboid-based methods (PhyScene, CASAGPT) under the same evaluation protocol. We thank the reviewer for this suggestion. The official CASAGPT repository currently does not provide training code or pretrained checkpoints, preventing a matched comparison. For PhyScene, we use the publicly released LivingRoom checkpoint for comparison since it is the only available pretrained weight. We treat PhyScene as stage 1 box-layout generator and evaluate the completed scenes under the same mesh-based protocol used in Table 1 (main paper).

MethodFIDMBLOOBVBL
PhyScene42.8870.0520.2420.294
ProScene (w/ PhyScene)42.1740.0380.1280.166

ProScene further reduces MBL from 0.0520.052 to 0.0380.038 and OOB from 0.2420.242 to 0.1280.128, resulting in a substantially lower VBL of 0.1660.166 compared with 0.2940.294 for PhyScene. This shows that our finer cuboid representation and cross-scale spatial-validity guidance provide additional improvement beyond PhyScene's box-level spatial guidance.

(W2 & Q2). Object geometry and cuboid-mesh consistency. To evaluate whether the reduction in spatial violations is achieved without losing object geometry, we first measure the consistency between the final meshes and their structural representations. We voxelize the boxes, cuboids, and meshes at a voxel size of 5cm5cm and report IoU and mesh coverage, where coverage measures the fraction of mesh volume contained within the corresponding box or cuboid structure. The statistics are computed on generated bedroom scenes from the test set.

Retrieved mesh & pred. boxGenerated mesh & pred. cuboidsGround-truth mesh & cuboids
IoU33.22%40.84%51.19%
Coverage97.53%95.86%99.66%

The predicted cuboids cover 95.86% of the generated mesh volume, close to the 97.53% coverage achieved by predicted boxes for retrieved meshes, despite cuboids providing a substantially more compact geometric representation. Their IoU with the generated meshes reaches 40.84%, compared with only 33.22% for retrieved meshes and their predicted boxes, and approaches the 51.19% IoU between ground-truth meshes and their cuboid abstractions. These results indicate that the predicted cuboids provide both high mesh coverage and a tighter structural representation of the generated geometry.

We further examine whether spatial-validity guidance reduces spatial violations by suppressing cuboids. Below table reports the average number of cuboids per object, cuboid-union volume (m3m^3) per object, cuboid/box occupancy ratio, cuboid occupied volume (m3m^3) as well as cuboid-based VBL (m3m^3) per scene, computed from the generated bedroom scenes.

Active cuboids/objectVolume/object (m3m^3)Box occupancy rateVolume/scene (m3m^3)VBL/scene (m3m^3)
Before guidance2.9721.227260.768686.687630.475
After guidance2.9721.223450.768456.667280.463

With the cuboid guidance step size ηC=0.002\eta_C=0.002 used in our experiments, the average number of active cuboids remains unchanged, while cuboid volume decreases by only 0.3%. In contrast, cuboid-level VBL decreases by approximately 2.5%. Thus, late-sampling guidance does not suppress meaningful cuboid structure as a shortcut for reducing spatial violations; instead, it makes small adjustments to cuboid geometry, including a slight reduction in cuboid size, which leads to a substantially larger reduction in spatial violations.

AuthorsOpenReview ↗

Response to Reviewer Jwnh (cont.)

(W3). Details on cuboid existence weight and cuboid-box containment enforcement. We thank the reviewer for identifying these missing details. For the existence weight of cuboid ckℓc_{k\ell} in Eq. 10, our implementation uses πkℓ=Sigmoid⁡(4e^kℓ)\pi_{k\ell}=\operatorname{Sigmoid}(4\widehat e_{k\ell}). Thus, for predicted existence states of −1-1 and +1+1, the corresponding weights are approximately 0.0180.018 and 0.9820.982, respectively. The factor of 44 was missing from Eq. 10. Thank you for pointing this out. We will correct it in the final paper.

To ensure that cuboid-level collision screening is valid, we explicitly constrain each predicted cuboid to lie inside its refined parent box. Let

sˉi(2)=∥s^i(2)∥∞\bar{s}_i^{(2)} = \Vert \widehat{\mathbf{s}}_i^{(2)} \Vert_{\infty}

denote the largest dimension of the predicted parent box, and let

s~i(2)=s^i(2)/sˉi(2)\widetilde{\mathbf{s}}_i^{(2)} = \widehat{\mathbf{s}}_i^{(2)} / \bar{s}_i^{(2)}

be its normalized size. For a predicted cuboid c^ij=(x^ij,d^ij,e^ij)\widehat c_{ij}=(\widehat{\mathbf{x}}_{ij},\widehat{\mathbf{d}}_{ij},\widehat e_{ij}), we bound its size element-wise as

d^ijproj=min⁡(max⁡(d^ij,10−4),s~i(2))\widehat{\mathbf{d}}_{ij}^{\mathrm{proj}} = \min\big(\max(\widehat{\mathbf{d}}_{ij}, 10^{-4}), \widetilde{\mathbf{s}}_i^{(2)}\big)

and move its center inward when necessary using

x^ijproj=clip⁡(x^ij,−12(s~i(2)−d^ijproj),12(s~i(2)−d^ijproj))\widehat{\mathbf{x}}_{ij}^{\mathrm{proj}} = \operatorname{clip}\Big(\widehat{\mathbf{x}}_{ij}, -\tfrac{1}{2}\big(\widetilde{\mathbf{s}}_i^{(2)} - \widehat{\mathbf{d}}_{ij}^{\mathrm{proj}}\big), \tfrac{1}{2}\big(\widetilde{\mathbf{s}}_i^{(2)} - \widehat{\mathbf{d}}_{ij}^{\mathrm{proj}}\big)\Big)

Therefore, oversized cuboids are resized and off-center cuboids are shifted inward so that the complete cuboid remains inside its parent box. This projection is applied only when computing cuboid-level spatial-validity guidance and to the final cuboids passed to the object generator; it is not imposed on the main cuboid denoising process. We will add these details to the appendix.

AuthorsOpenReview ↗

Follow-up on Author Response and Further Discussion

Dear Reviewer Jwnh,

Thank you for your time and constructive feedback. We have provided detailed responses and additional results addressing your questions and concerns.

As the author-reviewer discussion period is closing soon, we would be grateful if you could take a look at our clarifications. We hope they have addressed your concerns; if anything remains unclear, we would be very happy to discuss further.

Thank you again for your time.

Best regards,
The Authors

Comments on the paper

3 comments
AuthorsOpenReview ↗

Summary of Author Responses

We sincerely thank all reviewers for their careful reviews and thoughtful suggestions. We are glad that the reviewers recognize that our paper addresses a practical problem (Jwnh), with a sensible design that leverages cuboids as an intermediate representation (Jwnh, P7re, DmEm) and a well-motivated joint evolution of layout and cuboid representations (DmEm). The consistent gains across different components and baseline generators further support the effectiveness of our modular design (P7re, DmEm).

The major shared concerns focus on the lack of direct comparison with existing spatial-validity methods and the quality of the generated object geometry. We summarize our responses as follows.

  • Direct comparison with PhyScene (Jwnh, P7re): Consistent with the results in Table 1 of the main paper, applying our proposed method to PhyScene improves both realism and spatial validity, reducing VBL from 0.294 to 0.166. This direct comparison shows that our cross-scale guidance provides further improvement beyond existing spatial guidance.
  • Structural shortcut for reducing collisions (Jwnh, DmEm): Under the hyperparameters used in our experiments, the average number of active cuboids per object remains unchanged at 2.972 before and after guidance and remains close to the average number of ground-truth cuboids (2.956). The guidance causes only a small reduction in cuboid volume (-0.3%), while producing a larger reduction in cuboid VBL (-2.5%). The predicted cuboids also cover 95.86% of the generated mesh volume. These results show that our method preserves object structure across stages rather than reducing spatial violations through a structural shortcut.
  • Quality of the generated objects (P7re): Comparison with EchoScene using MMD, COV, and 1-NNA shows that our method outperforms EchoScene across all three metrics for 10 most populous object categories. This indicates that our generated objects are geometrically closer to real objects, cover a broader portion of the real shape distribution, and show better agreement with the real 3D-FUTURE shape distribution.

We have also clarified the construction of target cuboids and cuboid-box containment enforcement, the effect of box-overlap penalization on valid interleaving, and corrected the cuboid existence weight formula. Please see the responses to each reviewer below for detailed analyses and results.

We sincerely appreciate the reviewers' constructive feedback and suggestions, which have helped us strengthen the paper. We believe that our responses have addressed the reviewers' concerns, and we would be grateful if they are taken into account in the final assessment.

AuthorsOpenReview ↗

Follow-up on Author Response and Further Discussion

Dear Reviewer P7re,

Thank you for your time and constructive feedback. We have provided detailed responses and additional results addressing your questions and concerns.

As the author-reviewer discussion period is closing soon, we would be grateful if you could take a look at our clarifications. We hope they have addressed your concerns; if anything remains unclear, we would be very happy to discuss further.

Thank you again for your time.

Best regards,
The Authors

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer P7re,

Thank you again for your thoughtful comments on our paper. We have provided detailed responses and additional experiments to address the concerns raised in your review.

As the discussion period is now nearing its end, we would be happy to clarify any remaining questions or concerns. If you find our responses satisfactory, we would be grateful if you could take them into account when finalizing your assessment.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.