acceptodds
Under review as a conference paper at ICLR 2027

Dovetail3D: Unifying Layout and Styling for VLM-Based 3D Indoor Scene Synthesis

Abstract

Vision-language models (VLMs) play an increasingly crucial role in 3D indoor scene synthesis, yet they rarely acquire the ability to construct scenes directly. Current systems broadly follow two design paradigms: tool-based agentic pipelines utilize explicit optimization tools but can impose rigid or mutually incompatible constraints, whereas model-driven generation preserves flexibility but lacks reliable physical control. Beyond the spatial optimization problems, style coherence is largely absent across both paradigms, despite its importance to overall scene aesthetics. We close this gap by presenting Dovetail3D, which post-trains one VLM model to jointly retrieve, select and place 3D assets, optimizing functional layout and stylistic coherence as a whole. Driven by the spatial properties of indoor scenes, we formulate scene synthesis as a sequential decision-making process. We first design a compact set of general-purpose, physically grounded rules to prune the agent's vast coordinate action space into feasible regions, while leaving the final placement decision to the agent. To train the model, we introduce a hierarchical RL post-training algorithm that integrates coarse-to-fine credit assignment for joint layout and styling. Experiments on 3D-FRONT scenes across diverse room types demonstrate that our post-training paradigm successfully internalizes layout and styling as core model capabilities, combining the reliability of tool-based pipelines with the flexibility of model-driven generation.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Scene Synthesis, Vision Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WEp3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Dovetail3D, which post-trains a single VLM (Qwen3.5-9B) to build 3D indoor scenes by retrieving, selecting and placing assets one at a time. A hand-written rule compiler prunes the coordinate space into feasible regions, and the agent picks the final placement inside them. Training uses GRPO with hierarchical credit: feasibility-based layout credits, and an LLM-judge style score distributed to individual assets. On 3D-FRONT, it reports zero out-of-bounds and collisions, higher relation scores, and small gains in judged style, compared with seven baselines.

Critical strengths
  1. Well-motivated design: Exposing state-dependent feasible regions as the action space, while leaving the final choice to the policy, is a sensible middle ground between tool-based reliability and model-driven flexibility.
  2. Useful learning signal: The feasible-domain return rewards placements that keep space open for later assets. This is a new and intuitive way to give credit for long-horizon layout decisions.
  3. Strong layout results and transparency: The method reaches zero out-of-bounds and collisions with clear gains in relation metrics. The appendix gives detailed rules, rubrics and hyperparameters, and code is provided.
Critical weaknesses
  1. Evaluation overlaps with the method: The pruning rules likely resemble the rule-based OOR/OAR/ACN metrics, and OOB = COL = 0 holds by construction. CNT also trails ReSpace in some rooms (e.g., 88.4 vs. 99.8 in living rooms) without discussion.
  2. Ablations are incomplete: There is no ablation of the pruning itself, and Table 3 is a single run on one room type.
Questions for authors

I would like the authors to address the three weaknesses above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer WEp3

We sincerely thank the reviewer for your recognition of our work and address each concern directly below.

W1: Evaluation overlaps with the method

W1a: The pruning rules likely resemble the rule-based OOR/OAR/ACN metrics, and OOB = COL = 0 holds by construction.

We first clarify that pruning enforces OOB and COL and overlaps only partially with the local rules assessed by OOR and OAR through LLM prompting. It does not encode ACN, which evaluates global activity-centric connectivity.

  1. Overlap with rule-based metrics is standard practice. Prior methods such as LayoutVLM, I-Design, Holodeck, and SceneWeaver likewise use solvers or optimizers and report related validity metrics.
  2. RL optimizes beyond the scope of local rules. Although our pruning rules interact with some simple metrics, they do not govern global scene quality, which is our primary focus. The compiler alone cannot capture completion, navigability, connectivity, or holistic style. Tables 1–2 in our manuscript show that Dovetail3D consistently leads on these unconstrained global objectives.
  3. OOB and COL are system-level guarantees, not learned-policy gains. We agree that OOB = COL = 0 for committed scenes is guaranteed by the local harness. However, the policy can still propose invalid actions before the check gate, and RL reduces this invalid-placement rate (PIR) from 16.2% to 0.3%, demonstrating learned compliance with the feasible domains.
  4. The scene-level gains are substantial. While OOB and COL are successfully bounded at zero by construction, Dovetail3D further translates these structural guarantees into superior compositional quality across all room types (outperforming the strongest baseline by 0.4–0.7 points in Style and 0.2–0.7 points in overall quality).

W1b: CNT also trails ReSpace in some rooms (e.g., 88.4 vs. 99.8 in living rooms) without discussion.

  • As reported in Table 1 of the main paper, ReSpace achieves higher Living Room CNT (99.8% vs. 88.4%) because it forces objects into rooms without strict clearance or relational validity—resulting in a massive drop in ACN (44.0% vs. 74.6%) and OOR/OAR (40.6%/88.2% vs. 86.4%/100.0%), alongside non-zero collisions (COL 15.1%) and out-of-bounds (OOB 1.0%). Dovetail3D prioritizes joint layout validity and usability rather than maximizing raw object count via invalid placements.

W2: Incomplete Ablations

W2a: There is no ablation of the pruning itself.

  • Pruning Ablation (Please see Table R1 in General Response): We added the full pruning ablation (Base w/o compiler, Base (w/o RL), Heuristic, and Dovetail3D) across 200 test rooms spanning three room types. Disabling the compiler in the Base model reduces ACN from 81.6% to 41.4% and OOR/OAR from 90.4%/97.9% to 25.9%/23.8%, while adding RL on top of the fixed compiler (Dovetail3D vs. Base (w/o RL)) boosts CNT by +7.0% (84.8% →\to 91.8%).

W2b: Table 3 is a single run on one room type.

  • Table 3 Scope: Table 3 is not a single-scene run, where every row reports the average across the complete Living Room test set with 53 cases (Section 5.1, Appendix B.3). We will make this explicit in the Table 3 caption.

Thank you for the authors' responses. The additional experiments have addressed most of my concerns. I have no further questions. I hope these experiments can be included in the final version of the paper. I will keep my current score.

We sincerely thank the reviewer for the encouraging feedback and for acknowledging our responses. We will carefully consolidate all reviewers' suggestions along with these additional experiments to update our manuscript toward the end of the rebuttal period.

Reviewer_Zyj4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Dovetail3D, a VLM-based framework for joint furniture layout generation and stylistic coordination in indoor 3D scenes. The method integrates sequential asset retrieval, selection, and placement with rule-based feasible-region construction and hierarchical RL credit assignment. Experiments on 3D-FRONT demonstrate improvements in geometric validity, functional relationships, and stylistic consistency compared with several existing approaches.

Critical strengths
  1. The paper addresses an important problem in 3D indoor scene synthesis by jointly considering functional layout and stylistic coherence. This is relevant to generating realistic and usable indoor environments.

  2. The proposed framework provides a unified pipeline for asset retrieval, selection, and placement. The combination of feasible-region constraints and RL post-training offers a practical approach to improving scene generation.

  3. The experiments include comparisons with several representative baselines, component-wise ablation studies, and human evaluations. The results show improvements in geometric validity, furnishing completeness, and stylistic consistency.

Critical weaknesses
  1. The methodological novelty is limited. The framework mainly combines existing techniques, including rule-based constraints, sequential asset placement, GRPO, and LLM-based rewards. The proposed credit assignment lacks clear advantages over simpler reward designs. The current ablation studies do not sufficiently justify its necessity.

  2. The contribution of learned spatial reasoning is unclear. Many geometric and relational constraints are directly enforced by the feasible-region compiler. These constraints also overlap with several evaluation metrics. Although RL improves object completion and reduces invalid placements, its improvements on some scene-level metrics are relatively small. A comparison with a non-learned placement strategy using the same constraints is needed.

  3. The baseline comparisons may not be fully fair. Different methods use different input conditions and geometric representations. For example, DiffuScene runs without target room geometry or furnishing requests, while I-Design uses bounding boxes for non-rectangular rooms. These differences may affect the reported performance and make the comparisons less convincing.

  4. The evaluation of stylistic coherence and generalization is insufficient. The style reward relies on contact sheets of individual assets rather than complete rendered scenes. The human study includes only 20 test cases and five annotators, without confidence intervals or inter-annotator agreement. The experiments also use only nine predefined styles, with limited evidence of generalization to unseen style requests.

Questions for authors
  1. Could the authors compare Dovetail3D with a non-learned placement strategy using the same feasible-region compiler and asset inventory? This would help clarify the contribution of the learned policy.

  2. The baselines use different input conditions and room representations (Appendix B.2). Could the authors provide more controlled comparisons or explain how these differences affect the results?

  3. How does the method perform on style requests outside the nine predefined categories?

11 comments
AuthorsOpenReview ↗

Response to Reviewer Zyj4

We thank the reviewer for the constructive feedback. And we address each concern as follows.

W1: Limited Novelty

W1a: The methodological novelty is limited. The framework mainly combines existing techniques, including rule-based constraints, sequential asset placement, GRPO, and LLM-based rewards.

We respectfully disagree that Dovetail3D is a naive combination of existing techniques. Every component is non-trivially co-designed for a distinct bottleneck in unified 3D scene generation and proven useful in our ablation studies:

  • Feasible-region compiler: Far from a generihis component is designed to enforce local physical constraints (e.g., collision) by restricting action spaces. As shown in Table R1 in General Response, removing it forces unconstrained LLM coordinate prediction and leads to strong performance collapse.

  • Decision-aligned hierarchical credit: Its two components are directly validated against matched GRPO controls in the General Response:

    • Layout credit (Table R2a): Dovetail3D beats GRPO in most layout metrics.
    • Style credit (Table R2b): Compared with uniform scene-level style GRPO, asset-specific attribution improves all five style dimensions, yielding a total gain of +4.99.
  • Verifier design: Our feedback is systematically decomposed into three complementary levels—action masks for local rules, direct metrics for layout quality, and task-specific rubrics for LLM style evaluation. This cohesive, task-tailored signal structure greatly enhances RL stability during training. Component-wise ablations are provided in Table 3 of our manuscript.

W1b: The proposed credit assignment lacks clear advantages over simpler reward designs. The current ablation studies do not sufficiently justify its necessity.

  • Direct Advantage Over GRPO (Please see Tables R2a–R2b in the General Response): Compared with vanilla layout GRPO (Table R2a), Dovetail3D produces lower PIR and fewer steps, while improving most downstream layout metrics. Compared with uniform scene-level style GRPO (Table R2b), it improves all five style dimensions. During training, we also found vanilla GRPO substantially less stable for this agentic RL system and prone to behavioral collapse.

W2 / Q1: Contribution of Learned Spatial Reasoning

W2a: The contribution of learned spatial reasoning is unclear... A comparison with a non-learned placement strategy using the same constraints is needed.

Q1: Could the authors compare Dovetail3D with a non-learned placement strategy using the same feasible-region compiler and asset inventory?

We evaluated a non-learned placement heuristic using the exact same compiler and asset inventory across all 200 rooms (see Table R1 in General Response):

  1. Dovetail3D vs. Heuristic: While the compiler enforces local validity, it cannot plan ahead. Under the exact same feasible-region compiler, Dovetail3D outperforms the non-learned heuristic by +15.3% on ACN, +14.6% on OOR, +11.7% on NAV, and +3.3% on CNT.
  2. Dovetail3D vs. Base Model (w/o RL): Holding the compiler fixed, Dovetail3D improves CNT by +7.0% (84.8% →\to 91.8%), ACN by +1.9%, and OOR by +1.8%.

W2b: Many geometric and relational constraints are directly enforced by the feasible-region compiler. These constraints also overlap with several evaluation metrics. Although RL improves object completion and reduces invalid placements, its improvements on some scene-level metrics are relatively small.

  1. Policy compliance is non-trivial even with the constraints. As Table R2a in the General Response shows, RL is essential to teach the policy to follow the actions in the feasible domain, dropping PIR from 16.2% to 0.3%.
  2. Metric Overlap: Metric overlap with rules is standard in scene synthesis. Prior methods such as LayoutVLM, I-Design, Holodeck, and SceneWeaver likewise invoke solvers or optimizers to satisfy local geometric rules and are still evaluated using collision, containment, and relational metrics.
  3. Global Objectives: The compiler cannot optimize global goals like completion, navigability, or style. As shown in Tables 1 & 2 in our manuscript, Dovetail3D consistently outperforms baselines on these unconstrained global objectives.
  4. Substantial Gains: Table 2 in the manuscript demonstrates that Dovetail3D consistently leads across all room types, outperforming the strongest baseline by 0.4–0.7 points in Style and 0.2–0.7 points in overall quality, representing substantial and non-trivial improvements rather than small gains.
AuthorsOpenReview ↗

Response to Reviewer Zyj4

W3 / Q2: Controlled Baseline Comparison Under Identical Input Geometry

W3: The baseline comparisons may not be fully fair. Different methods use different input conditions and geometric representations. For example, DiffuScene runs without target room geometry or furnishing requests, while I-Design uses bounding boxes for non-rectangular rooms.

  1. Input geometry strictly follows each baseline's native interface. For a fair comparison, we feed every method the richest inputs its native interface supports: DiffuScene's released unconditional generator accepts neither target room geometry nor furnishing/style requests, whereas I-Design receives only bounding boxes for non-rectangular rooms. Directly supporting arbitrary, non-rectangular floor plans is itself a practical strength of Dovetail3D.

  2. Style conditioning is held consistent across LLM-based methods. We provide the same target-style request to every method with an LLM interface.

Q2: Could the authors provide more controlled comparisons or explain how these differences affect the results?

Please see Table R3 in the General Response: on the maximum 100-case rectangular subset, all methods included in the controlled comparison receive identical room geometry and furnishing/style information, and Dovetail3D ranks first on most metrics.

W4 / Q3: Evaluation Statistics, and Unseen-Style Generalization

W4a: The evaluation of stylistic coherence and generalization is insufficient. The experiments also use only nine predefined styles, with limited evidence of generalization to unseen style requests.

Q3: How does the method perform on style requests outside the nine predefined categories?

  • Generalization to 5 Unseen Styles (Please see Table R5 in General Response): On the 100-case bedroom test set, Dovetail3D improves over Base on every unseen style (Industrial, Kids, Korean, Mediterranean, Southeast Asian), which shows generalization of RL in unseen cases.

W4b: The style reward relies on contact sheets of individual assets rather than complete rendered scenes.

We clarify that contact sheets are a deliberate training-time design choice for four reasons:

  1. Reduced evaluation bias: Complete scene renders introduce camera-position, viewpoint, occlusion, and lighting biases that can confound LLM-as-Judge style evaluation.
  2. Training efficiency: Launching Blender and producing multi-view renders at every online RL rollout would be computationally prohibitive. Cached contact sheets avoid this rendering loop and make training tractable.
  3. Richer asset-level evidence: Detailed product images expose fine-grained color, material, texture, and form information that may be too small to assess in full-scene renders.
  4. Empirical agreement with rendered-scene judgments: As shown in Table R4c of the General Response, contact-sheet scores show positive agreement with rendered-scene human consensus.

W4c: The human study includes only 20 test cases and five annotators, without confidence intervals or inter-annotator agreement.

  1. Human-Study Scale Relative to Prior Work: We first clarify that recruiting five annotators aligns with established evaluation protocols (e.g., LayoutVLM likewise uses five annotators). Moreover, unlike SceneWeaver's split-assignment protocol, every scene in our study is independently rated by all five annotators, yielding denser per-case coverage and reducing variance from individual raters.
  2. Agreement and Uncertainty Estimates (Please see Tables R4a–R4b in the General Response): Table R4a quantifies inter-annotator reliability via mean pairwise Kendall's τb\tau_b across all 160 method–scene outputs, showing values on par with or exceeding those reported by SceneWeaver. Table R4b quantifies cross-case uncertainty using 100,000-resample bootstrap 95% confidence intervals over the 20 test cases and Dovetail3D retains the highest mean across all four human-evaluation dimensions.
Reviewer_Zyj4OpenReview ↗

Rebuttal Comment

Thank you for the detailed rebuttal and for conducting the additional experiments. The new results substantially improve the paper's empirical support. In particular, the authors have now provided:

  • an ablation separating the procedural compiler from RL, including a non-learned heuristic using the same feasible regions;
  • comparisons with vanilla GRPO for layout credit and uniform scene-level GRPO for style credit;
  • a controlled evaluation on a subset where the compared methods receive identical inputs;
  • inter-annotator agreement and bootstrap confidence intervals for the human evaluation;
  • additional experiments on five unseen style categories.

These additions address several of the concerns raised in the original reviews. The rebuttal also makes the practical role of the system clearer: the compiler enforces local geometric feasibility, while RL reduces invalid proposals, improves completion, and provides some additional global coordination.

There are still several issues that need clarification.

The compiler ablation suggests that most of the geometric and relational improvements come from the procedural constraints. Removing the compiler reduces ACN from 81.6 to 41.4, OOR from 90.4 to 25.9, and OAR from 97.9 to 23.8. In comparison, adding RL on top of the fixed compiler improves CNT by 7.0 points, ACN by 1.9 points, and OOR by 1.8 points, while OAR slightly decreases. RL also reduces the invalid-placement rate from 16.2% to 0.3% and shortens the rollout. Thus, the rebuttal demonstrates that RL is useful within the proposed system, but it does not yet clarify the boundary of this contribution. In particular, it remains unclear whether RL can generalize when the feasible-region rules, room geometries, furniture combinations, or scene-construction framework are changed.

The GRPO comparisons are useful, but the experimental matching is not described in sufficient detail. The authors should clarify whether all methods use the same training budget, reward normalization, hyperparameter tuning procedure, and random seeds. Results across multiple training runs would also be useful for assessing stability. The heuristic comparison is an important addition, but the heuristic itself is not described well enough to determine whether it is a competitive planning baseline.

The related-work comparison should also be updated. Infinigen Indoors is directly relevant to procedural constraints and indoor scene generation. SceneSmith is a recent agentic/VLM-based framework for constructing physically feasible indoor scenes and should be discussed and, if technically possible, included in the empirical comparison. SceneSpinner, presented in Taming LLMs for Codematic Indoor Scene Generation at ECCV 2026, is also relevant because it fine-tunes an LLM for structured indoor scene generation. These comparisons would help clarify whether the main contribution lies in the procedural layout mechanism, the VLM-based workflow, or the RL post-training.

Finally, the scope of the RL contribution remains unclear. Does the learned policy provide useful gains when applied to another scene-construction framework, such as SceneSmith, or is it effective only with Dovetail3D's particular action schema, feasible-region compiler, and reward design? If the method is tightly coupled to the current workflow, this limitation should be stated explicitly.

Overall, the additional experiments make Dovetail3D a better-supported integrated system. The remaining question is whether the proposed RL and credit-assignment method is an independent and reusable contribution beyond the specific compiler and workflow used in this paper.

I also understand that the rebuttal period is necessarily short, and that a full cross-framework evaluation may not be feasible within this time. Nevertheless, I would appreciate a clearer discussion of the unique insight of this work and its potential impact on the community. In particular, it would be valuable to clarify whether this RL training strategy is mainly effective for the specific Dovetail3D system, or whether it could provide broader benefits to other 3D agents and scene-construction frameworks. I see this as the main potential value of the paper: procedural scene generation and programmatic checking are becoming increasingly accessible to agentic systems, while understanding whether RL-trained VLMs can systematically improve such 3D agents could be a more influential direction for future work. Even a careful discussion of the expected scope and limitations would help the community assess the significance of the contribution.

Reviewer_9yXzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Dovetail3D post-trains a VLM to retrieve, select, and place furniture while coordinating layout and style. A geometric compiler provides feasible placement regions, while RL combines local violation feedback, future feasible-space returns, terminal layout rewards, and asset-specific style credit. Evaluation on 200 held-out 3D-FRONT rooms, seven baselines, and a small human study reports improvements in furnishing completion, physical validity, and stylistic coherence.

Critical strengths
  1. The unified retrieval, selection, and placement workflow addresses both functional layout and stylistic coherence.
  2. The results show promising gains in furnishing completion and placement efficiency, supported by component ablations and human evaluation.
Critical weaknesses
  1. The contribution of the proposed RL method is not fully isolated from the extensive handcrafted constraints. The ablations also leave unclear whether the credit-assignment scheme outperforms simpler alternatives.
  2. Differences in baseline inputs complicate the comparisons, while the small human study and lack of uncertainty estimates limit confidence in the reported gains.
Questions for authors
  1. How would a simple placement heuristic perform using the same feasible regions? This would help distinguish learned improvements from the benefits of the constraint system.
  2. Does asset-specific style attribution outperform assigning the same scene-level reward to all retrieval and selection actions?
  3. Do the improvements hold on a subset where competing methods receive equivalent room geometry and furnishing/style information?
  4. How closely do contact-sheet style scores agree with human judgments of rendered scenes, and are the gains statistically reliable across the 20 evaluated cases?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 9yXz

We thank the reviewer for the constructive feedback and address each point directly below.

W1 / Q1: Isolating RL from Handcrafted Constraints

W1: The contribution of the proposed RL method is not fully isolated from the extensive handcrafted constraints.

We ran three controlled comparisons across the full 200-room test set (see Table R1 in General Response):

  1. With vs. Without the Constraints: Holding the base model and workflow strictly fixed, the handcrafted constraints improve NAV by +12.3%, ACN by +40.2%, OOR by +64.5%, and OAR by +74.1%, while reducing OOB and COL by 78.5% and 40.7%, respectively, to 0.0%. This isolates the contribution of the feasible domains without RL.
  2. Base vs. RL: Holding the constraints strictly fixed, RL post-training improves CNT by +7.0%, NAV by +0.4%, ACN by +1.9%, and OOR by +1.8%. This isolates the contribution of RL beyond the constraints.

Q1: How would a simple placement heuristic perform using the same feasible regions?

  1. Base vs. Heuristic vs. RL: Under the same compiler, the heuristic severely degrades global metrics (NAV/ACN/OOR) by up to 13.4%, exhibiting degenerated behavior. In contrast, RL reverses this drop, outperforming the heuristic by +3.3% to +15.3%.

W2 / Q2: Dovetail3D vs. Simpler alternatives

W2: The ablations also leave unclear whether the credit-assignment scheme outperforms simpler alternatives.

Q2: Does asset-specific style attribution outperform assigning the same scene-level reward to all retrieval and selection actions?

We address this concern directly with two matched GRPO controls in R2 of the General Response:

  1. Vanilla layout GRPO (Table R2a): Compared with vanilla GRPO, Dovetail3D reduces PIR from 10.1% to 0.3%, shortens the average rollout by 4.6 steps, and improves ΣF\Sigma F by +15.8, CNT by +8.4%, OOR by +4.0%, and OAR by +5.6%.
  2. Uniform scene-level style GRPO (Table R2b): Dovetail3D improves all five style dimensions, yielding a total gain of +4.99.

W3 / Q3: Controlled Comparison of Baselines

W3: Differences in baseline inputs complicate the comparisons.

  1. Input geometry strictly follows each baseline's native interface. For a fair comparison, we feed every method the richest inputs its native interface supports: DiffuScene's released unconditional generator accepts neither target room geometry nor furnishing/style requests, whereas I-Design receives only bounding boxes for non-rectangular rooms. Directly supporting arbitrary floor plans is itself a practical strength of Dovetail3D.
  2. Style conditioning is held consistent across LLM-based methods. We provide the same target-style request to every method with an LLM interface.

Q3: Do the improvements hold on a subset where competing methods receive equivalent room geometry and furnishing/style information?

Please see Table R3 in the General Response: on the maximum 100-case rectangular subset, all methods included in the controlled comparison receive identical inputs, and Dovetail3D ranks first on most metrics.

W4 / Q4: Human-Study Uncertainty

W4: while the small human study and lack of uncertainty estimates limit confidence in the reported gains.

  1. Human-Study Scale Relative to Prior Work: We first clarify that recruiting five annotators aligns with established evaluation protocols (e.g., LayoutVLM likewise uses five annotators). Moreover, unlike SceneWeaver's split-assignment protocol, every scene in our study is independently rated by five annotators, yielding denser per-case coverage and reducing variance from individual raters.

  2. Agreement and Uncertainty Estimates (Please see Tables R4a–R4b in the General Response): Table R4a quantifies inter-annotator reliability, showing values on par with or exceeding those reported by SceneWeaver. Table R4b quantifies cross-case uncertainty using bootstrap confidence intervals. Dovetail3D retains the highest mean across all four human-evaluation dimensions.

Q4: How closely do contact-sheet style scores agree with human judgments of rendered scenes, and are the gains statistically reliable across the 20 evaluated cases?

Please see Table R4c in the General Response. Contact-sheet scores show positive agreement with rendered-scene human consensus (r=0.464r=0.464, ρ=0.487\rho=0.487, τb=0.401\tau_b=0.401) and select the exact same top-1 method in 85% of cases. The case-bootstrap 95% CIs in Table R4b quantify cross-case uncertainty in the human ratings, with Dovetail3D retaining consistent gains across cases.

Thank you for the detailed responses and additional experiments. The controlled ablations and evaluation statistics strengthen the paper. I still have a few concerns, and clarifying the points below would help me better understand the paper’s contribution and decide on the final score.

  1. Sequential construction and diversity. Previously accepted objects appear to remain fixed, and recovery when an early valid placement prevents later objects from fitting is unclear. Quantitative diversity across repeated generations with identical room, inventory, and style inputs is also missing.

  2. Generalization and open vocabulary. The room-disjoint split and novelty statistics are helpful, but evaluation remains within the 3D-FRONT/3D-FUTURE setting following post-training on 442 rooms. Broader text-conditioned or open-vocabulary generalization remains untested. Evaluation on a description benchmark such as SceneEval-100/500, beyond adopting its metrics, would help establish the method’s scope.

  3. Attribution to RL. The rebuttal demonstrates useful RL gains, while the compiler accounts for the largest geometric improvements. Since Appendix A.1 also includes relational and look-ahead rules, the distinction between encoded constraints and learned planning remains insufficiently clear.

  4. Asset and style coverage. The exact retrieval catalog subset, size, per-style coverage, and mapping to the nine target styles are not specified. Whether the five held-out styles are already represented in the catalog metadata also remains unclear, limiting interpretation of the style-generalization results.

  5. Human evaluation. The added agreement statistics are useful, but scoring instructions and anchors, blinding, presentation randomization, and the calculation of the aggregate human “Scene Quality” score remain unclear.

  6. Efficiency. GPU models, training GPU-hours, and measured per-scene inference time and cost are missing. Comparable timings before and after RL would help establish whether fewer rollout steps translate into practical efficiency gains.

AuthorsOpenReview ↗

Response to Reviewer 9yXz

We thank the reviewer for the recognition of our work during rebuttal. And we address your concerns as follows.

W1a: Sequential construction and diversity. Previously accepted objects appear to remain fixed, and recovery when an early valid placement prevents later objects from fitting is unclear.

We clarify that the current system uses retry-based recovery after each harness rejection, but deliberately does not roll back an already accepted placement for the following reasonss:

  1. Retry after harness rejection, without rollback. If the harness rejects an asset or coordinate, the agent retries the corresponding generation step. Once the harness accepts a placement, however, that commitment remains fixed: the current action space does not expose rollback or whole-layout relayout. We use this bounded recovery mechanism to avoid excessively long trajectories, deadlocks, and oscillatory rollback--retry cycles that can prevent the agent from completing the scene.

  2. Proactive product selection through RL. Instead of rolling back accepted placements, RL learns furniture order and product choices that preserve feasible space for later objects, using discounted feasible-domain return as the training signal. Against the same workflow without RL, it raises this return from 141.7 to 157.9, reduces invalid placements from 16.2% to 0.3%, and improves completion from 79.0% to 88.4% in the livingroom test set.

W1b: Quantitative diversity across repeated generations with identical room, inventory, and style inputs is also missing.

We additionally sample five rollouts for each of 10 cases while keeping the room geometry, requested object set, retrieval catalog, and target style fixed. Comparing all 10 pairs among the five rollouts of each case gives 100 within-case pairs. Across matched object slots, 78.8% select different assets and 40.4% use different orientations. These representative results show diversity in both asset selection and spatial arrangement:

Figure W1b: https://img.cdn1.vip/i/6afd64bc8b08b_1794991292.webp (Bedroom), https://img.cdn1.vip/i/6afd64c381a34_1794991299.webp (Livingroom)

Figure W1b. Representative diversity under identical inputs. Each row is one fixed room–inventory–style case, and the five columns are independent rollouts.

W2: Generalization and open vocabulary. The room-disjoint split and novelty statistics are helpful, but evaluation remains within the 3D-FRONT/3D-FUTURE setting following post-training on 442 rooms. Broader text-conditioned or open-vocabulary generalization remains untested. Evaluation on a description benchmark such as SceneEval-100/500, beyond adopting its metrics, would help establish the method’s scope.

We and clarify the intended scope of our paper and an extension path:

  1. Current scope: realistic layout under real room geometry. Dovetail3D is designed to construct realistic indoor scenes within real architectural constraints. We therefore retain real room geometries as inputs rather than generating room shells from open-ended descriptions. Our central question is whether RL improves asset selection and long-horizon furniture layout under those constraints, and room-geometry generation is not a claimed contribution. Fixing the room shell also isolates layout quality from errors introduced by a separate geometry generator.

  2. A text-conditioned geometry front end is compatible with our pipeline. For open-vocabulary room descriptions, a room-geometry generator could be inserted before the current harness to produce the room polygon consumed by Dovetail3D. This is a modular decomposition already demonstrated by related systems: SceneSmith begins with a dedicated floor_plan stage that generates walls and floors before furniture placement, while SceneWeaver exposes modular initializer tools for open-vocabulary scene construction. Such a front end would extend the room-description and geometry scope without changing our core layout formulation.

  3. Limitation and future work. Integrating and validating this front end, and then evaluating the resulting end-to-end system on SceneEval-100/500, requires substantial work, which can not be completed reliably within the rebuttal period. We will therefore explicitly state this limitation and list this open-vocabulary asset support as future work.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Response

Dear Area Chair and Reviewers,

We sincerely thank all reviewers for their constructive feedback. In response to your suggestions, we have conducted the following experiments and statistical analyses that directly address shared concerns:

  1. R1 — Isolating RL from Handcrafted Constraints (Reviewer 9yXz, Zyj4, WEp3): We added a controlled comparison for constraints and RL, respectively, isolating the gains of learned spatial planning from the feasible-domain compiler.
  2. R2 — Decision-Aligned Credit vs. GRPO (Reviewer 9yXz, Zyj4): We added (i) a vanilla GRPO experiment for layout credit and (ii) a uniform scene-level GRPO experiment for style credit. Dovetail3D improves layout metrics over the former, and all five style dimensions over the latter.
  3. R3 — Fair Comparison Under Identical Input (Reviewer 9yXz, Zyj4): We added a controlled evaluation in which all included methods receive identical inputs, excluding DiffuScene as its unconditional generator does not accept conditioning inputs. Dovetail3D remains the strongest overall method under this setting.
  4. R4 — Evaluation Statistics (Reviewer 9yXz, Zyj4): We added inter-annotator agreement, bootstrap confidence intervals, and LLM–human agreement. The results support the reliability of the human study and the proxy metric.
  5. R5 — Style Generalization (Reviewer Zyj4): We added zero-shot evaluation on five unseen styles. RL improves under every unseen style, showing that its gains generalize beyond the training samples.
  6. R6 — Inference Efficiency (Reviewer 9yXz, Zyj4): We measured end-to-end generation efficiency before and after RL under the same model architecture and serving configuration. RL reduces both the number of actions and the resource-normalized inference cost without adding any inference-time module.

The detailed experiment results are listed below.


R1: Isolating RL from Handcrafted Constraints (Reviewer 9yXz, Zyj4, WEp3)

We first clarify that

  1. The feasible-region compiler is an integral part of Dovetail3D. Formulating local rules as an explicit action-space compiler is a core design choice of our framework and its gains are therefore part of Dovetail3D's methodological contribution.
  2. The compiler and RL optimize orthogonal objectives. The compiler uses action masking to restrict outputs to locally valid coordinates (OOB, COL), pruning degenerate trajectories during training. On top of this design, RL then focuses on global, long-horizon objectives (CNT, ACN), the two components are strictly complementary rather than overlapping.
  3. We separately ablate the constraints, heuristic, and RL across all 200 test rooms. Below, we compare the Qwen3.5-9B base model without constraints (Base w/o compiler), the same base model with the compiler but without RL (Base), a non-learned placement Heuristic using the same constraints, and full RL post-training (Dovetail3D). We additionally replace only the policy model with GPT-5.4, while keeping the Dovetail3D workflow, compiler, assets, inputs, and test set fixed. GPT-5.4 is evaluated directly without task-specific post-training. Layout metrics are reported in %, while REAL, FUNC, LAY, and Judge use a 0–10 scale.

Table R1. Ablation of handcrafted constraints and RL on test set.

VariantHandcrafted constraintsCNT↑NAV↑ACN↑OOB↓COL↓OOR↑OAR↑REAL↑FUNC↑LAY↑Judge↑
Base w/o compilerNo90.886.541.478.540.725.923.83.505.903.754.38
BaseYes84.898.881.60.00.090.497.95.456.204.905.50
Base - Base w/o compiler—-6.0+12.3+40.2↓78.5↓40.7+64.5+74.1+1.95+0.30+1.15+1.12
HeuristicYes88.587.568.20.00.077.687.75.305.384.585.08
Dovetail3D (GPT-5.4)Yes93.591.879.90.00.094.092.65.616.505.055.73
Dovetail3DYes91.899.283.50.00.092.297.85.746.805.175.90
RL - BaseYes+7.0+0.4+1.90.00.0+1.8−0.1-0.1+0.29+0.60+0.27+0.40
RL - HeuristicYes+3.3+11.7+15.30.00.0+14.6+10.1+0.44+1.42+0.59+0.82

For OOB and COL, ↓\downarrow denotes a reduction in the violation rate.

R2: Dovetail3D vs. GRPO (Reviewer 9yXz, Zyj4)

First, we compare Base (w/o RL), a matched GRPO control for layout credit, and Dovetail3D on the 53-room Living Room test set:

Table R2a. Comparison of Base, GRPO, and Dovetail3D in layout.

VariantPIR↓Steps↓ΣF\Sigma F↑CNT↑NAV↑ACN↑OOR↑OAR↑REAL↑FUNC↑LAY↑Judge↑
Base16.227.9141.779.099.873.282.01005.25.74.55.1
GRPO10.128.0142.180.098.875.182.494.45.25.84.35.1
Dovetail3D0.323.4157.988.499.874.686.41005.36.34.65.4

Second, we compare Base (w/o RL), Dovetail3D's asset-level style attribution, and a scene-level GRPO baseline that broadcasts the same style credit to all actions:

Table R2b. Comparison of Base, GRPO, and Dovetail3D in style.

Style dimensionBase (w/o RL)GRPODovetail3DΔ\Delta vs. Uniform GRPO
Style consistency7.77367.02868.4151+1.3865
Color harmony9.73588.85719.7358+0.8787
Material / texture8.15098.05718.7170+0.6599
Shape / form7.73587.71438.3774+0.6631
Visual atmosphere7.69816.97148.3774+1.4060

Notably, in our experiments, standard GRPO frequently suffers from policy collapse and is highly sensitive to local format-penalty tuning—yielding negligible or even negative gains (as shown in Table R2a, R2b)—which directly motivates our fine-grained, decision-aligned credit assignment.

R3: Fair Comparison Under Identical Input (Reviewer 9yXz, Zyj4)

In our full benchmark, each baseline receives the richest inputs supported by its native API. To eliminate input differences, we evaluate all compatible baselines on the 100-case rectangular subset of the test set, where all included methods receive identical room geometry and furnishing/style information. DiffuScene is excluded from this controlled comparison because its unconditional generator accepts none of the conditioning inputs.

Table R3. Evaluation of compatible methods on the rectangular subset with identical inputs.

MethodCNT↑NAV↑ACN↑OOB↓COL↓OOR↑OAR↑Style↑REAL↑FUNC↑LAY↑Judge↑
LayoutGPT73.495.125.65.451.070.275.77.35.95.25.25.4
LayoutVLM98.784.727.310.885.049.460.77.24.84.03.64.2
I-Design60.497.148.615.016.043.047.46.64.34.83.84.3
ReSpace86.798.032.44.77.033.669.27.05.85.44.95.3
SceneWeaver71.495.252.30.62.241.061.97.64.95.14.24.7
Holodeck83.898.370.20.00.046.279.77.15.36.24.35.3
Dovetail3D88.799.384.10.00.094.898.88.25.97.05.26.1
Dovetail3D Δ\Delta−10.0-10.0+1.0+13.9↓0.6↓2.2+24.6+19.1+0.6+0.1+0.8+0.3+0.7

Δ\Delta is computed against the strongest baseline w.r.t. Dovetail3D.

R4: Evaluation Statistics (Reviewer 9yXz, Zyj4)

For the human study, we first report human–human agreement. Following SceneWeaver, for each evaluation dimension, we treat each annotator's scores over the same 160 method–scene outputs as a ranking, compute Kendall's τb\tau_b for each of the (52)=10\binom{5}{2}=10 annotator pairs (with τb\tau_b accounting for tied scores), and report the mean pairwise agreement:

Table R4a. Human–human agreement measured by Kendall's τb\tau_b.

Human–Human agreement (τb\tau_b)REALFUNCLAYStyle
SceneWeaver0.430.420.45—
Ours0.410.500.500.40

Then we report case-bootstrap confidence intervals below. For each method and evaluation dimension, we first average the five annotator ratings within each of the 20 cases. We then resample the 20 cases with replacement 100,000 times, using the same sampled case indices across methods, and recompute the mean for each bootstrap sample. The 2.5th and 97.5th percentiles of this bootstrap distribution form the reported 95% confidence interval.

Table R4b. Human evaluation scores reported as Mean [95% CI].

MethodREALFUNCLAYStyle
LayoutGPT5.85 [5.23, 6.53]7.25 [6.75, 7.70]5.78 [5.10, 6.48]6.23 [5.70, 6.80]
LayoutVLM3.75 [3.35, 4.20]5.48 [5.23, 5.75]3.20 [2.88, 3.55]5.97 [5.47, 6.50]
I-Design5.21 [4.68, 5.71]6.29 [5.43, 6.96]4.86 [4.25, 5.46]5.20 [5.03, 5.82]
SceneWeaver5.13 [4.50, 5.79]5.61 [4.79, 6.34]4.55 [3.84, 5.29]6.49 [5.61, 7.33]
Holodeck7.18 [6.68, 7.68]8.12 [7.71, 8.50]7.03 [6.44, 7.62]7.33 [6.86, 7.76]
DiffuScene3.50 [2.65, 4.45]5.55 [4.70, 6.40]3.25 [2.60, 3.95]7.00 [6.10, 7.90]
ReSpace5.60 [5.00, 6.25]6.85 [6.10, 7.55]5.05 [4.20, 5.95]7.90 [7.30, 8.50]
Dovetail3D8.13 [7.85, 8.40]8.38 [8.05, 8.68]7.88 [7.48, 8.28]8.93 [8.10, 9.30]

Finally, we compare the contact-sheet style scores with human judgments of the corresponding rendered scenes across all 160 method–scene outputs:

Table R4c. Agreement between contact-sheet style scores and rendered-scene human judgments.

Human targetPearson rrSpearman ρ\rhoKendall τb\tau_b
Human consensus0.4640.4870.401

At the scene level, the contact-sheet proxy selects the exact same top-1 method as human consensus in 17 of 20 scenes (85%), supporting its utility as a proxy during training.

R5: Style Generalization (Reviewer Zyj4)

We evaluate style generalization of Base and Dovetail3D on the 100-case bedroom test set used in the main paper. The seen-style row summarizes the original nine training styles, while the remaining rows report zero-shot results on five unseen style categories:

Table R5. Zero-shot style generalization on the 100-case bedroom-family test set.

StyleBaseDovetail3DΔ\Delta
Seen styles average (9 styles)8.568.82+0.26
Industrial5.316.13+0.82
Kids5.276.32+1.05
Korean8.048.99+0.95
Mediterranean6.426.81+0.39
Southeast Asian4.946.69+1.75
Unseen styles average (5 styles)6.006.99+0.99

R6: Inference Efficiency (Reviewer 9yXz, Zyj4)

Base and Dovetail3D have the same inference footprint: both use the identical Qwen3.5-9B architecture and serving configuration, and RL changes only the weights without adding an inference-time module. We deploy eight independent vLLM replicas on 8 H20 GPUs (one replica per H20, TP=1) with bfloat16, and report end-to-end wall-clock generation over the complete 100-scene Bedroom test split. Timing starts immediately before the 100 cases are dispatched to the eight initialized and warmed-up workers and stops after the last final layout is written. It includes policy calls, asset retrieval, feasible-region compilation, action validation and retries, and environment-state updates. Model loading, model compilation and warm-up are excluded. The resource-normalized cost is the wall-clock runtime multiplied by eight H20 GPUs and divided by 100 scenes, corresponding to an estimated single-H20 inference time per scene.

Table R6. Before/after-RL inference efficiency on the Bedroom100 test split.

MethodActions / sceneBedroom100 runtimeEffective throughputResource-normalized cost
Base20.0141 min 02 s0.410 min / scene3.28 min / scene
Dovetail3D14.1728 min 48 s0.288 min / scene2.30 min / scene

We thank the reviewers again for the constructive feedback, and we hope these clarifications and additional results fully address your concerns, and we are happy to provide further details if needed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.