acceptodds
Under review as a conference paper at ICLR 2027

SceneWise: Learning a Decision-Consequence World Model for 3D Indoor Scene Generation

Abstract

Recent agentic frameworks for 3D indoor scene generation integrate generation, placement, and editing tools to construct scenes from natural language instructions, refining intermediate layouts through feedback. However, they overlook how currently valid decisions can prevent subsequent spatial and functional requirements from being met, leading to delayed failures and costly revisions. We propose SceneWise, a framework that learns a decision-consequence world model to guide scene generation and repair. During generation, the world model predicts how candidate decisions affect placement feasibility and constraint satisfaction, guiding agents to avoid downstream conflicts. During repair, an attribution model uses the failure context and historical decision graph to rank earlier decisions for revision. The same world model evaluates replacement candidates to guide local regeneration while limiting changes to valid scene content. We further introduce SceneWiseDB, a dataset of 100k records across 20 room categories that provides consequence and intervention supervision for both models. Experiments show that SceneWise improves task success and constraint satisfaction while reducing repeated editing, and enables effective repair with limited changes to existing scenes.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D scene generation, world models, large language models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_SmVnOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents SceneWise, a framework for indoor 3D scene generation that introduces a learned decision-consequence world model and a historical attribution model. The world model predicts the downstream consequences of candidate scene-editing decisions, including changes in placement feasibility, constraint margins, potential failures, and affected objects. These predictions are used to rank candidate actions before execution-based verification. When scene generation fails, the attribution model ranks historical decisions according to whether tested revisions have been effective, and the system performs local regeneration to repair the scene while minimizing unnecessary changes. The paper also introduces SceneWiseDB, a dataset containing 100,000 decision-consequence and matched-intervention records. Experiments on 94 room-level prompts, 100 tasks with coupled spatial and functional requirements, and additional repair and generalization settings suggest that SceneWise improves complete requirement satisfaction and reduces editing operations compared with several existing scene-generation methods.

Critical strengths
  1. A learned world model for predicting downstream decision consequences. SceneWise introduces a decision-consequence world model that predicts structured outcomes of candidate scene-editing decisions, including placement-feasibility changes, constraint-margin changes, and potential failures. This goes beyond checking whether an action is immediately valid and provides a principled way to assess its potential downstream effects.
  2. A learned historical attribution model for failure recovery. The paper introduces a historical attribution model that ranks previous decisions according to whether tested revisions can effectively resolve a failure.
Critical weaknesses

Insufficiently specified training data generation pipeline. SceneWiseDB is central to training both the decision-consequence world model and the historical attribution model. However, although Appendix B describes how alternative decisions are executed in copies of partial scenes and how execution outcomes are converted into supervision, the upstream source of the offline generation trajectories remains unclear. Specifically, the paper does not clearly identify which agent or scene-generation system produces these trajectories, how the initial source scenes are generated and sampled, or how candidate decisions are proposed during the data collection process. This is an important limitation because the distribution and diversity of the source trajectories directly affect what the learned models can capture. Without understanding the upstream generation process, it is difficult to assess the coverage of SceneWiseDB, determine whether the learned models are biased toward a particular generation policy, or reproduce the proposed dataset and training pipeline. I encourage the authors to provide a more complete description of the source-scene generation system, trajectory collection procedure, and candidate-sampling strategy.

Questions for authors
  1. Construction of the 70,000 decision-consequence records. Could the authors clarify how the 70,000 decision-consequence records are generated? In particular, which agent or scene-generation system produces the initial offline trajectories, and how are source scenes, partial-scene checkpoints, and alternative candidate decisions sampled? A more detailed description of the complete data-generation pipeline would help assess the dataset's coverage and reproducibility.
  2. Clarification of the mathematical notation. Could the authors simplify or clarify the notation in Sections 3.2–3.4, particularly the candidate-selection and execution-verification equations? For example, explicitly distinguishing predicted consequences and risks from execution-measured consequences and risks, and providing a step-by-step explanation of the roles of (K), (B), and (H), would make the proposed framework easier to understand and reproduce.
  3. Comparison with GPT-6 Astra. Could the authors compare SceneWise with a strong general-purpose LLM agent, such as GPT-6 Astra, using the same scene-editing tools, task inputs, and execution-verification budget? Such a comparison would help clarify whether the proposed learned consequence model and historical attribution mechanism provide benefits beyond those of a capable LLM agent with iterative feedback and search.
  4. Potential data leakage between training and evaluation. Although SceneWiseDB is partitioned by source-scene groups to prevent overlap across training, validation, and test splits, the separation between the training data and the downstream evaluation tasks remains unclear. Could the authors clarify whether the 94 SceneEval prompts and 100 coupled-requirement tasks are entirely independent of the source scenes used to construct SceneWiseDB? In particular, were any checks performed to identify overlapping or highly similar scene layouts, asset configurations, or task requirements? Further clarification would help rule out potential data leakage and strengthen confidence in the reported results.
4 comments
AuthorsOpenReview ↗

Authors' Response to Reviewer SmVn

Thank you for recognizing the methodological strengths of our work. We appreciate your constructive feedback and respond to each of your concerns below.

Q1. Construction of the 70,000 decision-consequence records

We sampled checkpoints from selected source scenes and executed alternative decisions to measure their immediate and delayed consequences, yielding 70,000 records after deduplication. We describe this process step by step below.

  1. Source-scene generation and selection. Our scene-generation pipeline built on SceneSmith [1] generated new scenes and offline trajectories from structured prompts. We selected completed scenes restorable with all required assets, cycling through room types during expansion.

  2. Trajectories and partial-scene checkpoints. From the offline generation trajectories, we selected checkpoints before successful tool calls. We also moved up to four independently movable furniture objects per restored scene, saving up to three checkpoints before moves with a subsequent action.

  3. Alternative candidate decisions. At each checkpoint for a move, we constructed 25 candidates. We kept the recorded move as a control and sampled 24 alternative positions from a stratified grid covering the room interior and areas near its boundaries, keeping the object and its orientation fixed.

  4. Execution and consequence measurement. We executed each candidate in an independent scene copy under the same continuation policy. We measured its immediate and delayed consequences over a horizon of at most three decisions, including the candidate itself.

  5. Record construction. One decision–consequence record consists of a partial-scene context, a candidate decision, and the consequences measured from its executed continuation. Different candidates evaluated from the same partial scene count as separate records.

[1] Pfaff, N., Cohn, T., Zakharov, S., Cory, R., & Tedrake, R. (2026, February). Scenesmith: Agentic generation of simulation-ready indoor scenes. In Forty-third International Conference on Machine Learning.

Q2. Clarification of the mathematical notation in Sections 3.2–3.4

1. Predicted versus execution-measured consequences and risks. Eq. (1) defines the predicted and measured consequences of candidate decision d at step t:

Ĉₜᴴ(d) = Wᵩ(Sₜ, Qₜ, Gₜ, P, d), Cₜᴴ(d) = (ΔF, ΔR, V, A).

The hat distinguishes predictions from execution-derived measurements. The four components are Placement Feasibility Changes (ΔF), Constraint Margin Changes (ΔR), Potential Failures (V), and Affected Objects (A).

Eq. (3) defines the risk function applied to either set of consequences:

R(C) = α R_F(C) + β R_R(C) + γ R_V(C).

The three terms penalize decreases in placement feasibility, decreases in comparable constraint margins, and failure events, respectively. Thus, R(Ĉₜᴴ(d)) is the predicted risk, and R(Cₜᴴ(d)) is the execution-measured risk.

2. Roles of K, B, and H in candidate selection and execution verification. Eq. (4) contains two expressions. The first shortlists candidates by predicted risk, and the second selects the next decision by execution-measured risk.

  • K — propose candidates. The Designer proposes K candidate decisions. Uₜᵛᵃˡⁱᵈ contains those that pass the immediate collision, boundary, and hard-constraint checks.

  • B — shortlist candidates. The first expression in Eq. (4) is

    𝒦ₜ = TopB_{d ∈ Uₜᵛᵃˡⁱᵈ} [−R(Ĉₜᴴ(d))].

    TopB returns up to B candidates with the highest scores inside the brackets. Because the score is negative predicted risk, the shortlist 𝒦ₜ contains the candidates with the lowest predicted risks.

  • H — verify each shortlisted branch. We execute each candidate and follow the continuation policy π for up to H − 1 further decisions. Thus, H includes the candidate itself. 𝒦ₜᵒᵏ contains candidates whose branches complete the task or reach the horizon without failure. The second expression in Eq. (4) is

    d*ₜ₊₁ = argmin_{d ∈ 𝒦ₜᵒᵏ} R(Cₜᴴ(d)).

    argmin returns the candidate with the lowest execution-measured risk. We commit only that decision, d*ₜ₊₁. If all shortlisted branches fail, we invoke scene repair.

In Section 3.4, the world model likewise predicts the consequences of proposed replacements, and matched interventions measure their effects.

AuthorsOpenReview ↗

Authors' Response to Reviewer SmVn (continue)

Q3. Comparison with GPT-6 Astra

We compared SceneWise with a GPT-6 Astra agent using the same scene-editing tools, task inputs, and execution-verification budget of 60 calls per attempt. Both tables below report means with 95% confidence intervals.

Generation on 100 tasks with coupled spatial and functional requirements

Table R1. Comparison with GPT-6 Astra on 100 coupled-requirement tasks.

MethodSR ↑ (%)CSR ↑ (%)EditOps ↓Edits ↓
GPT-6 Astra agent79.0 ± 7.392.6 ± 1.57.6 ± 1.14.2 ± 0.6
SceneWise84.0 ± 6.494.1 ± 1.84.8 ± 0.73.2 ± 0.5

SceneWise increased complete requirement satisfaction from 79.0% to 84.0% while reducing edit operations from 7.6 to 4.8 per attempt. This suggests that consequence-guided generation helps SceneWise satisfy coupled requirements with fewer editing operations.

Repair on 100 shared failed-scene checkpoints

Table R2. Comparison with GPT-6 Astra on 100 shared repair checkpoints.

MethodSR ↑ (%)CSR ↑ (%)SR (E ≤ 5) ↑ (%)Edits ↓Calls ↓Time ↓ (min/attempt)
GPT-6 Astra agent82.7 ± 5.894.7 ± 1.769.0 ± 7.94.2 ± 0.621.4 ± 1.611.2 ± 0.8
SceneWise86.0 ± 6.196.2 ± 1.578.0 ± 7.12.6 ± 0.416.0 ± 1.48.4 ± 0.7

SceneWise successfully repaired the scene while changing at most five existing objects in 78.0% of attempts, compared with 69.0% for GPT-6 Astra. It also modified fewer checkpoint objects on average (2.6 vs. 4.2) and used fewer verification calls per attempt (16.0 vs. 21.4). These results suggest that historical attribution and local repair help SceneWise target effective revisions while preserving more of the existing scene.

Q4. Potential data leakage between training and evaluation

Neither evaluation set was used to construct SceneWiseDB. We found no exact or near-duplicate matches for any of the 94 SceneEval prompts or 100 coupled-requirement tasks (see table below). We compared each task with both template-generated and imported SceneWiseDB source prompts based on required object categories, quantities, spatial relations, and functional constraints. We also compared the layouts and asset configurations of the generated evaluation scenes with those of the source scenes.

Table R3. Overlap between evaluation tasks and SceneWiseDB source scenes.

Aspect checkedSceneEval promptsCoupled-requirement tasks
Task specifications0/940/100
Scene layouts0/940/100
Asset configurations0/940/100
Reviewer_SmVnOpenReview ↗

response

Thank you for the detailed rebuttal. The authors’ clarifications address my concerns regarding data construction and potential data leakage between training and evaluation. The additional comparison with GPT-6 Astra further supports the effectiveness of the proposed method. I therefore support acceptance of the paper.

AuthorsOpenReview ↗

Thank you for your feedback

We are glad that your concerns have been addressed. Thank you for your positive feedback. We will incorporate your suggestions into the revised manuscript.

Reviewer_HiMeOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

SceneWise introduces a decision–consequence learning framework for 3D indoor scene generation and repair. During generation, a learned model predicts how candidate operations affect subsequent placement feasibility, functional constraints, and potential failures. When a failure occurs, a separate attribution model ranks earlier decisions for revision, while matched execution verifies the effectiveness of proposed repairs. The authors also introduce SceneWiseDB, containing 100,000 supervision records. Experiments show improved requirement satisfaction with fewer editing operations and lower runtime.

Critical strengths

S1. Well-motivated problem and coherent integration of generation and repair The paper addresses a practical challenge in sequential scene construction: a currently valid placement can cause downstream failures, and the object exhibiting a failure may not be the one that needs modification. Explicit consequence prediction guides candidate selection, while historical attribution prioritizes earlier decisions for revision. Reusing the consequence model during repair connects these components coherently, and matched execution helps verify whether improvements result from the proposed modification. S2. Comprehensive evaluation with reasonably controlled ablations The paper provides process-level supervision and evaluates prediction quality, scene generation, failure recovery, editing cost, runtime, and generalization under specified held-out conditions. Internal ablations share the backbone, tools, and verification budget: removing generation guidance or learned attribution reduces success from 84% to 76% and 78%, respectively. These controlled comparisons provide direct evidence that both components contribute to the overall performance.

Critical weaknesses

W1. Incomplete coverage of closely related methods and limited comparability of external baselines The paper omits SceneReVis, a closely related learned iterative scene-generation method. Consequently, the additional benefits of explicit consequence prediction and historical attribution over an agent already trained for step-wise construction and correction remain insufficiently established. Moreover, external baselines retain their original pipelines without a shared backbone, and their specific model configurations are not sufficiently documented, making it difficult to attribute the overall performance gap entirely to the proposed mechanisms. W2. The additional benefit of structured consequence prediction needs stronger evidence The paper includes a direct-risk prediction baseline, but the full consequence model improves success from 85.0% to 88.3%, and the corresponding table provides neither confidence intervals nor a paired significance test. This result suggests that structured prediction is useful, but does not yet establish how consistently—or by how much—the more elaborate formulation improves upon a simpler risk-scoring approach.

Questions for authors

Q1. Comparison with related learned methods Could the authors discuss and compare with recent learned iterative scene-generation methods, such as SceneReVis, under reasonably aligned tasks, model capabilities, and execution budgets, to clarify the additional benefits of explicit consequence prediction and historical attribution? Q2. Handling delayed conflicts beyond the lookahead horizon How does SceneWise perform when an early placement decision causes a conflict that emerges beyond its three-step lookahead horizon? A controlled evaluation across different failure delays would help establish whether consequence-guided generation prevents these conflicts and reduces subsequent repair effort.

4 comments
AuthorsOpenReview ↗

Authors' Response to Reviewer HiMe

Thank you for recognizing the motivation and comprehensive evaluation of our work. We appreciate your constructive feedback and respond to each of your concerns in detail below.

Q1 & W1. Comparison with related learned methods

Comparison with SceneReVis

SceneReVis learns step-wise construction and correction from visual feedback, while SceneWise uses explicit consequence prediction to assess how candidate operations affect subsequent spatial and functional requirements. We evaluated SceneReVis using its original pipeline. The table below compares both methods on the same 100 tasks with coupled spatial and functional requirements, using three seeds per task, a common evaluator, and a budget of 60 execution-verification calls per attempt.

Table R1. SceneReVis comparison and SceneWise ablations.

MethodSR ↑ (%)CSR ↑ (%)EditOps ↓Edits ↓
SceneReVis78.391.59.75.1
SceneWise w/o consequence-guided generation76.0 ± 7.490.8 ± 2.111.7 ± 1.65.8 ± 0.8
SceneWise w/o learned attribution (historical attribution)78.0 ± 7.191.6 ± 2.09.6 ± 1.35.0 ± 0.7
SceneWise (Full)84.0 ± 6.494.1 ± 1.84.8 ± 0.73.2 ± 0.5

SceneWise achieves a higher task success rate than SceneReVis (84.0% vs. 78.3%) while using about half as many editing operations per attempt (4.8 vs. 9.7). During generation, SceneWise benefits from anticipating how a placement affects later requirements. With consequence-guided generation enabled, it completes more tasks and requires fewer edits. During repair, SceneWise uses historical attribution to identify earlier decisions worth revising, achieving higher task success while modifying fewer distinct objects (3.2 vs. 5.0 per attempt).

Comparison with external baselines under the shared backbone

Table R2. External baselines comparison.

SettingMethodSR ↑ (%)CSR ↑ (%)EditOps ↓Edits ↓
Original pipelineHolodeck36.7 ± 8.069.8 ± 3.45.4 ± 0.82.6 ± 0.4
Original pipelineSAGE62.0 ± 8.185.3 ± 2.512.8 ± 1.85.6 ± 0.8
Original pipelineSceneSmith57.3 ± 8.482.8 ± 2.816.4 ± 2.27.2 ± 1.0
Shared backboneHolodeck strategy43.3 ± 3.774.2 ± 3.05.6 ± 1.02.8 ± 0.5
Shared backboneSAGE strategy67.3 ± 6.087.5 ± 2.29.9 ± 0.75.3 ± 0.7
Shared backboneSceneSmith strategy69.7 ± 5.286.8 ± 2.513.6 ± 1.76.1 ± 1.0
Shared backboneSceneWise (Full)84.0 ± 6.494.1 ± 1.84.8 ± 0.73.2 ± 0.5

Values are means ± the half-width of the 95% confidence interval.

Table R2 reports the original-pipeline results for Holodeck (gpt-4-1106-preview), SAGE (gpt-oss-120b and Qwen3-VL-30B-A3B-Instruct), and SceneSmith (GPT-5.2). We reran all three strategies on the same 100 tasks using SceneWise’s Qwen3.8-27B backbone and execution framework. All three improved in SR. SceneWise still achieved the highest SR (84.0% vs. 69.7% for SceneSmith) and CSR (94.1% vs. 87.5% for SAGE), with the fewest editing operations (4.8 per attempt). This suggests that SceneWise’s generation and repair strategy contributes to its advantage even when the backbone is held fixed.

AuthorsOpenReview ↗

Authors' Response to Reviewer HiMe (continue)

Q2. Handling delayed conflicts beyond the lookahead horizon

We compared SceneWise with and without consequence-guided generation across failure delays. We grouped cases by failure delay, measured from the early placement decision to the first verifiable target conflict on the reference trajectory. The table below reports results for 100 paired cases per group.

Table R3. Target conflicts and repair effort by failure delay.

Failure delayPaired casesPre-repair target conflict rate ↓, w/o guidancePre-repair target conflict rate ↓, FullRepair Edits/case ↓, w/o guidanceRepair Edits/case ↓, Full
1–3 steps10030%15%0.760.38
4–6 steps10029%21%0.750.56
≥7 steps10034%29%0.830.70

w/o guidance denotes SceneWise without consequence-guided generation. Repair Edits/case counts distinct objects modified during repair, averaged over all cases, including those requiring no repair.

Beyond the initial three-step horizon, SceneWise still reduced both the target conflict rate and subsequent repair effort. The reductions were larger at 4–6 steps (8 percentage points and 0.19 fewer objects modified per case) than at ≥7 steps (5 percentage points and 0.13 fewer objects per case). Thus, even for conflicts that emerged beyond the initial horizon, better decisions during generation reduced the amount of editing needed later.

W2. Additional benefit of structured consequence prediction

To assess how much structured consequence prediction improves lookahead selection over direct-risk prediction and how reliable the observed gains are, we compared the two models on the same partial scenes in the table below.

Table R4. Lookahead selection and computational cost.

PredictorSR ↑ (%)CSR ↑ (%)Candidate-set scoring ↓ (s/step)Execution-verification calls ↓ (calls/attempt)End-to-end time ↓ (min/attempt)
Direct-risk predictor85.0 ± 4.294.6 ± 1.61.6 ± 0.125.1 ± 1.231.2 ± 0.8
Full consequence model88.3 ± 3.896.0 ± 1.42.1 ± 0.123.8 ± 1.031.5 ± 0.7
Two-sided paired permutation p-value0.0340.041———

Predictor results are reported as point estimates ± the half-width of the 95% confidence interval.

The full consequence model led to more successful completions (SR 85.0% to 88.3%, p = 0.034) and satisfied a larger fraction of the constraints (CSR 94.6% to 96.0%, p = 0.041). Although scoring took 0.5 seconds longer per step, SceneWise used fewer verification calls, and end-to-end time increased by only 0.3 minutes per attempt. These results show that structured consequence prediction helps SceneWise select placements that better preserve downstream spatial and functional requirements, improving scene completion at nearly the same overall cost.

Thank you for the detailed response and comprehensive new experiments. The comparison with SceneReVis and the shared-backbone evaluations of external baselines clearly establish the benefits of your framework, while the failure-delay analysis and paired significance tests (with 95% CIs) convincingly support the value of consequence-guided generation and structured consequence prediction. My concerns are satisfactorily addressed, and I maintain my positive evaluation of the paper.

AuthorsOpenReview ↗

Thank you for your feedback

We are glad that your concerns have been addressed. Thanks very much for your reply. We will further improve the paper in the next version based on your reviews.

Reviewer_N7rvOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SceneWise addresses delayed spatial and functional conflicts in text-guided 3D indoor scene generation. A learned consequence model evaluates construction and repair candidates, while a historical attribution model identifies earlier decisions worth revisiting. Repairs are verified through matched interventions. The paper introduces SceneWiseDB with 100k supervision records and validates the framework through generation, prediction, and repair experiments, supported by controlled ablations and generalization tests.

Critical strengths
  1. Well-motivated and coherent method. The framework addresses an important limitation: a locally valid placement can obstruct later requirements. Reusing consequence prediction for generation and targeted repair provides a consistent approach to this problem.
  2. Strong empirical support. On coupled-requirement tasks, SceneWise achieves 84.0% success versus 62.0% for the strongest external baseline, with fewer editing operations (Table 2). Controlled ablations and repair comparisons from shared failed checkpoints support the contributions of the individual components.
Critical weaknesses
  1. The benefit of structured consequence prediction needs stronger support. Table 15 shows that the full model improves success from 85.0% to 88.3% over a simpler direct-risk predictor, but provides no uncertainty estimates or computational-cost comparison. Although the overall framework performs well, the robustness and practical benefit of this central modeling choice remain less clearly established.
  2. Dataset diversity is insufficiently characterized. SceneWiseDB reports room and operation frequencies, but provides limited statistics on asset/layout diversity, scene complexity, delayed failures, and attribution-label balance. Since its 100k records include multiple branches and interventions from 5,000 source-scene groups, these statistics are needed to assess the coverage of distinct decision dependencies and repair scenarios.
Questions for authors
  1. Can you provide paired confidence intervals for the improvement over direct-risk prediction in Table 15, together with a computational-cost comparison?
  2. How many unique assets, object categories, and layout families does SceneWiseDB contain? What are the distributions of objects and constraints per source scene?
  3. What proportion of consequence records contain delayed failures? How are failure types and positive/negative attribution labels distributed, and how many distinct failure contexts underlie the intervention records?
  4. What specific Designer and continuation-policy configurations are used, and how are functional constraints and their validation thresholds defined?
  5. Why do Tables 3 and 16 report different failure-event F1 scores for the full model—87.9 and 85.0, respectively? Are different subsets or aggregation rules used?
6 comments
AuthorsOpenReview ↗

Authors' Response to Reviewer N7rv

Thanks for recognizing the motivation behind SceneWise and the empirical support for its effectiveness. We sincerely appreciate your thoughtful questions and constructive feedback. We respond to each of your concerns in detail below.

Q1. Paired confidence intervals and computational-cost comparison

We used the setup of Table 15 to compare the full consequence model with the direct-risk predictor. Both predictors used the same backbone, training records, candidate sets, continuation policy, and execution-verification budget. We measured scoring time on identical candidate sets and averaged execution-verification calls and elapsed time over all 600 attempts per predictor, from each shared partial scene to termination. Table R1 summarizes the results.

Table R1. Lookahead-selection performance and computational cost of the direct-risk predictor and the full consequence model.

PredictorSR ↑ (%)CSR ↑ (%)Scoring time ↓ (s/step)Calls ↓ (calls/attempt)End-to-end time ↓ (min/attempt)
Direct-risk predictor85.0 ± 4.294.6 ± 1.61.6 ± 0.125.1 ± 1.231.2 ± 0.8
Full consequence model88.3 ± 3.896.0 ± 1.42.1 ± 0.123.8 ± 1.031.5 ± 0.7

Values are point estimates ± the half-width of the 95% confidence interval. Scoring time is measured per candidate set. Calls are execution-verification calls.

Compared with the direct-risk predictor, the full consequence model improves SR by 3.3 % (paired 95% CI [−0.3, +7.0]) and CSR by 1.4 % (paired 95% CI [−0.1, +2.9]). Scoring a candidate set takes 0.5 s longer per step, but execution-verification calls fall from 25.1 to 23.8 per attempt, and end-to-end time increases by only 0.3 min per attempt. These results suggest that structured consequence prediction improves candidate selection, helping SceneWise complete more scenes at nearly the same end-to-end runtime.

Q2. SceneWiseDB composition and per-scene distributions of objects and constraints

SceneWiseDB contains 8,231 unique assets across 155 object categories and covers 326 layout families. Across 5,000 source scenes, the mean object count is 31.5 ± 4.7 (mean ± standard deviation [SD], range 22–41), and about 80% of scenes contain 26–37 objects. Each scene has an average of 8.2 ± 1.3 spatial and functional constraints (range 5–11), with about 75% of scenes containing 7–9 constraints. We present the detailed dataset statistics and per-scene distributions in the four tables below.

Table R2. SceneWiseDB composition.

StatisticTotal
Unique assets8,231
Object categories155
Layout families326
Source-scene groups5,000
Room categories20

Table R3. Per-source-scene statistics.

Statistic per source sceneMean ± SDMin–max
Objects31.5 ± 4.722–41
Spatial and functional constraints8.2 ± 1.35–11
Distinct object categories15.2 ± 3.18–23
Object density (objects/m² of floor area)1.05 ± 0.280.78–1.52

Per-source-scene statistics are reported as mean ± SD (min–max).

Table R4. Distribution of objects per source scene.

Objects per source sceneSource scenesShare (%)
22–254929.84
26–291,26325.26
30–331,50630.12
34–371,23124.62
38–4150810.16
Total5,000100.00

Table R5. Distribution of spatial and functional constraints per source scene.

Number of constraintsNumber of source scenesShare of source scenes (%)
5961.92
63396.78
799119.82
81,53730.74
91,19423.88
1065413.08
111893.78
Total5,000100.00
AuthorsOpenReview ↗

Authors' Response to Reviewer N7rv (continue)

Q3. Delayed failures, failure types, attribution labels, and distinct failure contexts

Delayed failures occur in 27.0% of consequence records. Insufficient clearance and entrance obstruction are the two most common first failures, together accounting for 60.0% of delayed-failure records. Of the 12,099 attribution labels, 36.0% are positive and 64.0% are negative. The 30,000 intervention records cover 4,310 distinct failure contexts. The table below gives the full breakdown.

Table R6. Delayed failures, first failure types, attribution labels, and distinct failure contexts in SceneWiseDB.

StatisticCountShare or counting unit
Consequence records containing delayed failures18,90027.0% of 70,000 consequence records
First failure typeCountShare of the 18,900 delayed-failure records
Insufficient clearance6,61535.0%
Entrance obstruction4,72525.0%
Relational violation3,78020.0%
Collision2,26812.0%
Inability to continue generation1,5128.0%
Positive attribution labels4,35636.0% of 12,099 attribution labels
Negative attribution labels7,74364.0% of 12,099 attribution labels
Distinct failure contexts underlying the intervention records4,310Each context is counted once across the 30,000 intervention records.

Each delayed-failure record is counted once under its first observed failure type. Within a given failure context, a historical decision receives a positive label if at least one completed replacement test is effective and a negative label if none of its completed tests is effective.

Q4. Designer and continuation-policy configurations, functional constraints, and validation thresholds

Designer and continuation policy. The Designer uses Qwen3.8-27B (temperature 0.7, top-p 0.95) to propose K=5K=5 candidate decisions. After collision, boundary, and hard-constraint checks, the world model ranks valid candidates by predicted risk. SceneWise verifies up to B=2B=2 candidates. Each rollout includes at most H=3H=3 decisions, counting the candidate itself. The continuation policy π\pi processes remaining requirements in a fixed order and takes the first valid proposal at each step. Consequence supervision and inference-time verification use the same policy and horizon.

Functional constraints and validation thresholds. Functional constraints specify the minimum space needed for each activity: 0.80 m of passage width, 0.60 m of clearance in front of a cabinet, and 0.55 m of pull-out clearance behind a chair. A negative constraint margin indicates a violation.

Q5. Failure-event F1 scores for the full model in Tables 3 and 16

The two tables differ in what counts as a correct failure prediction. Table 3 reports 87.9% micro-F1 for the full model when matching by failure type alone. Table 16 reports 85.0% micro-F1 for VV under a stricter matching rule that requires agreement on the failure type, the constraint identifier, and the unordered set of participating object identifiers. For example, a prediction that correctly identifies an entrance obstruction but names the wrong blocking object counts as a match only in Table 3.

Both tables use the same held-out consequence records and the same aggregation rule. In both cases, we pool true positives, false positives, and false negatives across all test records to compute micro-F1. We will clarify the matching and aggregation rules in both captions.

Thank you for the detailed rebuttal and additional experiments. The responses address most of my original concerns, and I remain inclined to accept the paper. After reading the other reviewers’ comments and the authors’ responses, I still have concerns about the following points:

  1. Data construction. The SceneSmith-based collection pipeline is clearer, but the asset sources, offline generator configuration, and construction of training records for operations other than movement remain insufficiently specified. Further details would improve reproducibility.

  2. Data quality. The added statistics describe dataset coverage, but quantitative evidence of the quality of the 5,000 source scenes and the reliability of execution-derived labels is still missing. An audit of source-scene validity and label correctness would strengthen confidence in the training data.

  3. Overlap checking. The reported absence of exact and near-duplicate overlap is helpful, but the similarity measures, matching criteria, and thresholds are unclear. Explaining how task semantics, layouts, and asset configurations were compared would make the separation from the evaluation sets easier to assess.

  4. Human evaluation. No user study is reported. A blinded human evaluation of layout quality and usability would complement the automatic metrics and provide additional evidence of practical benefits.

  5. Scene-quality ablations. The controlled ablations report task success, constraint satisfaction, and editing effort, but omit the general scene-quality metrics in Tables 1 and 11. Including these metrics for key ablations, either on SceneEval or the existing coupled-requirement test set, would clarify how the proposed components affect final scene quality.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Author Response for Rebuttal

Dear AC,

We appreciate your time and the care you have taken in overseeing the discussion of our paper.

We introduce SceneWise, a framework for 3D indoor scene generation and repair that anticipates how early placement decisions affect later spatial and functional requirements. For example, a bed may fit in the current layout yet leave too little room to open a wardrobe added later. SceneWise uses a shared world model to evaluate candidate placements during generation and proposed changes during repair, while learned attribution identifies earlier decisions to revisit. We also introduce SceneWiseDB, a dataset of 100,000 decision–consequence and matched-intervention records across 20 room categories.

Reviewers highlighted several strengths of SceneWise, including the motivation for addressing delayed conflicts (N7rv, HiMe), the use of the same consequence model during generation and repair (N7rv, HiMe), the prediction of downstream effects of candidate edits (HiMe, SmVn), historical attribution for failure recovery (HiMe, SmVn), and controlled ablations showing the contributions of individual components (N7rv, HiMe). More specifically, N7rv noted higher success rates with fewer editing operations on coupled tasks, HiMe emphasized the breadth of the evaluation, and SmVn highlighted the model’s structured predictions of placement feasibility, constraint margins, and potential failures.

Across all reviews, we note a common thread—all reviewers recognized the motivation and value of anticipating how current scene placements affect downstream requirements, and their main requests focused on additional experiments and clearer documentation of how SceneWiseDB was constructed and validated.

Addressing Key Concerns. We have made diligent efforts to directly address these requests and made corresponding modifications (highlighted in blue) in the updated manuscript:

  • External baselines and shared-backbone comparisons. HiMe (W1, Q1), SmVn (Q3)

    We compared SceneWise with SceneReVis and a GPT-6 Astra agent on generation (Table 2) and with the same agent on repair (Table 6). With the backbone and execution framework held fixed, SceneWise satisfied all requirements in 84.0% of attempts, compared with 69.7% for SceneSmith, the strongest baseline in this setting (Table 3).

  • Consequence prediction and delayed conflicts. HiMe (W2, Q2), N7rv (Q1, Q4–Q5), SmVn (Q2)

    We added paired tests, confidence intervals, and runtime measurements to the existing comparison with direct risk scoring (Appendix D.5). A new experiment found that consequence guidance reduced conflicts in all three delay groups, with the largest reduction within the three-step lookahead horizon (Table 24). We also clarified candidate verification, functional-clearance thresholds, and the two matching rules used to compute failure-event F1.

  • SceneWiseDB construction, quality, and overlap. SmVn (Q1, Q4), N7rv (Q2–Q3 and follow-up Q1–Q3)

    We documented the asset sources, scene generation and edit sampling procedures, and distributions of delayed failures and attribution labels. All 100 sampled scenes passed the validity and prompt checks, and an independent audit confirmed the labels for 120 sampled decision–consequence records and 120 sampled interventions (Table 9). Under the criteria described in Appendix C.1, we found no exact or near matches with either evaluation set in task requirements, layouts, or asset configurations.

  • Human evaluation. N7rv (follow-up Q4)

    In a blinded study, 102 participants compared layouts from four methods across eight rooms. SceneWise received the most votes for layout quality, perceived usability, prompt faithfulness, and overall preference, including 617 of 816 overall-preference votes (75.6%, Table 4).

  • Scene-quality ablations. N7rv (follow-up Q5)

    We extended the SceneEval evaluation to four component ablations. Removing consequence-guided generation and learned attribution together lowered object-relation satisfaction from 74.0% to 63.4% and accessibility from 88.0% to 76.9% (Tables 1 and 18).

All three reviewers initially rated the paper Weak Accept. After our response, HiMe wrote, “My concerns are satisfactorily addressed, and I maintain my positive evaluation of the paper.” SmVn concluded, “I therefore support acceptance of the paper.” N7rv raised five follow-up questions while noting, “The responses address most of my original concerns, and I remain inclined to accept the paper.” We answered all five and incorporated the experiments and clarifications from both rounds into the revised manuscript to address the concerns of all reviewers.

Thank you again for your time and careful consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.