acceptodds
Under review as a conference paper at ICLR 2027

Unsure What Exists, Not Missing Physics: Intermediate Denoising Estimates Violate Support Constraints in 3D Scene Diffusion

Abstract

Intermediate denoising estimates of 3D scene diffusion models are often read as evidence about learned physical constraints: a lifted object is pulled back to the floor, or a large residual gap is reported as a violation. We show that, in joint semantic–geometric diffusion, the second reading is usually wrong for a simple reason. The clean-geometry estimate is a posterior mean over what each slot might contain, and for an affine constraint such as floor support this mean leaves the constraint plane exactly when the posterior places mass on states that do not satisfy it: an empty slot, whose zero-padded geometry decodes to a floating box, or a non-floor category. We test this account with controlled semantic-token interventions on official test rooms of three room-specific MiDiffusion checkpoints and, with continuous class channels, on DiffuScene. A predictor with no fitted parameter, the network's own auxiliary occupancy posterior times the decoded empty coordinate plus class-conditional training means, reproduces the geometry head's floor gap ( up to 0.96), tracks its growth with noise level, and matches the sign and magnitude of every context intervention. Replaying complete reverse trajectories step by step shows that the estimate settles on the floor before the categorical token commits, that tokens commit exactly once, and that every final failure among 592 known-scene trajectories is a target whose occupancy was unresolved at the intervention step. The consequence is practical: projecting intermediate estimates onto the floor at every step without conditioning on occupancy adds 0.8–1.1 spurious objects per scene and raises the colliding-object fraction by 24–35 points, and occupancy-blind collision guidance on the noisy state likewise creates objects; gating on the occupancy argmax removes most of these objects but does not make collision guidance effective. Paired response scores cannot see any of this, because they cancel the shared offset. Constraint diagnostics and guidance on intermediate estimates must therefore be conditioned on occupancy.

Primary area:
generative models
Keywords:
diffusion models, 3D scene generation, geometric constraints, semantic conditioning, causal interventions, model evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_HYE8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies the task of diffusion-based 3D scene generation. Specifically, the problems of the disobeyed floor support constraint during intermediate diffusion steps. The authors point out that, when a model has not yet determined whether a specific slot contains an actual object, its intermediate geometric prediction may manifest as a weighted average of geometric outcomes corresponding to different semantic states. This intermediate predictions may create deviation from the ground support plane. This paper did a thorough analysis to this effect on MiDiffusion and DiffuScene, The results indicate that a predictor constructed based on class posterior probabilities and class-conditional means—which requires no additional fitted parameters—can quite accurately account for the ground clearance derived from intermediate geometric predictions.

Critical strengths
  1. The paper provide a new aspect to the study of 3D scene generation. It provides a valuable perspective for understanding the intermediate representation of three-dimensional scene diffusion models.
  2. The theoretical analysis is relatively clear and provides a reasonable explanation for the experimental results.
  3. The qualitative experimental design is relatively systematic, incorporating a variety of complementary validation methods.
  4. The discovery: "additional object may be generated due to intermediate constraint on the diffusion process" provides a new discovery of diffusion-based 3D scene generation.
Critical weaknesses
  1. The motivation for this paper is not sufficiently justified. This paper has raised a actual problem on diffusion based 3D scene generation, the ground-support bias in intermediate denoising predictions, and pointed out this bias can be explained mistakenly as the lack of physics understanding. However, the prevalence of this issue across existing methods has not been sufficiently demonstrated (validation was limited to only two methods: MiDiffusion and DiffuScene).
  2. The experiment in Section 6 applies ground constraints to all slots, including empty slots and objects that do not need ground support. This setup seems unrealistic and may itself explain the extra objects and increased collisions.
  3. The practical value for final scene quality remains unclear. In the studied MiDiffusion trajectories, all retained ground-supported objects meet the 5 cm support threshold. If these objects already satisfy the constraint, what practical generation problem does analyzing intermediate floating help solve?
  4. The paper focuses on theoretical analysis and includes no figures explanation in either the main text or supplementary material. For a 3D scene generation task, visual examples would make the paper easier to understand and provide qualitative support for its conclusions.
Questions for authors

Please refer to the weakness part.

4 comments
AuthorsOpenReview ↗

Response to Reviewer HYE8 (1/2): figures and prevalence

Thank you. The revision addresses the four weaknesses directly. Its structure and numbering are those of the submission; the additions are new paragraphs, a redrawn Figure 1, two new figures, and Appendix E.

W4. "For a 3D scene generation task, visual examples would make the paper easier to understand and provide qualitative support for its conclusions."

The paper now has two rendered examples and a new plot.

  • Figure 1 opens the paper: a schematic of the mechanism next to a rendered test scene. The estimate of a masked nightstand slot is a large box 75 cm above the floor (Equation 3 gives 80 cm); it becomes a nightstand on the floor when its token alone is restored.
  • Figure 6 shows a generated bedroom under native sampling and under the all-slot edit, with the created object and the lamp pulled to the floor.
  • Figure 7 shows, over PhyScene's 1,000 sampling steps, how early the model's own posterior knows which slots hold objects and how late its noisy state shows it.

W1. "the prevalence of this issue across existing methods has not been sufficiently demonstrated (validation was limited to only two methods: MiDiffusion and DiffuScene)."

Across methods. Three models and six released checkpoints now carry the quantitative predictor (Table 11): MiDiffusion (R² 0.960 / 0.907 / 0.914), DiffuScene (0.945 / 0.960), and PhyScene (0.890 for the objects it places on the floor, along its own trajectories).

Within a method, the phenomenon is near-universal (Table 14). In 2,015 MiDiffusion scenes generated from pure noise, 84–88% of the objects that end on the floor have an estimate more than 20 cm off it at t = 900, half at t = 500, and 22–25% at t = 300. PhyScene's own trajectories show the same course (54% at t = 500, 23% at t = 300).

AuthorsOpenReview ↗

Response to Reviewer HYE8 (2/2): the all-slot setting and the practical value

W2. "The experiment in Section 6 applies ground constraints to all slots, including empty slots and objects that do not need ground support. This setup seems unrealistic and may itself explain the extra objects and increased collisions."

The all-slot setting is the published algorithm of an existing method. Algorithm 1 of the PhyScene paper guides all objects at every step (its Appendix E applies guidance on the final steps); its released code implements the two gates that Equation 3 calls for, an objectness mask and the last 10 steps. On PhyScene's released checkpoint (Table 21) the released code lowers the collision objective by 5.9 × 10⁻⁴ and moves no object by more than 50 cm; the all-slot form lowers it by 3.8 × 10⁻⁴ and moves 0.33% of the objects that far.

The new factorial separates the ingredients you name (Tables 17–18).

  • Editing empty slots creates 0.72–1.02 objects per scene.
  • Editing objects that need no floor support leaves the object count unchanged and adds 0.9–3.1 colliding pairs.
  • Editing floor objects alone changes nothing.

The extra objects come from slots that were empty or still undecided: of the 651 objects created by the all-slot edit, 411 come from slots whose token had committed to empty and 240 from masked slots, and none from a slot that held a real token (Table 19).

W3. "If these objects already satisfy the constraint, what practical generation problem does analyzing intermediate floating help solve?"

Three, each measured.

  1. Evaluation. Reading intermediate estimates as physics misdiagnoses a model whose samples are supported (14,121 of 14,129 floor objects within 5 cm; under rigid-body simulation they fall no more often than the objects of real scenes, Table 22). Subtracting the predictor corrects the readout and removes 78–86% of the apparent violation. Paired scores stay perfect throughout and cannot detect it.
  2. Guidance. Methods that impose physics during sampling act on exactly these estimates. Acting on a slot whose occupancy is unresolved can create an object, and the model's posterior says which slots are safe and from which step. PhyScene's released guidance already gates by occupancy and time; its noisy-state mask is right for 79% of the slots at t = 300, where the model's own posterior is right for 99.4%.
  3. Few-step sampling and early stopping read intermediate estimates as outputs: at t = 200, 8–9% of the objects that would end on the floor are still estimated more than 20 cm off it.

The finding also redirects effort: the final failures of the tracked targets are occupancy and identity failures, so the remedy lies in the padding representation and in posterior-gated guidance, which the analysis specifies.

We are glad that you found the perspective valuable and the creation of objects by constraints during sampling a new discovery. Each of the four weaknesses is now answered: rendered examples and a new plot (Figures 1, 6 and 7); the predictor on three models (Table 11), and in 2,015 generated scenes 84–88% of the objects that end on the floor estimated more than 20 cm off it at t = 900 (Table 14); the all-slot setting as the published algorithm of an existing method, PhyScene, with a factorial that separates its ingredients (Tables 17–21); and the practical value, measured for evaluation, guidance and few-step sampling. We hope the revision makes both the motivation and the practical value clear, and we would be grateful if you would consider raising your rating. We are glad to answer any remaining question before the discussion period ends.

Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns. The new results have strengthened the paper, especially the cross-model validation and the additional guidance experiments.

However, I still have concerns about the practical significance of this work. The all-slot projection setting seems less realistic, as existing methods like PhyScene already use objectness masks to avoid such issues. Although the additional experiments demonstrate that intermediate floating is common across different models, it remains unclear how often this phenomenon causes actual failures in existing guidance methods, or how the findings can improve existing scene generation methods in practice.

Overall, while the additional experiments have addressed several of my concerns, I remain concerned about the practical significance of this work. Therefore, I would like to keep my original rating.

AuthorsOpenReview ↗

Response to Reviewer HYE8: practical significance, measured

Thank you for the follow-up, and for noting that the cross-model validation and the guidance experiments strengthened the paper. We answer the remaining question, practical significance, with measurements.

"existing methods like PhyScene already use objectness masks to avoid such issues"

We agree that PhyScene's released code avoids the issue, and its mask is the clearest sign that the issue arises in practice. Every empty slot of PhyScene decodes to the same 2.4 × 1.8 × 1.4 m box, centred 1.70 m above the floor. Without the mask, the empty slots of a scene (8.75 on average) enter the collision objective as copies of this box: the gradient reaches its clip in 98% of the scenes at the final step, and the reduction of the collision objective falls by 39% (5.9 → 3.6 × 10⁻⁴; Table 21). The PhyScene paper does not describe the mask; its Algorithm 1 guides all N slots at every step. Our analysis explains why the mask is needed (the empty term of Equation 3) and when it is valid: the mask reads the posterior mean, which is close to the noisy state, and agrees with the final occupancy for 79% of the slots at t = 300 and for at least 99.9% only from step 91 on. The released code guides only the last 10 of 1,000 steps, which lie inside that range.

"it remains unclear how often this phenomenon causes actual failures in existing guidance methods"

As you note, intermediate floating is common across models. It becomes a failure whenever guidance acts without the occupancy gate, and then failure is the rule rather than the exception:

  • PhyScene in the form of its Algorithm 1 (all slots, every step): the collision objective falls by 3.8 instead of 5.9 × 10⁻⁴, and 0.33% of objects displaced by more than 50 cm instead of none (Table 21).
  • Every-step floor projection in MiDiffusion: 0.8–1.1 spurious objects per scene, and 23.7–40.8 points more objects that move in rigid-body simulation (Tables 3 and 22).

"how the findings can improve existing scene generation methods in practice"

  1. A rule with a measured value on an existing method. Gating guidance by occupancy and acting where occupancy is resolved gives PhyScene's Algorithm 1 a reduction of the collision objective 1.6 times as large and removes every displacement beyond 50 cm.
  2. A gate that holds at every step. The model's own posterior identifies the final occupancy for 96% of the slots at t = 900 and 99.4% at t = 300, where the mask is at 50% and 79% (Figure 7). Guidance that acts while the layout is still forming can use it: in MiDiffusion, the posterior gates keep the object count within 0.03 and the colliding pairs within 0.06 of native sampling, and change the share of objects that move by −0.4 to +1.1 points (Tables 17, 18 and 22).
  3. Evaluation. Subtracting the predictor removes 78–86% of the apparent violation that a diagnostic reading intermediate estimates would report; paired scores cannot detect it.

We hope these measurements answer the question of practical significance, and we would be grateful if you would take them into account in your final rating.

Reviewer_iu9bOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper examines how intermediate clean-geometry estimates should be interpreted in joint semantic–geometric scene diffusion. The authors show that, for an affine floor-support constraint, averaging supported states preserves the constraint, whereas including empty slots or non-floor categories can shift the mean away from the support plane.

The paper investigates this effect through semantic interventions on MiDiffusion, a predictor based on auxiliary class probabilities and training-set gap statistics, analyses of full reverse trajectories, and physical edits during sampling. The predictor shows strong agreement with geometry outputs. The experiments also distinguish occupancy and category errors from floor-support errors in retained objects, and find that occupancy-blind edits can lead to unintended object creation. DiffuScene is used for a more limited cross-architecture evaluation. Separately, the authors show analytically that the paired-response score can be perfect even when the absolute support plane is incorrect.

The main contribution is an explanation of these effects and supporting empirical evidence, rather than a new generation method.

Critical strengths

1.The paper provides a quantitative explanation for an important representation issue.** The semantic-head predictor is not fitted to geometry-head predictions, yet achieves R² values of 0.960 and 0.907 under native corruption. The results remain strong on the originally masked subsets (0.911 and 0.818; Table 1). The target and context interventions, including those involving padding tokens, provide further evidence for the proposed explanation. I also appreciate the analytical counterexamples showing that the paired-response score cannot guarantee correctness of the absolute support plane (Section 2; Table 10). Together, these results go beyond the observation that intermediate diffusion predictions are uncertain.

  1. The paper connects the analysis to what actually happens during sampling.** The trajectory experiments distinguish object deletion and category changes from support errors in retained objects. The editing experiments also reveal a practical problem: applying occupancy-blind floor projection at every step adds approximately 0.8–1.1 objects per scene (Tables 2–3). The authors acknowledge that gating does not consistently improve collisions, which helps clarify the limitations of the proposed interventions. These experiments make the analysis relevant to both evaluation and editing of joint scene generators.
Critical weaknesses
  1. I am not fully convinced of the significance of the contribution relative to existing work.** PIDM already discusses the distinction between mean and sample estimates and how the former can violate constraints (Section 3.3). PIDDM further investigates this issue in physical generation (2025 preprint). I recognize that the explanation in this paper is more specific: it concerns affine constraints and uncertainty over occupancy and applicability, rather than the general nonlinear Jensen-gap argument. The quantitative evidence is also a useful addition. Still, the broader concern about interpreting intermediate mean estimates is not new.

    There is a similar issue with the practical recommendation. PhyScene already constructs an objectness mask in its public guidance implementation and uses it for collision loss computation (sampling code, collision loss). The paper's quantitative explanation may be new, but I would like to see more clearly what it changes in practice. For example, does it reveal a failure in an existing diagnostic, or a problem with a guidance method that already handles occupancy? I am not yet convinced that the new quantitative findings establish a sufficiently broad diagnostic insight beyond the specific representation and constraint studied here. A concrete connection to an existing evaluation or guidance procedure, or an equally strong quantitative validation in another representation, would make the significance clearer.

  2. The floor-projection experiment does not clearly separate occupancy effects from constraint applicability.** In the blind setting, every slot is projected, including occupied objects that should not be floor-supported. In contrast, the gated setting only projects predicted real objects belonging to floor-applicable categories (Section 6; Appendix A.7). Thus, the reported 24–35 percentage-point increase in collisions cannot be attributed solely to empty or unresolved slots. Some of the increase may come from moving existing non-floor objects, such as lamps, onto the floor.

    The increase in object count and the real-class-gated separation experiments still provide useful evidence of occupancy-related side effects. However, an additional comparison would help establish how much of the collision increase is actually caused by occupancy uncertainty, rather than by applying the wrong constraint to existing objects.

Questions for authors
  1. Novelty and practical significance:** Could the authors identify an existing physical diagnostic or guidance method whose behavior or interpretation would change based on the proposed explanation? In particular, how does that method handle empty slots and category applicability, and what does the present analysis add beyond existing occupancy masks? I would find a concrete comparison or an explanation based on an existing implementation more helpful than a broader benchmark.

  2. Attribution in the floor-projection experiment:** Could the authors compare an occupancy-only gate with a gate that accounts for both occupancy and floor applicability? It would also help to separate the added objects and collisions associated with initially empty or unresolved slots from those involving already occupied non-floor slots. This would clarify how much of the reported collision increase directly supports the occupancy-based explanation.

3 comments
AuthorsOpenReview ↗

Response to Reviewer iu9b (1/2): relation to PIDM / PIDDM; what changes for PhyScene

Thank you for the two precise points; both led to new experiments. Section, equation and table numbers and Figures 2–5 of the submission are unchanged; Figure 1 is redrawn; new tables and figures are in Appendix E.

W1. "PIDM already discusses the distinction between mean and sample estimates and how the former can violate constraints […] the broader concern about interpreting intermediate mean estimates is not new."

PIDM (Sec. 3.3) and PIDDM analyse the Jensen gap: for a nonlinear residual R, R(E[x₀ | x_t]) ≠ E[R(x₀) | x_t]. For an affine constraint this gap is identically zero, so that account predicts no violation of floor support at all. The 39–51 cm we measure under a masked token have a different source: the posterior averages over states the constraint does not apply to, an empty slot or a lamp. Three consequences follow, none available from the Jensen-gap account:

  • The violation is quantifiable case by case. The parameter-free predictor reproduces it: R² 0.91–0.96 on MiDiffusion, 0.945 / 0.960 on DiffuScene, 0.890 on PhyScene.
  • It is removed by resolving one discrete variable at unchanged noise (17.35 → 2.25 cm in bedrooms when a token is restored). For an affine constraint the Jensen gap is zero at every noise level, so it cannot account for this change.
  • It says where and when guidance may act.

Section 2 now states this and cites both papers. The source is also absent from the fixed-grid fields those papers study, where every element exists.

W1 / Q1. "PhyScene already constructs an objectness mask in its public guidance implementation […] how does that method handle empty slots and category applicability, and what does the present analysis add beyond existing occupancy masks?"

We ran PhyScene's released guidance on its released checkpoint (Section 6, paragraph "An existing implementation"; Tables 20–21, Figure 7).

How it handles empty slots. The mask you point to has a companion on the next line of the released sampler: guidance only for t < 10 of 1,000 steps. Algorithm 1 of the PhyScene paper guides all objects at every step (its Appendix E applies guidance on the final steps). Equation 3 explains both gates, and our runs measure when each is needed.

  • Every empty slot decodes to the same box, 2.4 × 1.8 × 1.4 m, centred 1.70 m above the floor. Without the mask, the 8.75 empty slots per scene enter the collision objective as overlapping copies of it; the largest gradient component per scene grows 33-fold (average over the guided steps), and the reduction of the objective falls from 5.9 to 3.6 × 10⁻⁴.
  • The mask reads the sign of the empty channel of the posterior mean, which at intermediate steps is close to the noisy state. It agrees with the final occupancy for 79% of the slots at t = 300 and for 99.9% only from step 91 on. The t < 10 window lies inside that range.

What the analysis adds beyond the mask.

  1. The gate for guidance that acts while occupancy is still being decided, as the all-object, every-step form and our every-step edits do: the model's own posterior. The clean estimate's empty channel agrees with the final occupancy for 96% of the slots at t = 900 and 99.4% at t = 300, where the noisy-state mask agrees for 50% and 79% (Figure 7). The floor-class gates built from the posterior (argmax, thresholds of 0.5 and 0.9, soft weight) remove both failure channels in our factorial (Tables 17–18).
  2. A correction for diagnostics that read intermediate estimates: subtracting the predictor removes 78–86% of the apparent violation. Paired scores stay perfect throughout and cannot provide it.
  3. An explanation of why the mask is needed at all, in a form that transfers to other objectives: the padding box is the empty term of Equation 3.
AuthorsOpenReview ↗

Response to Reviewer iu9b (2/2): occupancy versus applicability; validation in another representation

W2 / Q2. "Could the authors compare an occupancy-only gate with a gate that accounts for both occupancy and floor applicability? It would also help to separate the added objects and collisions associated with initially empty or unresolved slots from those involving already occupied non-floor slots."

Table 17 runs this factorial (Section 6, paragraph "Occupancy versus applicability"). Known-scene cohorts, bedroom / living room, paired difference from native sampling:

slots editedobjectscolliding pairs
floor-class slots only (existing gate)+0.00 / +0.01+0.01 / +0.02
all predicted objects, lamps included (occupancy-only gate)+0.01 / +0.02+1.29 / +2.91
predicted-empty slots only+0.83 / +0.72+2.67 / +2.68
all slots+0.83 / +0.80+4.06 / +6.40
  • Applicability: moving predicted non-floor objects to the floor adds 1.3–3.0 colliding pairs at an unchanged object count.
  • Occupancy: editing predicted-empty slots creates 0.7–0.8 objects and adds 2.7 pairs.
  • Slot by slot (Table 19): the 651 created objects come from slots whose token was empty (411) or masked (240), never from a slot holding an object. In the guided samples 2,364 colliding pairs involve a created object and 1,655 a moved lamp.

Occupancy and applicability are the two terms of Equation 3, and each is one failure channel of guidance.

W1. "[…] or an equally strong quantitative validation in another representation, would make the significance clearer."

The quantitative validation now holds in the continuous representation, on two models (Table 11; Section 7): DiffuScene, R² 0.945 / 0.960 from its class channels, and PhyScene along its own unguided trajectories, 0.890 for the objects it places on the floor. In both, occupancy-only predictors fail (R² at most 0.10), because the non-floor term carries the gap.

Your review asked for "a concrete connection to an existing evaluation or guidance procedure, or an equally strong quantitative validation in another representation." The revision provides both. PhyScene's released guidance, run on its released checkpoint, applies the two gates that Equation 3 calls for, and the model's own posterior agrees with the final occupancy for 99.4% of the slots at t = 300, where the released mask agrees for 79% (Tables 20–21, Figure 7). The predictor reaches R² 0.945 / 0.960 on DiffuScene and 0.890 on PhyScene (Table 11). With the factorial that separates occupancy from applicability (Tables 17–19) and the distinction from the Jensen-gap account (Section 2), we hope this establishes the significance you asked about, and we would be grateful if you would consider raising your rating. We are glad to answer any remaining question before the discussion period ends.

AuthorsOpenReview ↗

Response to Reviewer iu9b: A short summary

Thank you again for your review and its two precise points. Before the discussion period closes, we summarize how the revision meets the request in your review: "A concrete connection to an existing evaluation or guidance procedure, or an equally strong quantitative validation in another representation, would make the significance clearer." The revision provides both.

  • An existing guidance procedure. PhyScene's released sampler applies two gates: the objectness mask you pointed to, and guidance only in the last 10 of 1,000 steps. Equation 3 explains both. In the form of the PhyScene paper's Algorithm 1, which guides all slots at every step, the collision objective falls by 3.8 instead of 5.9 × 10⁻⁴, and 0.33% of objects are displaced by more than 50 cm instead of none (Table 21).
  • What the analysis adds beyond the mask. The mask agrees with the final occupancy for 79% of the slots at t = 300; the model's own posterior agrees for 99.4% (Figure 7). In the factorial you asked for, gates built from that posterior remove both failure channels, occupancy and applicability (Tables 17–18).
  • Another representation. With continuous class channels, the predictor reaches R² 0.945 / 0.960 on DiffuScene and 0.890 on PhyScene (Table 11).

If these results resolve your concerns, we would be grateful if you would consider raising your rating.

Reviewer_knqYOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper focues on an interesting topic about intermediate estimates of 3D indoor scene diffusion.

The authors focuses on a failure mode between object occupancy and missing physical constraints.

Semantic interventions on MiDiffusion and DiffuScene are conducted.

Experiments on two standard scenes and pretrained checkpoints yield some valuable conclusions.

Critical strengths
  • Interpretation of intermediate denoising estimates seems to be solid and makes sense.

  • I think the controlled intervention in this paper is convincing.

  • Experiments are solid and convincing.

  • Overall the paper is easy to follow.

Critical weaknesses
  • Some cited references are discussed, but novelties over some prior work on mixed-diffusion representations need clarification. For instance,

[1] Mixed diffusion for 3d indoor scene synthesis. WACV, 2026.

[2] Structured denoising diffusion models in discrete state-spaces. Neurips 2021.

  • The two network heads share representations, but their quantitative agreement does not independently establish an occupancy-weighted computation. Posterior calibration and state-dependent conditional means are not validated.

  • Missing comparison with some occupancy-only, prior-based, and posterior-randomization baselines.

  • The analysis is on MiDiffusion and DiffuScene. Whether the conclusion can be scaled to newer alternative representations remains unjustified.

  • All experiments use 3D-FRON. More diverse benchmarks such as InternScenes and M3DLayout are missing.

  • Physics-guidance baselines are missing. As a result, the observed failures may be specific to the chosen interventions.

  • Insufficient evaluation of occupancy-gating alternatives. Its effectiveness over simpler alternatives remain unclear.

  • Some broader simulation-based evaluations, for instance, physical contact or realistic interactions, are missing.

  • Pratical application or other downstream scene-generation tasks are limited and remains unclear.

  • trajectory conclusions seem to be overgeneralized.

Questions for authors
  • What theoretical or mechanistic contribution does it claim over conditional-mean denoising and discrete/continuous diffusion models?

  • Can the authors provide independent mechanistic evidence and compare against occupancy-only, class-prior, timestep-conditioned, and posterior-randomization baselines?

  • Would the reported violations persist if empty slots were encoded differently? Can the authors separate occupancy uncertainty from artifacts caused by zero-padding?

  • Can the authors demonstrate the phenomenon in a substantially different generative representation, ideally from recent alternative scene-generation architectures?

  • Can the authors reproduce the main findings on InternScenes, M3DLayout, or another independent dataset with different scene statistics and object distributions?

  • How would the findings change when using a physics-guidance method? Does occupancy-aware guidance improve its physical validity under matched conditions?

  • What advantage does the argmax gate provide over oracle occupancy, fixed confidence thresholds, or alternative simple guidance?

  • Can the authors evaluate physical stability under simulation?

  • Can the authors demonstrate improved scene completion or interaction performance while preserving scene quality, object recall, and diversity?

5 comments
AuthorsOpenReview ↗

Response to Reviewer knqY (1/4): contribution; independent mechanistic evidence

Thank you. We answer every weakness and question in order, in four comments. Section and table numbers and Figures 2–5 of the submission are unchanged; Figure 1 is redrawn; new tables and figures are in Appendix E.

W1 / Q1. "What theoretical or mechanistic contribution does it claim over conditional-mean denoising and discrete/continuous diffusion models?"

MiDiffusion (your [1]) and D3PM (your [2]) define the generative processes. That a denoiser estimates a conditional mean is textbook. This paper adds three things that none of these contain:

  1. An identity that locates the violation. For an affine constraint, a posterior mean leaves the constraint plane only through discrete states that are off it: an empty slot or a class the constraint does not apply to. The Jensen-gap account of constraint violation predicts exactly zero here.
  2. A parameter-free predictor of each slot's violation from the model's own class posterior, validated case by case on three models: R² 0.960 / 0.907 / 0.914 on MiDiffusion, 0.945 / 0.960 on DiffuScene, 0.890 on PhyScene.
  3. Consequences for the two uses of intermediate estimates. Paired diagnostics stay perfect while the absolute readout is 2–94 cm off. Occupancy-blind guidance creates objects: 0.8–1.1 per scene.

W2 / Q2. "The two network heads share representations, but their quantitative agreement does not independently establish an occupancy-weighted computation. Posterior calibration and state-dependent conditional means are not validated." "Can the authors provide independent mechanistic evidence […]?"

Four independent lines of evidence, none of which relies on the agreement of the two heads alone.

Interventions. Restoring one token at unchanged noise removes the violation: 17.35 → 2.25 cm in bedrooms. Forcing the empty token gives the decoded padding coordinate of each encoding (93.5, 80.5, 81.4 cm). Masking one padding slot in the context raises the target's empty posterior by 0.05–0.06 and its gap accordingly; masking a real object does not.

State-dependent conditional means, measured (Table 12). We set the target token to every class in turn at the same noisy geometry. The posterior-weighted sum of these class-conditional estimates, which uses no training statistic, predicts the masked estimate with R² 0.905 / 0.800 / 0.842; the fixed-mean predictor reaches 0.897 / 0.806 / 0.825. State dependence of the class means costs 3.5–4.1 cm RMS.

Calibration. The expected calibration error of the empty posterior on masked context slots against reference tokens is 0.10–0.12 at t = 300 and 0.04–0.09 at t = 100. Along full trajectories the posterior predicts the outcome: targets that end deleted have an empty posterior of 0.56 at t = 300, against 0.19–0.42 for retained ones.

Sampling. The gate factorial (Tables 17–19) intervenes on the sampler itself: editing only the slots whose posterior argmax is empty creates 0.72–1.02 objects per scene, and none of the 651 objects created by the all-slot edit comes from a slot that held a real token.

AuthorsOpenReview ↗

Response to Reviewer knqY (2/4): baselines, zero padding, other representations

W3 / Q2. "[…] compare against occupancy-only, class-prior, timestep-conditioned, and posterior-randomization baselines?"

All four are in Table 11. R² at t = 300, native inputs:

predictor (fitted parameters)MiDiffusion bed / living / diningDiffuScene bed / living
predictor of Eq. 3 (0)0.960 / 0.907 / 0.9140.945 / 0.960
occupancy only: p_∅ × padding gap (0)0.67 / 0.61 / 0.64−0.08 / −0.19
non-floor term only (0)0.29 / 0.34 / 0.310.83 / 0.87
least-squares fit a + b·p_∅ (2)0.70 / 0.64 / 0.670.10 / 0.06
class prior instead of the posterior (0)−3.27 / −2.57 / −3.12−2.08 / −1.25
timestep-conditioned constant (1)00
posterior shuffled across cases (0)−1.00 / −1.08 / −1.07−0.75 / −0.81

Each term of Equation 3 alone carries one representation: the empty term the discrete model, the non-floor term the continuous one (0.85 on PhyScene). Only the full case-level posterior reproduces the geometry head in every model.

Q3. "Would the reported violations persist if empty slots were encoded differently? Can the authors separate occupancy uncertainty from artifacts caused by zero-padding?"

Yes to both; Equation 3 is the separation. Its first term is occupancy uncertainty (p_∅) times a padding constant (the decoded padding gap); its second term does not involve padding at all.

  • Padding sets the size of the empty term only: 64–73% of the mean gap of masked MiDiffusion targets at t = 300. The same zero padding decodes to 94.0 cm in the bedroom encoding and 80.3 cm in the living-room encoding, and the forced-empty readouts follow it (93.5 and 80.5 cm).
  • The violation persists without it. With the padding point placed on the floor plane and the posterior unchanged, Equation 3 still gives 12–14 cm, from the non-floor classes. In DiffuScene this term carries the gap (16.6 of 19.8 cm in bedrooms), and in PhyScene 12.1 of 13.0 cm for the objects that end on the floor.
  • Uncertainty is the cause; padding is a scale. At fixed padding, resolving the slot's state removes the violation (17.35 → 2.25 cm in bedrooms).

W4 / Q4. "Can the authors demonstrate the phenomenon in a substantially different generative representation, ideally from recent alternative scene-generation architectures?"

The predictor holds on three models that cover both slot representations in use: discrete tokens with an absorbing mask state (MiDiffusion) and continuous class channels diffused with the geometry (DiffuScene; PhyScene, with floor-plan conditioning and physics guidance, R² 0.890 along its own unguided trajectories). The identity itself needs no architecture-specific quantity: it holds for the posterior mean over the states of a padded slot. Whether a trained network's estimate behaves as that mean is measured, not assumed: the predictor test (Table 11) and the semantic interventions (Tables 4, 9 and 13) measure it, and on all three models it does. Autoregressive generators emit one object at a time and have no intermediate estimate to read or guide.

AuthorsOpenReview ↗

Response to Reviewer knqY (3/4): other datasets, physics-guidance baseline, gating alternatives

W5 / Q5. "Can the authors reproduce the main findings on InternScenes, M3DLayout, or another independent dataset with different scene statistics and object distributions?"

We ran the released M3DLayout diffusion model (trained on 3D-FRONT, Matterport3D and Infinigen; 120 slots, 101 classes) on 600 of its test prompts. The phenomenon reproduces and is stronger than on MiDiffusion: at t = 300, 97% of the objects that end on the floor are estimated more than 20 cm off it (MiDiffusion: 22–25%; Table 14). The quantitative account does not carry over unchanged: its class channels do not form a calibrated distribution, the predictor tracks the gap only in correlation (r = 0.76), and asserting a slot's final class at t = 300 does not lower its estimate. Whether a model's geometry follows its slot semantics is thus a property of the model, which these two tests measure (Tables 4, 9 and 11); on the three models of the paper it does. For InternScenes we found no released layout-generation checkpoint, so it would first require training a model.

W6 / Q6. "How would the findings change when using a physics-guidance method? Does occupancy-aware guidance improve its physical validity under matched conditions?"

We ran PhyScene's released collision guidance on its released checkpoint and code (Section 6, paragraph "An existing implementation"; Tables 20–21, Figure 7): 192 floor plans, two draws each, all arms with shared noise.

guidance armcollision objective vs. unguidedobjects moved > 50 cm
released code: objectness mask, last 10 of 1,000 steps−5.9 × 10⁻⁴0%
PhyScene paper, Algorithm 1: all slots, every step−3.8 × 10⁻⁴0.33%
last 10 steps without the mask−3.6 × 10⁻⁴0%

The released sampler is the occupancy-aware one: it applies the two gates that Equation 3 calls for. Under matched conditions it lowers the collision objective by 5.9 × 10⁻⁴, against 3.6–3.8 × 10⁻⁴ for the two forms without the mask. Without the mask, the 8.75 empty slots of a scene enter the objective as copies of one padding box. The findings carry over to this method: the predictor holds along its unguided trajectories (R² 0.890), and its noisy-state mask agrees with the final occupancy for 79% of the slots at t = 300, where the model's own posterior agrees for 99.4% (Figure 7).

W7 / Q7. "What advantage does the argmax gate provide over oracle occupancy, fixed confidence thresholds, or alternative simple guidance?"

We ran all of them (Tables 17–18): oracle occupancy (the slots that end as floor objects), fixed thresholds of 0.5 and 0.9 on the floor-class probability, and a soft weight equal to that probability. Each behaves like the argmax gate: object count within 0.03 and colliding pairs within 0.06 of native sampling, in all four cohorts. The argmax gate therefore matches the oracle. Each of the floor-class gates built from the posterior (argmax, thresholds of 0.5 and 0.9, soft weight) suffices, and the oracle adds nothing over them; the all-slot alternative adds 0.80–1.09 objects per scene. The same holds in rigid-body simulation of the final scenes: these gates change the share of objects that move by −0.4 to +1.1 points, the all-slot edit by +23.7 to +40.8 (Table 22).

Reviewer_gFXtOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether apparent floor-support violations in intermediate denoising estimates indicate deficient physical knowledge in joint semantic–geometric scene diffusion models. Its central observation is that floor support is an affine constraint: averaging supported geometries preserves support. However, averaging these geometries with zero-padded empty slots or non-floor-supported categories can produce a nonzero floor gap. The authors evaluate this explanation using semantic interventions on three room-specific MiDiffusion checkpoints and two DiffuScene checkpoints. In MiDiffusion, a parameter-free predictor combining auxiliary class probabilities with empty-slot geometry and training-set class means closely approximates the geometry head. Reverse-trajectory analyses distinguish semantic retention from final support, while guidance experiments show that indiscriminately projecting intermediate estimates onto the floor can create additional objects and increase collisions. The paper also demonstrates that paired response rankings can conceal large absolute errors. The contribution is primarily an empirical diagnosis of intermediate representations and their evaluation, supported by an elementary affine identity. The MiDiffusion evidence is substantial, but several broader theoretical and practical conclusions require tighter qualification.

Critical strengths
  1. A useful distinction between intermediate estimates and realized objects. The paper identifies a concrete reason why a decoded intermediate box need not correspond to an object that the model intends to generate. Distinguishing empty-slot coordinates, legitimate non-floor categories, and actual support failures improves the interpretation of scene diffusion models.

  2. Strong quantitative evidence beyond qualitative examples. The parameter-free semantic-head predictor is a particularly compelling result. Table 1 reports (R^2=0.960) and (0.907) under native corruption, and agreement remains substantial within the natively masked subsets ((0.911) and (0.818)). This makes the finding more convincing than an aggregate correlation driven exclusively by already-known categories. The context interventions provide additional tests of the proposed explanation.

  3. Careful experimental controls and statistical units. Shared randomness, explicit no-op checks, prediction-independent target selection, scene-equal effects, and source-cluster bootstrap intervals strengthen the comparisons. The authors correctly avoid treating repeated targets, noise draws, and lift magnitudes as independent scenes. They also disclose incomplete source mapping and distinguish exploratory analyses from later confirmation.

  4. A clear demonstration of a diagnostic blind spot. Equation 4 and Table 10 establish that shared output offsets cancel in the paired response score. Wrong-plane projection and constant-output counterexamples show precisely why perfect ranking cannot establish correct absolute support. This conclusion is mathematically transparent and does not depend on interpreting network internals.

  5. Informative negative results and appropriate local caveats. The paper reports that occupancy gating does not reliably improve collision outcomes, that incorrect floor categories are not fully equivalent to correct categories, and that several discrete-model effects do not transfer to DiffuScene. These qualifications increase confidence in the reported empirical observations.

Critical weaknesses
  1. Theoretical approximatio.I am not entirely convinced that fixed training-set class means adequately approximate the input-dependent conditional expectations in Equation 3. The influence of scene context and geometry remains unclear.

  2. Floor-guidance experiment.The comparison in Section 6 changes both occupancy selection and category applicability. It is unclear whether the improvement comes from occupancy awareness or simply from excluding non-floor object categories.

  3. Cross-architecture generalization.The DiffuScene experiments demonstrate semantic sensitivity, but I am unsure whether they support the same underlying mechanism, since class information and shape-latent information are not independently controlled.

Questions for authors
  1. Could the authors explain why fixed training-set class means are a reasonable approximation of the input-dependent conditional expectations in Equation 3? How should the remaining prediction errors be interpreted?

  2. Could the authors provide an occupancy-only gating experiment to distinguish whether the improvement comes from occupancy prediction or from selecting floor-applicable object categories?

4 comments
AuthorsOpenReview ↗

Response to Reviewer gFXt (1/3): exact identity, fixed-mean approximation, network error

Thank you for the careful reading. Each of your six questions is now answered, with a new measurement wherever one applies. Section, equation and table numbers and Figures 2–5 of the submission are unchanged; Figure 1 is redrawn; new tables and figures are in Appendix E.

W1 / Q1. "Do you intend Equation 3 to use input-dependent conditional class means or fixed marginal training means? Can you explicitly separate these quantities and quantify the remaining prediction residual?"

The three quantities are now separated in the text and by measurement.

  • Exact identity. Section 2 states E[g | x_t, c_t] = p_∅ μ_∅ + Σ_c p_c E[g | x_t, c_t, c], with state-dependent conditional means.
  • Approximation. Equation 3 is written as the predictor ĝ(x_t, c_t) that replaces those conditional means by fixed training class means.
  • Measurement. We measured the conditional means directly: for every case the target token is set to each real class and to the empty token in turn, at the same noisy geometry (Table 12; Section 4, paragraph "Comparison predictors, conditional means, and the residual").
MiDiffusion, t = 300, masked tokenbedroomlivingdining
R² of the state-dependent sum (network's own class-conditional estimates, no training statistic)0.9050.8000.842
R² of the fixed-mean predictor (Eq. 3)0.8970.8060.825
residual of Eq. 3, RMS (cm)10.212.511.5
component (RMS): fixing the class means3.94.13.5
component (RMS): network (masked estimate vs. its own weighted sum)9.712.710.9

The class-conditional estimates sit at the fixed means: within 2.6–4.5 cm RMS of the floor for the common floor classes, 93.5 ± 1.2 / 80.5 ± 1.7 cm for the empty state in bedrooms / living rooms (padding points 94.0 / 80.3 cm), and, in bedrooms, 227 ± 22 / 170 ± 27 cm for ceiling / pendant lamps (class means 231 / 179 cm). Your example of a lamp whose height depends on the room is visible in that spread; fixing all class means together costs 3.5–4.1 cm RMS.

"A residual analysis by category, timestep, and occupancy confidence would help bound the explanation."

  • By occupancy confidence: 5.6–8.8 cm where the posterior is concentrated on one state, 11–15 cm where it is spread.
  • By timestep: R² 0.92 / 0.92 / 0.92 / 0.91 at t = 100 / 150 / 200 / 300 on natively masked bedroom targets, 0.76–0.82 in living rooms.
  • By target class: RMSE 5.3–14.6 cm (bedroom), 3.5–18.2 cm (dining), 6.1–18.3 cm (living, one class of 96 cases at 30.9 cm).

Subtracting the predictor from a masked readout removes 78–86% of its distance to the restored-token readout.

AuthorsOpenReview ↗

Response to Reviewer gFXt (2/3): occupancy versus applicability; DiffuScene

W2 / Q2. "What happens with an occupancy-only gate that projects all predicted real categories, including non-floor categories? How does this compare with the existing floor-applicable gate, and which slots account for the additional objects?"

Table 17 is this factorial (Section 6, paragraph "Occupancy versus applicability"). At every step each slot is E (posterior argmax empty), N (a non-floor class), or F (a floor class). Paired differences from native sampling, known scenes, bedroom / living room:

slots editedobjectscolliding pairs
F (existing gate)+0.00 / +0.01+0.01 / +0.02
F + N (occupancy-only gate)+0.01 / +0.02+1.29 / +2.91
N only+0.00 / −0.01+1.30 / +2.96
E only+0.83 / +0.72+2.67 / +2.68
E + N + F (all slots)+0.83 / +0.80+4.06 / +6.40

The occupancy-only gate leaves the object count unchanged and adds the collisions of the lamps it moves to the floor (1.29 / 2.91 pairs, against 1.30 / 2.96 when only the non-floor objects are edited). Only gates that edit E change the object count.

Which slots become objects (Table 19). Of 651 created objects, 411 come from slots whose token had already committed to empty, 240 from masked slots, and none from a slot holding a real token; 611 had an empty posterior argmax at the first edited step. Your reading of the original comparison is now quantified: the applicability channel carries 18–46% of the added colliding pairs in the four cohorts (Tables 17–18) and leaves the object count unchanged. The two terms of Equation 3 are the two channels.

W3 / Q3. "In DiffuScene, what are the separate effects of removing class signal and removing shape-latent signal? Can an occupancy-based predictor quantitatively explain its gaps, or should the claim be limited to qualitatively consistent semantic sensitivity?"

The predictor explains DiffuScene's gaps quantitatively (Table 11; Section 7, paragraph "The predictor transfers quantitatively"). With the posterior read from the class channels of its clean estimate, Equation 3 reaches R² 0.945 / 0.960 at t = 300 and 0.837 / 0.933 at t = 150 (bedroom / living room), with no fitted parameter. In this representation the non-floor term carries the gap (16.6 of 19.8 cm in DiffuScene bedrooms), so occupancy alone is uninformative there.

Class signal versus shape latent (Table 13). Removing the class signal alone raises the gap by 12.1 cm [10.8, 13.4] and 7.3 cm [6.9, 7.8]; removing the shape latent alone raises it by 0.6 and 0.1 cm.

The same test passes on a third model, PhyScene, along its own sampling trajectories (R² 0.890 for the slots that end as floor objects). The title of Section 7 stands as submitted.

AuthorsOpenReview ↗

Response to Reviewer gFXt (3/3): all floor objects, occupancy timing, external relevance

W4 / Q4. "Across all retained floor-applicable objects in the existing native-generation cohort, what are the gap distribution and failure rates at 1, 2, and 5 cm? Does the selected-target result generalize to that population?"

It generalizes. In the existing native-generation cohort, 264 bedroom floor objects have 6 / 4 / 2 beyond 1 / 2 / 5 cm, and 626 living-room objects 9 / 6 / 0. We also generated 2,015 scenes from pure noise: of 14,129 floor objects, 111 / 40 / 8 lie beyond 1 / 2 / 5 cm (0.79% / 0.28% / 0.06%), the median gap is 0.12–0.14 cm, and the largest is 9.1 cm (Table 16 lists every cohort). The abstract and Sections 5 and 9 state the population and the criterion.

Q5. "How is occupancy resolution defined separately from token commitment? Can its temporal ordering be measured directly rather than inferred from the geometry gap?"

It is now measured directly (Section 5, paragraph "From pure noise"; Table 15). Three events are defined separately: occupancy is resolved from the step after which the empty posterior stays at or below 0.1; geometry is settled from the step after which the estimate stays within 5 cm of the floor; the token commits when it leaves the mask. Over whole trajectories from pure noise (4,445 and 9,676 objects):

  • occupancy resolves at a median step of 501 [487, 514] / 479 [469, 490];
  • geometry settles at 293 [287, 299] / 284 [280, 288];
  • occupancy is resolved no later than geometry for 98.4% / 96.2% of objects;
  • for 22% / 29% of objects the posterior resolves before the token commits, from geometry alone.

Equation 3 predicts this order: the estimate can stay within 5 cm of the floor only once p_∅ times the padding gap is below 5 cm.

W5 / Q6. "Which published diagnostic uses the specific paired score in Equation 4, and which existing guidance implementation is directly implicated by your experiments?"

Diagnostic. Preference between matched inputs under shared conditions is the scoring rule of IntPhys (relative error rate, over matched sets of possible and impossible videos) and of LikePhys (ICLR 2026; plausibility preference error, over valid–invalid pairs). Any score component shared by the matched inputs cancels there exactly as in Equation 4, and the occupancy term is such a component (Section 2).

Guidance. We now evaluate PhyScene's own procedure: its released collision guidance on its released checkpoint and code (Section 6, paragraph "An existing implementation"; Tables 20–21, Figure 7). The released sampler acts only on slots whose empty channel in the posterior mean (close to the noisy state) is negative, and only in the last 10 of 1,000 steps, the two gates that Equation 3 calls for. Without the mask, the 8.75 empty slots per scene enter the objective as copies of one padding box and the reduction of the objective falls from 5.9 to 3.6 × 10⁻⁴. The mask agrees with the final occupancy for 79% of the slots at t = 300, while the model's own posterior already agrees for 99.4%. The practical claim is specific: gate guidance by the model's occupancy posterior, and act where it is resolved.

Your review found the MiDiffusion evidence substantial and asked that the broader conclusions be bounded more tightly. Each of your six questions is now answered with new evidence rather than by narrowing a claim: the identity, its fixed-mean approximation and the network error are separated (Table 12); the occupancy-only gate and the origin of every created object are measured (Tables 17–19); the predictor holds quantitatively on DiffuScene, with class and shape signals separated (Tables 11 and 13); all floor objects of the generated scenes are reported (Table 16); occupancy resolution is timed directly (Table 15); and PhyScene's released guidance is evaluated (Tables 20–21). We hope these results resolve the concerns behind your rating, and we would be grateful if you would consider raising it. We are glad to answer any remaining question before the discussion period ends.

AuthorsOpenReview ↗

Response to Reviewer gFXt: A short summary

Thank you again for your careful review. Before the discussion period closes, we summarize how the revision answers your overall assessment: "The MiDiffusion evidence is substantial, but several broader theoretical and practical conclusions require tighter qualification." Rather than narrowing these conclusions, the revision tests them directly:

  • Identity versus approximation. The state-dependent conditional means are now measured directly; replacing them by fixed class means changes R² by at most 0.02 (0.897 / 0.806 / 0.825 against 0.905 / 0.800 / 0.842; Table 12).
  • Occupancy versus applicability. The occupancy-only gate you proposed leaves the object count unchanged and adds only the collisions of the non-floor objects, such as lamps, that it moves to the floor. All 651 objects created by the occupancy-blind edit come from slots whose token was empty or masked, none from a slot holding an object (Tables 17–19).
  • Other models. The predictor explains DiffuScene's gaps quantitatively, R² 0.945 / 0.960 with no fitted parameter, and PhyScene's with R² 0.890 (Table 11).
  • Final support. Of all 14,129 floor objects in 2,015 scenes generated from pure noise, 0.79% / 0.28% / 0.06% lie beyond 1 / 2 / 5 cm (Table 16).
  • Existing guidance. Your review named PhyScene and noted: "The submission's interventions are not a direct evaluation of that procedure." We now evaluate it with its released checkpoint and code. The released sampler applies the two gates that Equation 3 calls for: an objectness mask, and guidance only in the last 10 of 1,000 steps. In the form of the PhyScene paper's Algorithm 1, which guides all slots at every step, the collision objective falls by 3.8 instead of 5.9 × 10⁻⁴, and 0.33% of objects are displaced by more than 50 cm instead of none (Table 21).

If these results resolve the concerns behind your rating, we would be grateful if you would consider raising it.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of the revision (to all reviewers and the ACs)

We thank the four reviewers. The reviews converge on three requests: separate occupancy from constraint applicability in the guidance experiment; connect the finding to an existing guidance implementation and validate it quantitatively beyond MiDiffusion; and show the phenomenon visually and at scale. Each is answered by new experiments on released checkpoints. Point-by-point replies follow under each review.

Unchanged. The title, the thesis, and the nine sections with their order and titles. Equations 1–4, Tables 1–10, and Figures 2–5 keep their numbers, so every reference in the reviews remains valid. Beyond the additions, 43 of the 155 sentences of the submitted abstract and main text were reworded or removed; no number or result changed.

New. New paragraphs in Sections 2–7 state each new result on the paper's three models, and a new Appendix E gives the complete results (Tables 11–23, Figures 6–7). Figure 1 now shows a schematic of the mechanism next to a rendered example; Figure 2 moved to Appendix B. The "Limitations" paragraph of Section 9 became "Scope and outlook", because the new experiments remove limitations it listed or the reviews raised: the predictor now holds on three models, a published guidance method is evaluated with its released code, final support is reported for all floor objects of 2,015 generated scenes, and final scenes are simulated as rigid bodies.

  1. The parameter-free predictor of Equation 3 holds quantitatively in three models (Table 11; Section 7). R² at t = 300: MiDiffusion 0.960 / 0.907 / 0.914 (bedroom / living / dining); DiffuScene, with the posterior read from its class channels, 0.945 / 0.960; PhyScene, read the same way along its own sampling trajectories, 0.890 for the slots that end as floor objects. Occupancy alone, a two-parameter fit, a class prior, a per-timestep constant, and posteriors shuffled across cases all fall far behind (R² at most 0.70).
  2. Identity, approximation, and network error are separated (Table 12; Sections 2 and 4). We measured the network's class-conditional estimates directly by setting the token to every class in turn. Replacing the fixed class means of Equation 3 by these state-dependent estimates changes R² by less than 0.02. Fixing the means costs 3.5–4.1 cm RMS.
  3. A factorial over slot gates isolates occupancy from applicability (Tables 17–19; Section 6). Editing slots predicted empty creates 0.72–1.02 objects per scene. Editing predicted non-floor objects leaves the object count unchanged and adds 0.9–3.1 colliding pairs. Of the 651 objects created by the all-slot edit, none comes from a slot that already held an object. Argmax, threshold, soft, and oracle gates all remove both effects.
  4. PhyScene's released guidance applies the two gates that Equation 3 calls for (Tables 20–21, Figure 7; Section 6). Algorithm 1 of the PhyScene paper guides all objects at every step (its Appendix E applies guidance on the final steps); the released code guides only slots whose empty channel in the posterior mean is negative, and only in the last 10 of 1,000 steps. On the released checkpoint the released code lowers the collision objective by 5.9 × 10⁻⁴ and moves no object by more than 50 cm; the all-object, every-step form lowers it by 3.8 × 10⁻⁴ and moves 0.33% of the objects that far. The mask agrees with the final occupancy for 79% of the slots at t = 300; the model's own posterior agrees for 99.4%.
  5. Scale and timing (Tables 14–16; Section 5). In 2,015 scenes generated from pure noise, half of the objects that end on the floor have an estimate more than 20 cm off it at t = 500 (84–88% at t = 900). The occupancy posterior resolves about 200 steps before the geometry settles. 14,121 of 14,129 final floor objects rest within 5 cm of the floor.
  6. Physical stability and scene completion (Tables 22–23; Sections 5–6). In rigid-body simulation, the floor-applicable objects of the 2,015 generated scenes fall no more often than those of the real test scenes of the same floor plans (0.5% / 0.1% against 1.4% / 0.3%; paired differences −0.9 [−2.5, 0.4] and −0.2 [−0.4, 0.1] points). The blind edit raises the share of objects that move in simulation by 23.7–40.8 points; the floor-class gates built from the posterior (argmax, thresholds of 0.5 and 0.9, soft weight) change it by −0.4 to +1.1. In scene completion these gates keep the recall of the known scene's objects, its category distribution and the between-draw variability, while edits of empty slots add objects that are not in the known scene.

Every new experiment uses released checkpoints, first reproduces saved outputs or the released sampler before it measures anything new, and was specified before its outputs existed.

AuthorsOpenReview ↗

Summary of the rebuttal: where each review point is addressed

We thank the reviewers and the Area Chair. To support the discussion, we list where each weakness (W) and question (Q) is addressed, in the revised paper and in our replies; labels follow our replies. Section, equation and table numbers of the submission are unchanged, so every reference in the reviews remains valid. The new experiments use released checkpoints and are reported in Appendix E (Tables 11–23, Figures 6–7); their results are summarized in our comment "To all reviewers: summary of the revision".

Reviewer gFXt

  • W1 / Q1 (exact identity, fixed-mean approximation, network error; residual analysis): Sections 2 and 4 (paragraph "Comparison predictors, conditional means, and the residual"); Tables 11–12, with residuals by timestep, class and occupancy confidence (Appendix E.1–E.2). Reply 1/3.
  • W2 / Q2 (occupancy-only gate; which slots become objects): Section 6 (paragraph "Occupancy versus applicability"); Tables 17–19 (Appendix E.5). Reply 2/3.
  • W3 / Q3 (DiffuScene: quantitative predictor; class signal versus shape latent): Section 7 (paragraph "The predictor transfers quantitatively"); Tables 11 and 13 (Appendix E.1, E.3). Reply 2/3.
  • W4 / Q4 (support of all final floor objects; physical validity): Section 5 (paragraph "From pure noise"); Table 16 (Appendix E.4); rigid-body simulation in Table 22 (Appendix E.7). Reply 3/3.
  • Q5 (occupancy resolution versus token commitment, timed directly): Section 5 (same paragraph); Table 15 (Appendix E.4). Reply 3/3.
  • W5 / Q6 (published diagnostic; existing guidance implementation): Section 2 (paragraph "Why paired response scores cannot see this"); Section 6 (paragraph "An existing implementation"); Tables 20–21 and Figure 7 (Appendix E.6). Reply 3/3.

Reviewer knqY

  • W1 / Q1 (contribution beyond conditional-mean denoising, MiDiffusion and D3PM): Sections 1, 2 and 8. Reply 1/4.
  • W2 / Q2 (independent mechanistic evidence; calibration; state-dependent conditional means): Section 4; calibration in Appendix E.1; Table 12 (Appendix E.2); gate factorial in Tables 17–19. Reply 1/4.
  • W3 / Q2 (occupancy-only, class-prior, timestep-conditioned and posterior-randomization baselines): Section 4 (paragraph "Comparison predictors, conditional means, and the residual"); Table 11 (Appendix E.1). Reply 2/4.
  • Q3 (zero padding versus occupancy uncertainty): Section 4; Appendix E.1. Reply 2/4.
  • W4 / Q4 (another generative representation): Section 7; Tables 9, 11 and 13; PhyScene in Appendix E.6. Reply 2/4.
  • W5 / Q5 (another dataset): the released M3DLayout model, trained on 3D-FRONT, Matterport3D and Infinigen. Reply 3/4.
  • W6 / Q6 (physics-guidance baseline): Section 6 (paragraph "An existing implementation"); Tables 20–21 and Figure 7 (Appendix E.6). Reply 3/4.
  • W7 / Q7 (oracle, threshold and soft gates): Tables 17–18 (Appendix E.5) and Table 22. Reply 3/4.
  • W8 / Q8 (simulation): Table 22 (Appendix E.7). Reply 4/4.
  • W9 / Q9 (scene completion: recall, precision, diversity): Table 23 (Appendix E.8). Reply 4/4.
  • W10 (trajectory conclusions): Section 5 (paragraph "From pure noise"); Tables 14–16 (Appendix E.4). Reply 4/4.

Reviewer iu9b

  • W1 (relation to PIDM and PIDDM): Section 2, after the proposition; Section 8. Reply 1/2.
  • W1 / Q1 (PhyScene's objectness mask; what the analysis adds): Section 6 (paragraph "An existing implementation"); Tables 20–21 and Figure 7 (Appendix E.6). Reply 1/2.
  • W2 / Q2 (occupancy-only versus occupancy-and-applicability gate; slot-level attribution): Section 6 (paragraph "Occupancy versus applicability"); Tables 17–19 (Appendix E.5). Reply 2/2.
  • W1 (validation in another representation): Section 7 (paragraph "The predictor transfers quantitatively"); Table 11 (Appendix E.1). Reply 2/2.

Reviewer HYE8

  • W1 (prevalence across methods): Section 7 and Table 11 (three models); Section 5 and Table 14 (2,015 scenes generated from pure noise). Reply 1/2.
  • W2 (the all-slot setting): Section 6 (paragraphs "Occupancy versus applicability" and "An existing implementation"); Tables 17–21. Reply 2/2.
  • W3 (practical value): Section 9 (paragraph "Implications"); Tables 21–22. Reply 2/2.
  • W4 (visual examples): Figure 1 (Section 1), Figure 6 (Appendix E.5) and Figure 7 (Appendix E.6); Appendix E.9. Reply 1/2.
  • Follow-up (practical significance): Reviewer HYE8 wrote: "The new results have strengthened the paper, especially the cross-model validation and the additional guidance experiments." The remaining question, practical significance, is answered with measurements in "Response to Reviewer HYE8: practical significance, measured".

We thank the reviewers again; their questions shaped every experiment of Appendix E.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.