- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper studies the task of diffusion-based 3D scene generation. Specifically, the problems of the disobeyed floor support constraint during intermediate diffusion steps. The authors point out that, when a model has not yet determined whether a specific slot contains an actual object, its intermediate geometric prediction may manifest as a weighted average of geometric outcomes corresponding to different semantic states. This intermediate predictions may create deviation from the ground support plane. This paper did a thorough analysis to this effect on MiDiffusion and DiffuScene, The results indicate that a predictor constructed based on class posterior probabilities and class-conditional means—which requires no additional fitted parameters—can quite accurately account for the ground clearance derived from intermediate geometric predictions.
Critical strengths
- The paper provide a new aspect to the study of 3D scene generation. It provides a valuable perspective for understanding the intermediate representation of three-dimensional scene diffusion models.
- The theoretical analysis is relatively clear and provides a reasonable explanation for the experimental results.
- The qualitative experimental design is relatively systematic, incorporating a variety of complementary validation methods.
- The discovery: "additional object may be generated due to intermediate constraint on the diffusion process" provides a new discovery of diffusion-based 3D scene generation.
Critical weaknesses
- The motivation for this paper is not sufficiently justified. This paper has raised a actual problem on diffusion based 3D scene generation, the ground-support bias in intermediate denoising predictions, and pointed out this bias can be explained mistakenly as the lack of physics understanding. However, the prevalence of this issue across existing methods has not been sufficiently demonstrated (validation was limited to only two methods: MiDiffusion and DiffuScene).
- The experiment in Section 6 applies ground constraints to all slots, including empty slots and objects that do not need ground support. This setup seems unrealistic and may itself explain the extra objects and increased collisions.
- The practical value for final scene quality remains unclear. In the studied MiDiffusion trajectories, all retained ground-supported objects meet the 5 cm support threshold. If these objects already satisfy the constraint, what practical generation problem does analyzing intermediate floating help solve?
- The paper focuses on theoretical analysis and includes no figures explanation in either the main text or supplementary material. For a 3D scene generation task, visual examples would make the paper easier to understand and provide qualitative support for its conclusions.
Questions for authors
Please refer to the weakness part.
4 comments
Response to Reviewer HYE8 (2/2): the all-slot setting and the practical value
W2. "The experiment in Section 6 applies ground constraints to all slots, including empty slots and objects that do not need ground support. This setup seems unrealistic and may itself explain the extra objects and increased collisions."
The all-slot setting is the published algorithm of an existing method. Algorithm 1 of the PhyScene paper guides all objects at every step (its Appendix E applies guidance on the final steps); its released code implements the two gates that Equation 3 calls for, an objectness mask and the last 10 steps. On PhyScene's released checkpoint (Table 21) the released code lowers the collision objective by 5.9 × 10⁻⁴ and moves no object by more than 50 cm; the all-slot form lowers it by 3.8 × 10⁻⁴ and moves 0.33% of the objects that far.
The new factorial separates the ingredients you name (Tables 17–18).
- Editing empty slots creates 0.72–1.02 objects per scene.
- Editing objects that need no floor support leaves the object count unchanged and adds 0.9–3.1 colliding pairs.
- Editing floor objects alone changes nothing.
The extra objects come from slots that were empty or still undecided: of the 651 objects created by the all-slot edit, 411 come from slots whose token had committed to empty and 240 from masked slots, and none from a slot that held a real token (Table 19).
W3. "If these objects already satisfy the constraint, what practical generation problem does analyzing intermediate floating help solve?"
Three, each measured.
- Evaluation. Reading intermediate estimates as physics misdiagnoses a model whose samples are supported (14,121 of 14,129 floor objects within 5 cm; under rigid-body simulation they fall no more often than the objects of real scenes, Table 22). Subtracting the predictor corrects the readout and removes 78–86% of the apparent violation. Paired scores stay perfect throughout and cannot detect it.
- Guidance. Methods that impose physics during sampling act on exactly these estimates. Acting on a slot whose occupancy is unresolved can create an object, and the model's posterior says which slots are safe and from which step. PhyScene's released guidance already gates by occupancy and time; its noisy-state mask is right for 79% of the slots at t = 300, where the model's own posterior is right for 99.4%.
- Few-step sampling and early stopping read intermediate estimates as outputs: at t = 200, 8–9% of the objects that would end on the floor are still estimated more than 20 cm off it.
The finding also redirects effort: the final failures of the tracked targets are occupancy and identity failures, so the remedy lies in the padding representation and in posterior-gated guidance, which the analysis specifies.
We are glad that you found the perspective valuable and the creation of objects by constraints during sampling a new discovery. Each of the four weaknesses is now answered: rendered examples and a new plot (Figures 1, 6 and 7); the predictor on three models (Table 11), and in 2,015 generated scenes 84–88% of the objects that end on the floor estimated more than 20 cm off it at t = 900 (Table 14); the all-slot setting as the published algorithm of an existing method, PhyScene, with a factorial that separates its ingredients (Tables 17–21); and the practical value, measured for evaluation, guidance and few-step sampling. We hope the revision makes both the motivation and the practical value clear, and we would be grateful if you would consider raising your rating. We are glad to answer any remaining question before the discussion period ends.
Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to address my concerns. The new results have strengthened the paper, especially the cross-model validation and the additional guidance experiments.
However, I still have concerns about the practical significance of this work. The all-slot projection setting seems less realistic, as existing methods like PhyScene already use objectness masks to avoid such issues. Although the additional experiments demonstrate that intermediate floating is common across different models, it remains unclear how often this phenomenon causes actual failures in existing guidance methods, or how the findings can improve existing scene generation methods in practice.
Overall, while the additional experiments have addressed several of my concerns, I remain concerned about the practical significance of this work. Therefore, I would like to keep my original rating.
Response to Reviewer HYE8: practical significance, measured
Thank you for the follow-up, and for noting that the cross-model validation and the guidance experiments strengthened the paper. We answer the remaining question, practical significance, with measurements.
"existing methods like PhyScene already use objectness masks to avoid such issues"
We agree that PhyScene's released code avoids the issue, and its mask is the clearest sign that the issue arises in practice. Every empty slot of PhyScene decodes to the same 2.4 × 1.8 × 1.4 m box, centred 1.70 m above the floor. Without the mask, the empty slots of a scene (8.75 on average) enter the collision objective as copies of this box: the gradient reaches its clip in 98% of the scenes at the final step, and the reduction of the collision objective falls by 39% (5.9 → 3.6 × 10⁻⁴; Table 21). The PhyScene paper does not describe the mask; its Algorithm 1 guides all N slots at every step. Our analysis explains why the mask is needed (the empty term of Equation 3) and when it is valid: the mask reads the posterior mean, which is close to the noisy state, and agrees with the final occupancy for 79% of the slots at t = 300 and for at least 99.9% only from step 91 on. The released code guides only the last 10 of 1,000 steps, which lie inside that range.
"it remains unclear how often this phenomenon causes actual failures in existing guidance methods"
As you note, intermediate floating is common across models. It becomes a failure whenever guidance acts without the occupancy gate, and then failure is the rule rather than the exception:
- PhyScene in the form of its Algorithm 1 (all slots, every step): the collision objective falls by 3.8 instead of 5.9 × 10⁻⁴, and 0.33% of objects displaced by more than 50 cm instead of none (Table 21).
- Every-step floor projection in MiDiffusion: 0.8–1.1 spurious objects per scene, and 23.7–40.8 points more objects that move in rigid-body simulation (Tables 3 and 22).
"how the findings can improve existing scene generation methods in practice"
- A rule with a measured value on an existing method. Gating guidance by occupancy and acting where occupancy is resolved gives PhyScene's Algorithm 1 a reduction of the collision objective 1.6 times as large and removes every displacement beyond 50 cm.
- A gate that holds at every step. The model's own posterior identifies the final occupancy for 96% of the slots at t = 900 and 99.4% at t = 300, where the mask is at 50% and 79% (Figure 7). Guidance that acts while the layout is still forming can use it: in MiDiffusion, the posterior gates keep the object count within 0.03 and the colliding pairs within 0.06 of native sampling, and change the share of objects that move by −0.4 to +1.1 points (Tables 17, 18 and 22).
- Evaluation. Subtracting the predictor removes 78–86% of the apparent violation that a diagnostic reading intermediate estimates would report; paired scores cannot detect it.
We hope these measurements answer the question of practical significance, and we would be grateful if you would take them into account in your final rating.
Response to Reviewer HYE8 (1/2): figures and prevalence
Thank you. The revision addresses the four weaknesses directly. Its structure and numbering are those of the submission; the additions are new paragraphs, a redrawn Figure 1, two new figures, and Appendix E.
The paper now has two rendered examples and a new plot.
Across methods. Three models and six released checkpoints now carry the quantitative predictor (Table 11): MiDiffusion (R² 0.960 / 0.907 / 0.914), DiffuScene (0.945 / 0.960), and PhyScene (0.890 for the objects it places on the floor, along its own trajectories).
Within a method, the phenomenon is near-universal (Table 14). In 2,015 MiDiffusion scenes generated from pure noise, 84–88% of the objects that end on the floor have an estimate more than 20 cm off it at t = 900, half at t = 500, and 22–25% at t = 300. PhyScene's own trajectories show the same course (54% at t = 500, 23% at t = 300).