acceptodds
Under review as a conference paper at ICLR 2027

Unsure What Exists, Not Missing Physics: Intermediate Denoising Estimates Violate Support Constraints in 3D Scene Diffusion

Abstract

Intermediate denoising estimates of 3D scene diffusion models are often read as evidence about learned physical constraints: a lifted object is pulled back to the floor, or a large residual gap is reported as a violation. We show that, in joint semantic–geometric diffusion, the second reading is usually wrong for a simple reason. The clean-geometry estimate is a posterior mean over what each slot might contain, and for an affine constraint such as floor support this mean leaves the constraint plane exactly when the posterior places mass on states that do not satisfy it: an empty slot, whose zero-padded geometry decodes to a floating box, or a non-floor category. We test this account with controlled semantic-token interventions on official test rooms of three room-specific MiDiffusion checkpoints and, with continuous class channels, on DiffuScene. A predictor with no fitted parameter, the network's own auxiliary occupancy posterior times the decoded empty coordinate plus class-conditional training means, reproduces the geometry head's floor gap ( up to 0.96), tracks its growth with noise level, and matches the sign and magnitude of every context intervention. Replaying complete reverse trajectories step by step shows that the estimate settles on the floor before the categorical token commits, that tokens commit exactly once, and that every final failure among 592 known-scene trajectories is a target whose occupancy was unresolved at the intervention step. The consequence is practical: projecting intermediate estimates onto the floor at every step without conditioning on occupancy adds 0.8–1.1 spurious objects per scene and raises the colliding-object fraction by 24–35 points, and occupancy-blind collision guidance on the noisy state likewise creates objects; gating on the occupancy argmax removes most of these objects but does not make collision guidance effective. Paired response scores cannot see any of this, because they cancel the shared offset. Constraint diagnostics and guidance on intermediate estimates must therefore be conditioned on occupancy.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.