Trace2Stable: Learning from Physical Search for 3D Scene Refinement

Image-based 3D reconstruction often produces visually plausible but physically invalid scenes: objects may fall or interpenetrate when simulated. Physical refinement must resolve these violations without replacing the recovered geometry or unnecessarily changing the observed arrangement. Existing per-scene search repeatedly simulates candidate pose corrections to find a stable, collision-free layout. We propose Trace2Stable, which amortizes physical search by learning reusable repair hypotheses from optimization traces. A graph-conditioned flow proposes multiple pose corrections, and candidate support relations represent ambiguous contacts. Simulation verifies the proposed repairs; measured outcomes guide residual refinement, while candidate ordering, alternative support hypotheses, and adaptive rollout continuation direct the remaining verification budget. Prior searches thus guide which repairs to test, while physical tests choose a valid arrangement or preserve an already valid input. We compare against the cross-entropy method (CEM), which searches for pose corrections from scratch in each scene. On 32 controlled layouts with matched simulation objects, Trace2Stable increases the proportion of stable outputs without illegal collisions from 21.88 to 59.38 percent while using 46.95 percent fewer candidate-environment simulation steps than this per-scene search baseline.