- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper introduces Trace2Stable, a framework designed to tackle the physical invalidity (e.g., collisions, interpenetrations, and gravitational collapse) commonly found in image-based 3D scene reconstructions. Rather than running sample-inefficient per-scene searches from scratch (such as the Cross-Entropy Method used in REST3D), the authors propose amortizing physical search by learning reusable repair hypotheses from optimization trajectories.
The framework employs a graph-conditioned flow to propose multi-modal pose corrections under ambiguous support relationships, followed by budgeted physical simulation verification and residual refinement. On a benchmark of 32 controlled layouts, Trace2Stable improves the proportion of stable, collision-free configurations from 21.88% to 59.38% while reducing candidate-environment simulation steps by 46.95%. Transfer capabilities are further probed on datasets including ReplicaCAD, HSSD, and ScanNet++.
Critical strengths
- Conceptual novelty of amortizing physical search: Formulating 3D physical scene refinement as an amortized learning problem over historical optimization trajectories is an insightful and principled direction. It effectively addresses the cold-start inefficiency of standard black-box heuristic search methods like CEM.
- Synergistic framework design: Coupling generative multi-modal hypothesis generation (via graph-conditioned flow) with budgeted simulator-in-the-loop verification strikes an appealing balance between leveraging learned physical priors and guaranteeing physical validity through simulation.
Critical weaknesses
-
The core quantitative claims in Section 4 heavily hinge on an evaluation over only 32 controlled layouts. In absolute terms, improving the collision-free stability rate from 21.88% to 59.38% corresponds to successfully resolving merely 12 additional scenes (an increase from 7/32 to 19/32). Indoor scene arrangements present an immense combinatorial configuration space characterized by multi-object contact topologies, mass distributions, and geometric friction boundaries. Evaluating amortized learning—whose foundational premise is the cross-scene transferability of optimization traces—on a test set of just 32 layouts is statistically fragile. Such a small sample size cannot rule out sampling variance, object-category bias, or overfitting to specific canonical arrangements, leaving the generalization capability of the graph-conditioned flow unsubstantiated.
-
A foundational premise established in the Introduction (Section 1) is that physical refinement must eliminate collisions and instability "without replacing the recovered geometry or unnecessarily changing the observed arrangement." However, the paper creates a significant evaluation disconnect by focusing almost exclusively on simulation stability and candidate-environment step counts, while omitting rigorous metrics quantifying visual fidelity and pose drift. In a rigid-body physics simulator, it is trivially easy to achieve "stability" and "zero collision" by pushing objects far apart, shifting them out of the camera's line of sight, or snapping them directly onto the floor plane. Without measuring translation drift (Delta T in cm), rotation drift (Delta R in degrees), 2D mask/depth reprojection errors, or Chamfer Distance relative to the initial reconstruction, there is no empirical guarantee that physical stability was not achieved at the cost of catastrophic layout distortion.
-
The paper touts a 46.95% reduction in "candidate-environment simulation steps" compared to CEM. While reducing simulator rollout queries is desirable, it paints an incomplete picture of computational complexity. Trace2Stable introduces considerable auxiliary overhead: graph neural network message passing, multiple ODE integration steps for the conditional flow to generate multi-modal candidates, support-tree hypothesis branching, and candidate re-ranking. The Cross-Entropy Method, while sample-inefficient in query counts, involves negligible per-sample neural latency. Without reporting full end-to-end wall-clock time (GPU forward sampling + CPU/GPU simulation verification + residual optimization), it remains unverified whether this method actually accelerates overall scene processing or merely transfers the computational bottleneck from the physics engine to the neural network.
-
Despite combining learned flow priors, support branching, candidate bank selection, and residual physics refinement, the terminal success rate plateaus at only 59.38%. This indicates that more than 40% of scenes in the controlled synthetic test suite still suffer from interpenetration or gravitational collapse, reflecting a high failure rate for an iterative refinement pipeline with simulator-in-the-loop residual search. Furthermore, when transferring to real-world/scanned datasets such as ScanNet++ and HSSD, the paper noticeably downgrades its evaluation to "support selection and layout preservation" rather than executing full rigid-body dynamic rollouts to measure terminal stability and penetration. Real-world meshes contain scanning artifacts, partial occlusions, and non-convex concavities where physics engines notoriously fail; bypassing end-to-end simulation rollouts on ScanNet++ sidesteps the core challenge of real-to-sim transfer.
Questions for authors
-
Can the authors provide experimental results across a substantially larger benchmark (e.g., hundreds of scenes from ProcTHOR, Habitat-Matterport3D, or the complete ReplicaCAD suite)? Specifically, what are the mean success rates, standard deviations across multiple random seeds, and confidence intervals to demonstrate that the amortized flow prior generalizes beyond the 32 controlled layouts?
-
To verify that the refinement does not achieve physical stability by drastically altering the scene geometry or violating the observed layout: (a) What are the mean/median translation drifts (in cm) and rotation drifts (in degrees) of objects after refinement relative to the unrefined initial reconstruction? (b) Could the authors report 2D visual reprojection metrics (e.g., multi-view rendering PSNR/SSIM, segmentation mask IoU, or depth L1 error) comparing the refined 3D layout against the input single-view image?
-
Could the authors provide a comprehensive wall-clock time breakdown (in seconds per scene) comparing Trace2Stable and the baseline CEM? Specifically, please dissect:
- Neural proposal generation time (GNN feature extraction + Flow sampling of K candidates).
- Physical verification time (simulator rollout steps).
- Residual refinement optimization time. Is Trace2Stable strictly faster in wall-clock execution than CEM on identical hardware?
-
Given that the terminal success rate is 59.38% on the controlled benchmark, what are the primary root causes for the remaining 40.62% failure cases? Does the graph-conditioned flow generate out-of-distribution initial proposals that trap the local residual search in poor local minima, or does the budgeted rollout prematurely terminate before convergence? A detailed quantitative failure mode breakdown (e.g., percentage failed due to unresolvable contact cycles, severe mesh penetration, or unstable stacking) would be highly illuminating.
-
Why was terminal dynamic simulation omitted in the ScanNet++ transfer experiments in favor of surrogate metrics like support selection? What is the actual collision-free stability rate when Trace2Stable is deployed on reconstructed ScanNet++ meshes inside Isaac Gym / PyBullet, and how does the method handle non-watertight geometry and noisy reconstructed surfaces?
Hi Reviewer HroF,
We sincerely apologize for the additional reminder, and we greatly appreciate your time and consideration throughout the discussion. As the discussion period is approaching its end, with only 6 hours remaining, we would like to kindly follow up on our responses and revisions.
In particular, regarding your concerns about the generalizability and statistical reliability of our results, the preservation of observed visual layouts during physical refinement, the actual end-to-end runtime, and the validity of dynamic simulation on raw reconstructed meshes, we have conducted additional evaluations and substantially expanded our analyses. Specifically, we have added independent source-level evaluations, further examined input-layout preservation and relative object arrangements, provided more detailed runtime measurements and failure analyses, and supplemented the physical validation on scan-derived meshes, including collision representations and physical-property consistency.
We have also revised the manuscript and supplementary materials to make these results and evaluation protocols clearer. We sincerely hope that these additional experiments, analyses, and clarifications have adequately addressed the concerns you raised. If you feel that our revisions have resolved these issues, we would be very grateful if you could reconsider the paper's rating.
Thank you again for your thoughtful feedback, constructive suggestions, and careful evaluation. We truly appreciate the time and effort you have devoted to helping us improve this work.
Best regards,
Authors of Submission 1727