- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a part-aware framework based on a spring-mass system for learning physically realistic object dynamics from real-world observations. The method builds upon an existing differentiable physics simulator and focuses on estimating physical parameters through a combination of intra-part and inter-part dynamics modeling. Specifically, the intra-part module adapts spring configurations to capture varying degrees of deformation and rigidity, while the inter-part module incorporates directional stiffness to model constrained relative motion between object components. Experimental results demonstrate that the proposed method outperforms existing baselines across deformable, articulated, and heterogeneous object scenarios.
Critical strengths
- Strong experimental performance. The proposed method consistently outperforms existing baselines across deformable, articulated, and heterogeneous object scenarios, demonstrating improved geometric accuracy and future dynamics prediction.
- Effective modeling of inter-part dynamics. The proposed inter-part spring connections effectively capture constrained relative motion between object components, improving the modeling of articulated structures while accommodating flexible inter-part interactions.
Critical weaknesses
This paper primarily addresses physics parameter estimation by extending the spring-mass framework introduced in PhysTwin [1]. The proposed method incorporates part-aware spring configurations and neural anisotropic stiffness to model deformable, articulated, and heterogeneous objects within a unified representation. Although the experimental results demonstrate promising improvements, my main concerns are related to the insufficient description of the physical simulation, limited baseline comparisons, and the novelty of the proposed formulation.
Weaknesses and Questions
1. Unclear initialization of spring rest lengths.
How is the rest length in Equation (7) determined? Is it directly calculated from the initial observed geometry, estimated from the interaction observations, or manually specified? In particular, how are the rest lengths of the newly introduced long-range springs initialized?
2. Insufficient details about the physical dynamics formulation.
In Section 3.4, the authors briefly mention that gravity, contact, and friction forces contribute to the particle dynamics. However, Equation (7) only describes the strain-based spring force. What are the mathematical formulations and numerical treatments of the contact and friction forces? Furthermore, the paper does not explicitly provide the complete equations of motion governing the evolution of particle positions and velocities. A more detailed description of the physical dynamics and numerical integration scheme would improve the technical clarity and reproducibility of the method.
3. Missing comparison with SoMA [2].
The paper cites SoMA [2] but does not provide an experimental comparison. SoMA also aims to learn physically realistic object dynamics from real-world interactions, although it adopts a neural dynamics formulation rather than an explicit spring-mass model. A comparison on applicable deformable-object scenarios would help demonstrate the relative advantages of the proposed physics-based framework, particularly in terms of prediction accuracy and generalization to unseen interactions. While I acknowledge that SoMA does not explicitly model inter-part constraints in the same manner as the proposed method, it remains a relevant baseline for evaluating generalized object dynamics modeling.
4. Insufficient details about the PhysTwin baseline on the newly collected scenarios.
The paper introduces 11 self-collected articulated and heterogeneous interaction scenarios. However, it is unclear how PhysTwin [1] is adapted and optimized for these new scenarios. Does PhysTwin use exactly the same training observations, reconstructed geometry, tracking data, and controller trajectories as the proposed method? Are its topology and physical parameters re-optimized for each scenario using comparable optimization budgets?
5. Clarification of controller trajectories and evaluation fairness. Are the controller trajectories used during future prediction obtained from the held-out ground-truth observations, or are they estimated independently? Furthermore, do all baseline methods use the same controller points and trajectories during evaluation? If not, how are differences in control inputs accounted for to ensure a fair comparison?
6. Limited novelty and insufficient ablation studies to isolate the contributions over PhysTwin.
In the Introduction, the authors raise the question: "Can we achieve generalized dynamics modeling across deformable, articulated, and heterogeneous objects within a common physical representation?" However, PhysTwin [1] has already introduced a unified spring-mass representation with optimized spring connectivity and stiffness. Although PhysTwin primarily focuses on deformable objects, its underlying representation is not inherently restricted to such objects.
The proposed method appears to extend PhysTwin primarily through part-aware spring configuration, rigidity-adaptive long-range connections, and direction-dependent inter-part stiffness. While these modifications are reasonable and potentially effective, the paper does not sufficiently establish their individual and combined contributions relative to the original PhysTwin formulation.
In particular, Table 2 separately evaluates the variants "w/ dense local" and "w/o anisotropy", but does not report a variant combining both modifications. I suggest adding an ablation that simultaneously replaces the adaptive long-range springs with dense local connections and removes the neural anisotropic stiffness. This configuration would be closer to the conventional spring-mass formulation in PhysTwin, although differences in material priors, part-aware parameterization, and preprocessing may still remain.
Such an ablation would help clarify how much of the improvement comes specifically from the proposed spring topology and directional stiffness modeling. Ideally, the comparison should use the same preprocessing pipeline and optimization settings to isolate the effects of the proposed dynamics representation.
References
[1] Hanxiao Jiang, Hao-Yu Hsu, Kaifeng Zhang, Hsin-Ni Yu, Shenlong Wang, and Yunzhu Li. PhysTwin: Physics-Informed Reconstruction and Simulation of Deformable Objects from Videos. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7219–7230, 2025.
https://openaccess.thecvf.com/content/ICCV2025/html/Jiang_PhysTwin_Physics-Informed_Reconstruction_and_Simulation_of_Deformable_Objects_from_Videos_ICCV_2025_paper.html
[2] Mu Huang, Hui Wang, Kerui Ren, Linning Xu, Yunsong Zhou, Mulin Yu, Bo Dai, and Jiangmiao Pang. SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-Body Manipulation. Proceedings of the 43rd International Conference on Machine Learning (ICML), PMLR 306:47834–47850, 2026.
https://proceedings.mlr.press/v306/huang26bh.html
Questions for authors
Please refer to the weaknesses.
8 comments
Response to reviewer DvFt (Part2)
[WQ2] Insufficient details about the physical dynamics formulation.
Thank you for requesting these details. We have expanded Appendix B.5 with the complete equations and numerical treatment. We summarize the main updates below.
Particle Dynamics. We advance the spring-mass system using substeps indexed by , each with duration . The spring and axial damping forces in Eq. (7), together with gravity, update particle velocities as
where denotes spring-connected neighbors, is gravitational acceleration, and controls drag damping.
What are the mathematical formulations and numerical treatments of the contact and friction forces?
Contact and Friction. Contact is activated for collision-enabled pairs with distinct collision labels when their separation falls below the collision threshold. For a contacting pair , let be the unit normal from to . We decompose relative velocity as
The impulse applied to particle is
Here, and are restitution and friction coefficients, and prevents division by zero. The normal component produces restitution, while the tangential component opposes sliding under Coulomb friction. We average impulses over the active contact set :
Without active contacts, .
Furthermore, the paper does not explicitly provide the complete equations of motion governing the evolution of particle positions and velocities.
Ground Contact and State Update. Ground collisions use normal restitution and Coulomb friction, with position updates split at the estimated time of impact. Without ground contact, the state update is
The complete ground-contact and position-update equations are included in Appendix B.5.
Response to reviewer DvFt (Part3)
[WQ3] Missing comparison with SoMA. A comparison on applicable deformable-object scenarios would help demonstrate the relative advantages of the proposed physics-based framework, particularly in terms of prediction accuracy and generalization to unseen interactions.
Thank you for your suggestion. We have adapted SoMA's official code release to the seven deformable-object scenarios from the PhysTwin dataset. Implementation details and qualitative and quantitative comparisons are provided in Appendix C.4. The key settings are summarized below.
For preprocessing, we convert the same RGB-D observations, camera calibrations, and object tracks used by GenTwin to SoMA's scene format, retaining the same train/test splits. For controller setting, we use the shared 3D hand trajectories to drive autoregressive prediction without reinitialization at the train/test boundary. For optimization, we train a separate model per scenario on four NVIDIA A100 GPUs, following SoMA's two-stage procedure with 15 epochs of coarse dynamics training followed by 10 epochs of frame-to-frame training, and perform inference on a single A100 GPU. For evaluation, we compute geometric and tracking metrics on the predicted object Gaussian centers throughout the rollout, assigning each reference track to its nearest Gaussian in the initial frame and keeping this correspondence fixed.
The table below reports results averaged over the seven deformable-object scenarios.
| Evaluation | Method | CD↓ | Track↓ | IoU↑ | PSNR↑ | SSIM↑ | LPIPS↓ |
|---|---|---|---|---|---|---|---|
| Reconstruction & Resimulation | SoMA | 0.014 | 0.029 | 0.732 | 26.842 | 0.948 | 0.061 |
| Reconstruction & Resimulation | Ours | 0.004 | 0.007 | 0.865 | 27.687 | 0.951 | 0.023 |
| Future Prediction | SoMA | 0.039 | 0.092 | 0.418 | 21.301 | 0.910 | 0.121 |
| Future Prediction | Ours | 0.010 | 0.020 | 0.745 | 25.097 | 0.941 | 0.048 |
Under this single-sequence training setting, SoMA achieves worse performance than GenTwin. Through discussions with SoMA's first author, we learned that its original configuration was designed for joint learning from multiple interaction sequences, and training on a single sequence may require substantial manual hyperparameter tuning. These results suggest that GenTwin performs better than SoMA under a single-interaction training configuration.
Response to reviewer DvFt (Part1)
Thank you for your careful review and constructive feedback. We have addressed your comments below and updated the revised manuscript, with revisions highlighted in blue.
Thank you for requesting this clarification. We have clarified the initialization of spring rest lengths in Sec. 3.4. For local springs, CMA-ES searches the connection radius and maximum neighbor count, which determine the connected particle pairs. For each connected pair (i,j), the rest length is computed directly as the Euclidean distance between the two sampled points in the initial object geometry P0:
ℓ(i,j)=∣xj0−xi0∣2.
For long-range springs, we connect sampled points in the initial object geometry within the same near-rigid part p, using their Euclidean distances as the rest lengths. Following Eq. (4), only connections within the prescribed distance range, scaled by the part extent Lp=maxi,j∈Vp∣xi0−xj0∣2, are retained:
β−Lp≤∣xi0−xj0∣2≤β+Lp,i,j∈Vp.
In our default implementation, the prescribed normalized range is (β−,β+)=(0.4,0.405). To evaluate this parameter choice, we added a long-range spring distance ablation on the 11 self-captured scenarios in Sec. 4.3. The table below summarizes future prediction results for different values of β−.
As shown in the table, springs near 0.4Lp strike a favorable balance across all metrics. Shorter springs provide weaker rigidity constraints, while longer springs yield comparable performance.