- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
HermiteTwin reconstructs a physical digital twin of a thin deformable object from calibrated multiview videos and recorded robot interactions. Its central idea is to decouple deformation and grasp resolution: a coarse bicubic Hermite shell models membrane and bending behavior, while dense material-space samples implement capture, clamp, and release through attachment constraints. Effective material and contact parameters are identified using CMA-ES followed by adjoint-based refinement. The simulated shell then drives a reconstructed Gaussian appearance model. Experiments on ten Deform360 interactions involving a red cloth and a white foam sheet show improved reconstruction, resimulation, and continuation prediction compared with PhysTwin and GS-Dynamics. Additional synthetic scenes demonstrate qualitative interactions with new actions and objects.
Critical strengths
-
The modeling choices are well motivated for thin deformable objects. Separating a compact shell state from dense grasp samples addresses the different resolution requirements of deformation and contact. Explicit, independently adjustable bending stiffness is appropriate for the cloth and foam examples. The attachment energy is also straightforward to integrate into the implicit solver: for fixed active samples and targets, it is quadratic in the shell state.
-
The reported improvements over the selected baselines are meaningful. Relative to PhysTwin, reconstruction CD decreases from 61.89 to 42.05 mm², and future-prediction CD decreases from 120.40 to 67.75 mm²—approximately 32% and 44% reductions, respectively. Silhouette and image metrics also improve. The qualitative examples illustrate better recovery of grasp-induced folds and preservation of the foam sheet’s shape.
Critical weaknesses
-
Limited scale and diversity of the evaluation data. The experiments use only 10 interaction sequences from Deform360, involving just two objects: a red cloth and a white foam sheet. This limited coverage does not sufficiently demonstrate the method’s generalization ability. Evaluation on additional datasets or a broader set of objects and interactions from Deform360 would strengthen the empirical evidence.
-
Limited baseline comparisons. The evaluation would benefit from comparisons with more recent baselines, such as SoMA: A Real-to-Sim Neural Simulator for Robotic Soft-body Manipulation, or stronger spring–mass methods, such as NeuSpring: Neural Spring Fields for Reconstruction and Simulation of Deformable Objects from Videos. Other relevant, stronger baselines that are easier to reproduce would also be suitable.
Questions for authors
-
Cross-interaction transfer: Can the parameters identified from one interaction be held fixed to predict another real interaction involving the same object? How much do the identified material and contact parameters vary across recordings of the same object?
-
Additional experiments: As suggested in Critical Weaknesses, could the authors evaluate the method on more diverse data and compare it against stronger baselines?