- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper reconstructs bimanual interactions with deformable objects from calibrated multi-view RGB video. It estimates hand and object geometry, then fits latent force and material fields through differentiable material-point simulation. The paper also introduces a 20-sequence dataset recorded with six cameras and evaluates reconstruction on ARCTIC and HMDO, with ablations that examine both geometry and penetration.
Critical strengths
- I find the joint treatment of hand motion, contact, and nonrigid object deformation well motivated. These quantities constrain one another under occlusion, so fitting them together is a sensible contribution to a difficult reconstruction problem.
- The object-surface results on HMDO are promising, and the paired ablations report penetration as well as geometry. Those checks are relevant to the intended use of the reconstructions and give some empirical support to the interaction-modeling approach.
Critical weaknesses
- Two parts of the physical formulation prevent me from assessing the method confidently. Appendix Eq. (9) is called Neo-Hookean but appears closer to a fixed-corotated energy. Eq. (4) uses unsigned distances for a penetration penalty, without explaining how interior and exterior points are distinguished. The implemented stress and collision objectives need to be clear because they define the proposed inverse-physics model.
- The evaluation is too narrow for me to judge how reliable the method is. Fig. 3 shows only two qualitative comparisons; the video adds examples, but offers little analysis of failure cases or performance across deformation types. The main HMDO comparison covers just five sequences (150 frames), selected because all methods returned valid outputs. The ten-sequence ablations are useful, but without broader reporting and comparisons using the same inputs, I cannot tell whether the gains hold beyond the selected cases.
Missing references: PhysHanDI: Physics-Based Reconstruction of Hand-Deformable Object Interactions (ICML 2026), AGILE: Hand-Object Interaction Reconstruction from Video via Agentic Generation (SIGGRAPH 2026).
Questions for authors
- Which constitutive energy and stress law are actually implemented? Could you reconcile Appendix Eq. (9) with the term “Neo-Hookean” and clarify whether this is a labeling issue or a difference in the implemented model?
- How does Eq. (4) distinguish penetration from nearby external contact? Is a signed-distance, occupancy, or separate collision term used in the experiments?
- How many HMDO sequences did each method attempt and complete, and what observations did each receive? Please identify any comparison that holds the inputs fixed and explain how including failed cases affects the conclusions.
- Could you show reconstruction sequences covering the evaluated objects and deformation types, including difficult or failed cases, alongside per-sequence errors? This would help me judge whether the examples and average results are representative.