Surface-Sync Human–Object Interaction Motion Generation via Contact Modeling and Joint Hand–Body Optimization
Abstract
Human–object interaction (HOI) motion generation remains challenging because realistic interactions require modeling not only human and object motions, but also their evolving physical relationship at the interaction interface. Existing approaches mainly represent interactions through joints, trajectories, or object poses, leaving surface correspondence, contact quality, and mechanical stability implicit. In this paper, we propose Surface-Sync, a unified framework for HOI motion generation that explicitly models interaction from representation and physical reasoning perspectives. We introduce a surface-centric interaction representation based on sparse surface markers, which captures dynamic human–object surface correspondences through contact activation and canonical object-surface anchors. This representation provides a unified basis for incorporating geometric compatibility and wrench-space physical priors, enabling the generation of physically stable and coherent interactions. Furthermore, we propose an interaction-driven whole-body coordination mechanism that propagates local interaction constraints from end-effectors to the surrounding body structure, improving the consistency between manipulation and global motion. Extensive experiments demonstrate that our approach generates more realistic, physically plausible, and coordinated human–object interaction motions compared with existing methods.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.