An Interaction Is More Than Two Motions: Unifying Motion Generation and Policy Control For Dyadic Humanoid Interaction
Abstract
Recent advances in physics-based humanoid motion imitation have improved individual robot agility, motivating growing interest in physically coupled humanoid interactions. However, control-oriented approaches often rely on predefined motion templates, while 3D motion generators synthesize diverse interactions but give limited consideration to policy-based execution on physical robots. Our key insight is that generation, embodiment adaptation, and execution should be guided by a shared representation of interaction relations. We present DyadLink, a generation-driven framework that synthesizes a compatible follower motion from a leader's reference motion under multimodal conditioning, such as language or music. DyadLink uses a time-varying, directed Shared Relational Graph (SRG) to maintain interaction relations throughout the pipeline. Its role-asymmetric leader–follower formulation prioritizes fidelity to the leader's prescribed motion while allowing the follower to adapt its motion to preserve the intended interaction. To make it actionable across heterogeneous stages, we design a series of stage-specific, learnable and differentiable relational operators that translate its shared specification into generation guidance, embodiment-aware optimization objectives, and policy-learning signals. Experiments on InterHuman and CoDance show improved contact fidelity and pairwise geometric consistency across generation, retargeting, and policy execution. We further demonstrate the resulting interactions on physical humanoids. Source code will be made available.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.