Close encounters: decentralized multi-agent policy learning for contact-rich embodied interaction

Contact-rich interaction is central to physical assistance and close cooperation, yet poses a challenging coordination problem: agents must make intentional contact, avoid interpenetration, and continually adapt to one another while using only partial, egocentric observations. We formulate this problem as a decentralized multi-agent control task involving two virtual human agents. Each agent has access to partial egocentric observations and fine-grained contact goals specified by body-surface marker pairs. Collaboration emerges through online multi-agent reinforcement learning (MARL) of a single policy that serves both agents across all tasks. We learn this coordination without captured two-person interaction data by introducing a shared latent action space for planning and control. Both agents operate in this space, which is learned from single-human motion data. The same representation enables us to generate synthetic interaction demonstrations through centralized trajectory optimization in the joint latent action space of the two agents, and use these demonstrations to bootstrap online RL. Across nine tasks with variation in body shape and initial pose, this combination increases success over online RL from 36% to 83% on two-contact tasks and from 23% to 75% on four-contact tasks.