ARIA: Assistive Teleoperation for Humanoid Loco-Manipulation via Residual Interaction Refinement

Whole-body teleoperation enables an operator to drive a humanoid as a physical avatar, but this intuitive body-level mapping has a blind spot: it cannot close the loop at the contact interface with objects. Kinematic mismatches between human and robot cause grasps to fail and objects to slip, yet the operator — limited by a restricted view of the contact and the cognitive load of coordinating locomotion with manipulation — cannot compensate in real time. Therefore, we introduce ARIA (Assistive Residual Interaction for Avatars), a co-pilot that closes this loop on the operator's behalf: it augments a frozen teleoperation policy with bounded, perception-gated corrections to task-relevant joints conditioned on egocentric object geometry, and reverts to pure teleoperation when perception is unavailable. In simulation, on three loco-manipulation tasks (picking up a box from a platform or from the ground, and carrying a table), ARIA achieves an average success rate of 84.2%, where all whole-body teleoperation baselines fail, and reduces the object error by 36–59% relative to the best baseline while keeping the root-relative tracking error within the range of the baselines. Deployed on a Unitree G1 with PICO VR or a motion-capture suit and ArUco-based onboard perception, ARIA enables assisted box pickup where unassisted teleoperation fails.