acceptodds
Under review as a conference paper at ICLR 2027

Xperience-0: Scaling Loco-Manipulation World Models across Embodiments with Human Experience

Abstract

World models enable embodied agents to anticipate action consequences, yet jointly modeling locomotion and manipulation across diverse embodiments remains challenging due to heterogeneous action spaces and limited multimodal supervision. We present Xperience-0, the first egocentric loco-manipulation world foundation model spanning humans and robots with diverse embodiments. We curate Xp-Data, comprising 10k hours of egocentric human data and 3k hours of robot data with unified multimodal annotations and systematic quality control. A unified kinematic representation encodes locomotion through camera poses and manipulation through projected skeletons, with robot skeletons derived from embodiment-specific forward kinematics. We also introduce Xp-Teleop, a long-horizon loco-manipulation benchmark based on high-quality dexterous humanoid teleoperation data. Evaluations demonstrate state-of-the-art performance on the DreamDojo, , -HOME, and Xp-Teleop benchmarks, from zero-shot inference to few-shot and full-data post-training. Scaling human pre-training data improves performance on both human and robot evaluations. After distillation, our model supports real-time, minute-long rollouts, enabling live VR interaction. Applications in policy evaluation, model-based planning, and world–action modeling further demonstrate the model's utility for embodied decision-making.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.