EgoX: Egocentric Cross-Embodiment Manipulation with Embodiment Dreaming

Scaling robot manipulation requires learning from demonstrations collected across humans and heterogeneous robots. These sources cover diverse objects, scenes, tasks, and interaction strategies, but gaps in sensing, kinematics, motion patterns, and action spaces make it difficult to train a shared policy effectively. We introduce EgoX, an egocentric cross-embodiment framework for robot manipulation. Its central design is to share manipulation structure at the level of semantically corresponding modalities, while preserving embodiment-specific sensing and control interfaces. EgoX stores human and robot demonstrations as unified egocentric observation-action trajectories, pretrains a Modularized Cross-Embodiment Transformer with Embodiment Dreaming (MXT+) on the mixture, and fine-tunes it on target-embodiment demonstrations. MXT+ combines embodiment-specific modality tokenizers and action experts with a shared Transformer trunk. An auxiliary embodiment dreaming objective predicts future embodiment latents from egocentric visual and proprioceptive features, using exponential-moving-average copies of the modality encoders; this training-only branch encourages the trunk to encode embodiment-specific temporal structure without adding deployment-time computation. We evaluate EgoX on five manipulation tasks across five real robots (wheeled humanoid, legged humanoid, quadrupedal manipulator, two tabletop manipulation platforms) and one simulated humanoid, using human demonstrations as an additional pretraining source. Across in-distribution and out-of-distribution settings, cross-embodiment pretraining improves downstream manipulation, broader embodiment mixtures improve transfer, and embodiment dreaming provides additional gains on contact-rich tool-use tasks. Anonymous project website: egox-embodiment.github.io.