- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents EGO2ANY, a reference-conditioned video diffusion model that retargets an egocentric manipulation video to another robot embodiment. Given the source video, its end-effector trajectory, camera intrinsics, and action-independent RGB references of the target robot, the model predicts a target-body layout over time and uses that layout to condition video synthesis. At inference it does not require target meshes, kinematic models, or a prepared target initial frame. Training pairs are obtained by retargeting source actions onto robot assets and compositing the renderings into the source scene. The resulting generator is used to build EGOXPAIR, with 1.0M retained videos from 100K source trajectories across 10 target embodiments. Experiments evaluate human-to-robot and robot-to-robot generation with a custom semantic score, and the effect of the generated videos on StarVLA policies in SIMPLER and LIBERO.
Critical strengths
- A shared inference interface, with the layout component isolated in a controlled study. EGO2ANY separates the source interaction from the target morphology. Morphology-conditioned temporal transport predicts the target-body layout from the source action and multi-view RGB references, and the video backbone is conditioned on that layout. Changing the references changes the embodiment within one model, without target meshes, kinematics, or a prepared initial frame at inference. Table 5 compares this design with LingBot-Control under the same training setup. Action conditioning alone moves embodiment from 15.2 to 17.3 and H2RCore from 21.4 to 23.6. Adding frame-wise body layout raises embodiment from 22.6 to 34.2 and H2RCore from 28.3 to 37.8; temporal transport and surface support bring the full model to 45.4. These results support a benefit from predicting the target-body layout, rather than an effect of adding an action channel alone.
Critical weaknesses
- The headline transfer score only partially measures whether the interaction is preserved. H2RCore and R2RCore place 80% of their weight on embodiment (Appendix A.3). An output with zero embodiment and perfect scene, action, goal, and temporal scores receives at most 20. The reported human-to-robot gain, from 21.8 to 45.4, is therefore carried mainly by embodiment: 16.7 for MiniMax H3 against 43.1 for EGO2ANY. On the interaction domains, VACE scores 95.0, 86.8, and 92.9 on action, goal, and scene, against 46.7, 44.4, and 71.8 for EGO2ANY. On robot-to-robot transfer, EGO2ANY raises embodiment from 9.4 to 50.0 over LingBot-Control, while scene falls from 80.7 to 72.6 and goal transition from 84.3 to 71.8. The domains are scored by Gemini 3.5 Flash, Qwen3.7-Plus, and GPT-6 Astra, with no reported agreement against human raters. A rendered geometric teacher is already available, but the main results do not report mask overlap or end-effector reprojection error on held-out composites. Those measurements would show whether the embodiment gain preserves the manipulation, which the weighted core does not establish by itself.
- The reported policy gains are not isolated from the amount of added data.
The abstract and Figure 1 highlight gains of 13.4 and 12.7 points on SIMPLER. Those comparisons add the full EGOXPAIR corpus, 1.0M videos from 100K trajectories, to benchmark-only training. In the matched setting, which regenerates the benchmark’s own demonstrations, StarVLA-π moves from 58.7 to 64.5 and StarVLA-GR00T from 61.6 to 67.4. On LIBERO, both backbones improve by 1.4 points from baselines already near 95. The paired actions are the source end-effector commands. There is no comparison, at the same sample count and optimization budget, against same-embodiment repetition or against the Appendix C renderings. Without that control, the success-rate difference does not isolate the contribution of cross-embodiment generation.
Questions for authors
Transfer score: Have you computed the core with equal domain weights? On the held-out routes, what are the visible-mask IoU and TCP reprojection error against the rendered teacher? Can you also break out embodiment and action for Panda, UR5e, Rizon 4, and Unitree G1?
Downstream attribution: In the SIMPLER runs, were the optimization steps and the number of added samples matched? At the same sample count, how does matched EGO2ANY data compare with same-embodiment repetition and with the Appendix C composites?