- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduce EgoLAP, a vision-language-action pre-training framework that learns from egocentric human and robot trajectories through shared language actions and motion-level reasoning. EgoLAP transfers human experience to robot control, achieving 80.1% mean real-world task progress on an unseen robot configuration and a 2.3× gain over alternative action representations.
Critical strengths
-
Strong and consistent empirical gains. EgoLAP reaching 80.1% real-world task progress and 38.3% zero-shot simulation success. The controlled comparisons and ablations provide useful evidence for the effectiveness of language-based motion supervision.
-
A practical approach to cross-embodiment learning. Language actions provide a common interface for human and robot trajectories without requiring a learned action tokenizer or text generation during control. The method is simple to deploy and demonstrates promising transfer to unseen tasks and robot configurations.
Critical weaknesses
see questions
Questions for authors
How much of the human-to-robot transfer benefit depends on the specific human datasets and their interaction with reasoning supervision?
The ablations show that human data improves performance and that motion-level reasoning further amplifies this gain. Can the authors clarify whether this interaction is consistent across different human-data subsets, or whether it is primarily driven by MECKA, the only human-data source with motion-level reasoning annotations? This would help assess the generality of the proposed co-training approach.
What exactly is unseen in the real-world zero-shot evaluation?
The real-world evaluation uses a customized bimanual YAM platform, while the training mixture includes demonstrations from the ABC YAM platform. Could the authors clarify which aspects of the robot embodiment, camera configuration, and control interface differ between training and evaluation, and whether any data collected on the evaluation hardware was used during pretraining? This would make the scope of the claimed zero-shot cross-embodiment transfer more precise.
2 comments
Thank you for the clarifications and revisions. Explicitly limiting the human-data × reasoning interaction to the current data mixture addresses my concern about overgeneralization, although per-subset validation remains valuable future work. The clarification that zero-shot evaluation uses a new configuration of a seen arm type, with no training data from the evaluation setup, also resolves my second concern.
I remain positive about the paper and maintain my Weak Acceptance rating.
Minor correction: in Appendix C.4, “19.9 versus 8.4 points” should read “19.9 versus 9.2 points,” consistent with Table 9.
We thank the reviewer for the positive assessment and precise questions. We are glad the reviewer found the empirical gains strong and consistent, and the ablations useful. We also appreciate that the reviewer sees EgoLAP as a practical, simple-to-deploy approach to cross-embodiment learning, with no learned tokenizer and no text generation at control time.
Q1. Does the human-data × reasoning interaction depend on MECKA?
This is a fair question, and we acknowledge that our current experiments cannot fully separate the two factors. MECKA contributes 20% of each batch, out of 25% egocentric data (App. A.7). It is therefore both the only human source with reasoning labels and the majority of our human data. The existing results support the following:
We agree that a per-subset study would make the generality claim more precise, e.g., MECKA only versus Scale + Aria only. We have revised the paper accordingly. It now states explicitly that all human-side reasoning comes from MECKA, and it scopes the human-data × reasoning interaction to this setting. We list a per-subset analysis as future work in the Limitations.
Q2. What exactly is unseen in the real-world zero-shot evaluation?
No data collected on the evaluation hardware or setup was used at any stage of training. The training mixture contains YAM data from the ABC platform and from MolmoAct2 (App. A.7). Our evaluation platform uses the same arm model but a different configuration from every training embodiment:
The claim is therefore zero-shot transfer to a new configuration (arm placement, camera setup, scenes) of a seen arm type, not to a new arm. We have revised Sec. 4.1 to state this scope precisely. The shift is not trivial. Informally, during development we observed that aligning our setup more closely with the MolmoAct2 configuration improved performance. Under our shifted configuration, the baselines degraded considerably more than EgoLAP: EgoLAP reaches 56.0% full-task success, versus at most 18.0% for any other policy (Fig. 4(b)).
We thank the reviewer again and welcome further questions.