- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes WING for transferring interaction knowledge from egocentric human videos to robot policies. The method introduces an interaction-centric latent action model that explicitly separates camera motion from interaction-related motion, and further uses low-frequency DCT components of latent-action trajectories as cross-embodiment guidance. Extensive representation analyses, ablations, simulation benchmarks, and real-world experiments consistently support the proposed design.
Critical strengths
Clear and well-motivated decomposition of cross-embodiment transfer. The paper identifies two concrete sources of mismatch in transferring egocentric human videos to robot policies: contamination of latent actions by observer-induced camera motion, and differences in fine-grained temporal dynamics across embodiments. The proposed interaction-centric latent action decomposition and low-frequency spectral guidance directly target these two issues, resulting in a method that is conceptually simple, intuitive, and well aligned with the underlying problem.
Exceptionally thorough validation of the proposed mechanisms. The experimental analysis goes substantially beyond downstream performance. The paper directly evaluates whether the interaction latent becomes less sensitive to camera motion while preserving interaction semantics, and whether low-frequency latent components indeed exhibit stronger correspondence across human and robot executions. Together with careful component ablations and consistently strong downstream results, these analyses provide a particularly convincing connection between the proposed mechanisms and the final performance gains.
Critical weaknesses
None
Questions for authors
Scaling to larger and less curated egocentric datasets. The latent-action model is pretrained on approximately 200 hours selected from Ego4D, EgoDex, and EgoVerse, although the full datasets are substantially larger. Could the authors clarify how these subsets were selected? In particular, were videos with strong camera motion, tracking failures, or other difficult conditions filtered out? I would also be interested in whether the authors have explored scaling to substantially larger and less curated subsets.
3 comments
Thank you for your clarification regarding the data selection process and the scalability of WING. They fully address my minor concern. I will maintain my positive assessment.
Thanks for your inspiring feedback! We are pleased to know that you agree with our response. We sincerely thank you for your dedication and effort on our work, and your expertise will definitely motivate our future research!
Response to B5jX
We are truly grateful to the reviewer for the time and effort devoted to reviewing our paper and for the thoughtful and insightful comments!
We follow commonly adopted data selection practices to construct our 200-hour egocentric pretraining subset, consisting of approximately 95 hours from EgoVerse, 85 hours from EgoDex, and 20 hours from Ego4D. Specifically, we prioritize videos with meaningful hand–object interactions while maintaining diversity in tasks and scenes. For Ego4D, we additionally leverage the provided action annotations to identify relevant interaction segments. Our selection is driven by general interaction relevance and data quality. In particular, we do not explicitly filter videos based on camera motion magnitude.
Regarding scalability, our current study focuses on validating the proposed representation-learning mechanism under a controlled 200-hour pretraining budget, rather than conducting a systematic data-scaling study. Nevertheless, this relatively modest pretraining corpus already yields meaningful downstream benefits: with only 25% of robot demonstrations, WING achieves 96.2% success on LIBERO, compared to 92.6% for the baseline. While this result demonstrates downstream data efficiency, it highlights the effectiveness of the learned representations under a limited pretraining budget. We agree that extending WING to larger and less curated egocentric datasets would provide a valuable assessment of its scalability and robustness, which we leave for future work.