- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents EgoForesight, a framework for scaling robot policy learning with noisy action annotations from egocentric human videos. It combines reliability-conditioned action corruption with privileged-future representation distillation in a joint visual-feature and action flow model. Experiments on RoboCasa and real-world dexterous manipulation and ablation settings support the practical benefits of the combined approach.
Critical strengths
- The paper is well-motivated. It addresses the important problem of learning from noisy egocentric data. Reliability-conditioned corruption and privileged-future distillation provide complementary ways to exploit broader human experience despite imperfect action labels.
- Component ablations show gains from both designs. Matched scaling comparisons and evaluations in simulation and real-world dexterous manipulation further support the method’s practical value.
Critical weaknesses
Camera-related uncertainty in teacher supervision is not fully explained. Appendix C.4 compensates action labels using estimated camera poses, while future visual features still contain viewpoint changes. It remains unclear how the teacher distinguishes camera motion from manipulation outcomes across data sources, and how residual pose-estimation errors affect its action representations.
Questions for authors
- Could you clarify whether camera motion is handled explicitly in the privileged-future teacher or learned implicitly? In particular, how are viewpoint changes in future observations and residual pose-estimation errors in action inputs accounted for when constructing distillation targets across datasets?
- The method name closely resembles “Ego-Foresight: Self-supervised Learning of Agent-Aware Representations for Improved RL (https://arxiv.org/abs/2407.01570)” (Nunes et al., ICLR 2026). Would you consider adopting a distinct name to avoid confusion with this existing work?
We sincerely thank Reviewer FUMS for your thoughtful review and insightful questions regarding camera motion handling and method naming. We have added action-to-image cross-attention visualizations in Appendix D.3 of the revised manuscript. Below, we address each of your comments and questions.
W1 & Q1: Handling Camera Motion, Viewpoint Changes, and Residual Pose-Estimation Errors in the Teacher
In large-scale, in-the-wild egocentric videos, geometric preprocessing (Appendix C.4) can only remove the dominant first-order camera-frame offset—residual camera-pose errors, hand-tracking drift, and retargeting mismatch are fundamentally unavoidable in the reconstructed labels a (Appendix C.5). Rather than assuming preprocessing yields clean labels, EgoForesight is specifically designed to tolerate and overcome these unavoidable residual errors during training:
Q2: Method Name Similarity to "Ego-Foresight"
Thank you for pointing this out. We named our method EgoForesight to directly reflect Egocentric human data and Foresight from future-aware supervision. Ego-Foresight (Nunes et al., ICLR 2026) investigates a different research domain (self-supervised agent-aware visual representation learning for RL), whereas our work focuses on scaling VLA policy pretraining with noisy egocentric human action annotations. Given the clear separation between the two domains and problem settings, we believe the likelihood of confusion in practice is small; moreover, renaming the method at this stage would require extensive changes across the manuscript, codebase, and pretrained checkpoints, so we prefer to keep EgoForesight.