acceptodds
Under review as a conference paper at ICLR 2027

EgoRecovery: Acquiring Failure Recovery Ability Through Human Recovery Demonstration

Abstract

Robust embodied robots should be able to recover from failures and retry tasks in order to operate reliably in unstructured and noisy real-world environments. Achieving this capability requires training policies on data that captures recovery behaviors. However, collecting such data through robot teleoperation is difficult to scale, as it is time-consuming to induce diverse failure states, perform corrective actions, and reset the environment. This challenge is further exacerbated by the high diversity of failure modes, which demands substantially more recovery data than success demonstrations. In this work, we show that egocentric human data capturing failure recovery processes provides a scalable alternative. By efficiently arranging task-level failure configurations and recording short recovery segments, human operators can generate more than 10x as much valid recovery data per hour compared to robot teleoperation under our protocol. To address the embodiment gap between human and robot, we propose EgoRecovery, a co-training framework for learning recovery behavior, where human recovery demonstrations are aligned to a compact corrective-intent space shared with robot data, which captures the timing and magnitude of correction. Only a small number of robot recovery demonstrations are required to connect this intent to executable robot actions. At deployment, a learned recovery gate predicts when correction is needed from robot observations and activates the corrective intent only in recovery states. Experiments on real-world recovery tasks show that EgoRecovery improves task success from failure states over robot-only recovery, direct co-training with human recovery data, and direct intent-transfer baselines.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robot Learning, Imitation Learning, Egocentric Human Demonstrations, Failure Recovery

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_Er2HOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

In this work, the authors show how human demonstrations of failure recovery can effectively improve robot recovery behavior. To do so, the authors introduce EgoRecovery, a co-training framework that bridges the gap between the human and robot embodiment. The method jointly uses human and robot recovery data by using the "corrective intent" of the recovery behavior, which contains the timing and magnitude of the necessary correction. This shared target space and encoding, together with separate action decoder models, are used to successfully leverage the information and diversity represented in the human recovery data.

In summary, the paper presents two components:

  • a corrective intent policy learning setup that leverages human recovery demonstrations to improve robot recovery behavior.
  • a gated intent modulation component that decides when a recovery is necessary.

Overall, the paper presents a clear idea that addresses how to effectively use human recovery demonstrations to learn better robot recovery. The results are clear and exciting. I enjoyed reading this work and recommend accepting it.

Critical strengths
  • The paper addresses a timely problem; teleoperation data for recovery is sparse and difficult to obtain. The authors show that collecting human recovery demonstrations scales by 10x, and that their method, EgoRecovery, can successfully extract and learn from this information despite the embodiment gap.

  • In general, the paper is well structured, and the ideas are presented cleanly and intuitively. The results are convincing, and the website very nicely shows video results.

  • Using the gated intent mechanism is a clever idea to guide the supervision signal.

  • The authors show results across four tabletop manipulation tasks and two systems, Aloha dual-arm and Franka Panda arm.

Critical weaknesses

I don't see critical weaknesses. In the appendix, the tables could be better structured, and a more detailed explanation could improve understanding of the results and findings.

Questions for authors
  1. I would like to see the system in action during the closed-loop rollouts: what labels are predicted, when do they switch, etc.

  2. The results show that some robot recovery data is required. The evaluation only varies human recovery data (or both at the same time); it would be interesting to see how success rates change when only the amount of robot recovery data changes. Did the human recovery data reduce the significance of robot recovery data in any way?

  3. I dont quite understand why the authors never evaluate using all four data mixes (Robot success, Robot Recovery, Human success, Human recovery).

Reviewer_DCM7OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Strength

  • Using ego human demo as recovery data is a very interesting and under-explored idea.
  • The method provides a clear approach to cross-embodiment recovery transfer.
  • The experimental analysis is substantial and includes meaningful controls.

Weakness

  • The related works are not discussed thoroughly, e.g., two highly related works are not discussed: [1] Li, Bowen, et al. "Recover, Discover, Plan: Learning Skills and Concepts from Robot Failures." arXiv preprint arXiv:2606.18328 (2026). [2] Vats, Shivam, et al. "Recoverychaining: Learning local recovery policies for robust manipulation." 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2025.
  • The evaluation primarily establishes interpolation within staged failure families.
Critical strengths

See summary

Critical weaknesses

See summary

Questions for authors

See summary

Reviewer_yUhGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

this paper proposes egocentric human recovery demonstrations as an efficient source of robot failure-recovery data. Under a protocol, human recovery collection is 10× faster than teleoperated robots. On four real tasks across ALOHA AgileX and Franka Panda, EgoRecovery reaches 85.0% recovery SR vs. 71.2% for direct co-training and 52.5% for robot-only. While human-only is 8.8%.

Critical strengths
  1. The data problem is well identified and honestly accounted.
  2. The intent bottleneck is designed and validated as a package. All the results supported by experiments. Nothing here is guesswork.
Critical weaknesses
  1. This paper has narrow and possibly optimistic evaluation. There is no transfer experiments across failure types or tasks. I think that the paper cannot answer its own loudest claim about covering high-diversity failure modes.
  2. 10.5× is measured under their protocol and then 1/10 converts human data into robot-equivalent budget. Could another team or staging procedure reproduce that number? Downgrade the claim to "about 10× under this protocol" and add sensitivity over conversion factors.
  1. Human wrist 3D positions are reconstructed from egocentric video. How reconstruction error propagates into the DCT target is unquantified.
Questions for authors
  1. Does the intent representation transfer across tasks ? e.g., do human recoveries from tasks A/B help task C? This goes directly to the "cover diverse failures" claim.
  2. What is the Recovery Gate's false-positive rate during nominal execution, and is the effect of a false injection on nominal success measured separately?
  1. Under sensitivity analysis of the 1/10 conversion factor, does the optimal human-to-robot data ratio move substantially?
  2. How large is the wrist-trajectory reconstruction error , and how much do main results drop if segments that fail reconstruction are removed and the model retrained?
Reviewer_CXmGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a framework called EgoRecovery to scale coverage for recovery from diverse failure modes for single-task policies trained on successful robot demonstrations by leveraging scalable egocentric human recovery data along with robot recovery data. The architecture is designed to share corrective intent instead of human motion, and employs a gating head to decide when recovery is needed. Experimental results validate the framework on four real-world tabletop manipulation tasks.

Critical strengths

The paper attempts to tackle a key problem in deploying manipulation policies: recovering from failures in diverse unseen scenarios. To this end, the paper shows that egocentric human recovery data can augment robot recovery data to scale recovery supervision. By incorporating developments in scalable data collection and representation learning for human data, the paper builds a system that outperforms one with variations in dataset composition, architecture and the training objective within the framework.

Critical weaknesses

The paper does not investigate adjacent approaches that would allow us to virtually anchor the results of this work in the literature. For instance, using an egocentric method like EgoBridge as a baseline, which could also ascertain the merit of aligning intent under the same human and robot data budget.

[Does not support rejection] Reviewing the writing would be beneficial to address a few minor points: RaC at first mention has no expansion or citation, related works mentions “All of them acquire corrective supervision on the robot, so covering diverse failure states costs robot time” for some world model methods, and minor wording discrepancies.

Questions for authors

N/A

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.