acceptodds
Withdrawn from ICLR 2027

EmbodiRSI: Recursive Self-Improvement for Data-Efficient Robot Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

15 views

Abstract

Adapting robot manipulation policies to new tasks and environments remains highly data-intensive, while the data needed for further improvement depends on the policy's current capabilities and failure modes. We introduce EmbodiRSI, an agentic system for recursive self-improvement (RSI) in a real-to-sim-to-real setting, where task-specific simulations are constructed from target deployment scenarios and used as low-cost environments for iterative policy improvement before transfer back to the physical world. EmbodiRSI uses policy execution feedback to guide subsequent experience acquisition and policy updates. Two complementary mechanisms close this loop: Collaborative Error Correction generates agent-assisted corrective trajectories from policy-reached states, while Adaptive Data Collection directs expert demonstration generation toward the current policy's weaknesses. The task-specific simulation serves as a reusable workspace for policy warm-up, repeatable evaluation, failure diagnosis, and targeted data generation across successive RSI rounds. Across three tabletop environments and 14 subtasks, EmbodiRSI increases scene-balanced autonomous simulation success from 50.4% to 83.5% over two RSI updates. With 400 adaptive simulated trajectories and only ten real-world refinement trajectories per subtask, EmbodiRSI achieves 83.1% scene-balanced autonomous real-world success, compared with 75.0% for adaptation using 200 real-world demonstrations per subtask. These results demonstrate that feedback-driven recursive improvement in deployment-specific simulations can enable data-efficient adaptation of embodied policies to physical environments.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Embodied AI, Robot Manipulation, Recursive Self-Improvement, Real2Sim2Real, Policy Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_dT1DOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper addresses the high cost of adapting robot manipulation policies to new deployment environments. EmbodiRSI builds task-specific simulations from target real-world scenes and performs recursive policy improvement in a Real-to-Sim-to-Real setting. Starting from an initial policy trained with generated demonstrations, the system repeatedly rolls out the current policy, diagnoses its failures, generates corrective trajectories through Collaborative Error Correction, and collects additional demonstrations targeted at weak configurations through Adaptive Data Collection. The accumulated experience is then used to retrain the policy, and the updated policy enters the next improvement round, forming a feedback-driven self-improvement loop. After several simulation rounds, the policy is further refined with a small amount of real-world human-in-the-loop data before deployment. Across two RSI updates, the method improves scene-balanced simulation success from 50.4% to 83.5%, and achieves 83.1% real-world success with 400 simulated trajectories and only 10 real refinement trajectories per subtask.

Critical strengths
  1. The overall Real-to-Sim-to-Real pipeline is well integrated, combining scene reconstruction, trajectory generation, failure recovery, adaptive data collection, and iterative policy updates.
  2. Collaborative Error Correction generates supervision from policy-induced imperfect states rather than simply resetting the task, which helps cover states missing from nominal demonstrations.
  3. The method shows clear gains over static data collection under matched trajectory budgets and is further evaluated on real robots.
Critical weaknesses
  1. The methodological novelty appears somewhat incremental. Collaborative Error Correction and Adaptive Data Collection build on ideas closely related to prior work on learner-induced-state supervision, failure recovery, and adaptive experience collection, as discussed in the introduction. The main contribution therefore appears to lie in integrating these mechanisms into a unified R2S2R self-improvement pipeline.
  2. The related-work section discusses several closely related RSI approaches, including SOAR, RECAP, VERITAS, RoboClaw, and RISE, but the experiments do not include external RSI baselines. The main comparisons are limited to Static collection and Real-only adaptation.
  3. The two core components are not independently ablated, making it difficult to determine how much improvement comes from Collaborative Error Correction, Adaptive Data Collection, or their combination.
  4. The R0–R2 comparison is confounded by the increasing training-data budget (50/200/400 trajectories per subtask). Although the matched-budget Static comparison partially addresses this issue, it still does not isolate the individual contributions of the RSI components.
  5. The implementation of several key agents, including the scene agent, trajectory agent, and rollout monitor, is under-specified. The paper states that the overall agent system is based on GPT-5.5, but provides limited details on prompts, tool configurations, and agent-specific decision logic, which may affect reproducibility.
Questions for authors

The following issues should be addressed:

  1. It would be helpful to provide separate ablations for Collaborative Error Correction and Adaptive Data Collection to determine how much of the improvement is contributed by each component individually and whether their combination provides additional benefits.

  2. The reported improvement from R0 to R2 is accompanied by an increase in the cumulative training budget from 50 to 200 and 400 trajectories per subtask. Therefore, it would be useful to further disentangle the effect of the proposed recursive self-improvement procedure from the effect of simply using more training data. The matched-budget Static comparison partially addresses this concern, but does not fully isolate the contribution of the individual RSI mechanisms.

  3. Several key agents, including the scene agent, trajectory agent, and rollout monitor, are described only at a relatively high level. Since the overall agent system is based on GPT-5.5, it would be helpful to provide more details on the prompts, tool interfaces, model roles, and decision logic used by these agents, as these choices may significantly affect reproducibility.

  4. The related-work section discusses several closely related recursive self-improvement approaches, such as SOAR, RECAP, VERITAS, RoboClaw, and RISE, while the experimental evaluation mainly compares EmbodiRSI with Static collection and Real-only adaptation. It would be better to include comparisons with stronger external RSI baselines.

  5. The current evaluation is limited to three custom tabletop environments constructed by the authors. It would be valuable to evaluate EmbodiRSI on standardized public benchmarks, such as RoboCasa365 or LIBERO-Pro, where applicable, to better assess generalization and facilitate comparison with prior methods. Such experiments would also help demonstrate that the observed gains are not specific to the particular reconstructed scenes and task configurations used in this study.

Reviewer_1oUROpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This submission introduces EmbodiRSI, an agentic pipeline for improving robotic manipulation policies through real2sim2real. Given real-world observations, it first invokes a real2sim workflow to reconstruct the task scene in Isaac Sim. An agentic DAgger-style pipeline (Collaborative Error Correction) and task curriculum (Adaptive Data Collection) are responsible to improve the policy performance across rounds. Finally, real-world human correction data are collected to help the policies overcome the sim2real gaps. Experiments show that the proposed pipeline is able to improve policy performance iteratively, and the data demand is mitigated to some extend.

Critical strengths
  • The paper is in generally clear written. The presentation is easy to follow.
Critical weaknesses
  • The proposed pipeline is a combination of existing components. It is hard to understand the uniqueness against existing agentic systems and real2sim2real pipelines. System-level comparison is needed.
  • The pipeline is mostly agent-orchestrated, it's hard to assess the human contributions.
  • Visual check is used for agentic rollout monitor. While in simulation you have the ground-truth states, which are more accurate than visual cues for monitoring.
  • Eq 2 the error correction seems to be derived purely from object's initial and goal poses. Can it handle motions that are more complex than simple pick-and-place?
  • The pipeline still requires real-world human-in-the-loop intervention data to close the sim2real gaps.
  • Performance improvement is marginal, and results are of high variance.
  • Claims made in this submission are not well supported by the limited experiments and simple pick-and-place tasks.
  • Failure modes are not presented.
Questions for authors

See above, and also

  • It's known that policy performance will improve from iterative processes. To quantify the gain, can you compare against baseline methods such as iterative DAgger?
  • Why the system still requires real-world human intervention data to close sim2real gaps?
  • How many rollouts are used for evaluation in both sim and real?
  • Can you demonstrate on more challenging tasks beyond pick-and-place, and more sophisticated robot embodiments beyond table-top manipulators?
Reviewer_4t1JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents EmbodiRSI, a real-to-sim-to-real system for adapting robot manipulation policies through iterative data collection and training. The system reconstructs a simulation of the target workspace and generates demonstrations to initialize a π0.5-based policy. During subsequent rounds, Collaborative Error Correction provides corrective continuations from states reached by the policy, while Adaptive Data Collection redirects demonstration generation toward configurations associated with failures. The accumulated data are used for advantage-conditioned policy training, followed by limited real-world human-in-the-loop refinement Experiments cover 14 individually evaluated subtasks across three tabletop scenes. The authors report improvements over static collection at matched retained-trajectory counts, and 83.1% scene-balanced real-world success using 400 simulated and ten real-world training trajectories per subtask, compared with 75.0% using 200 real-world demonstrations. Overall, these results suggest that feedback-driven data collection in simulation is a promising approach to improving robot adaptation with limited real-world training data.

Critical strengths
  1. The system addresses a practical gap between observing policy failures and obtaining useful training data. The combination of recovery from learner-induced states and targeted demonstration generation is well motivated. Reusing the reconstructed workspace for collection, diagnosis, and policy updates also gives the overall system a clear purpose beyond one-time simulation data generation.

  2. The experimental results provide useful evidence for the overall approach. The static and adaptive settings share the initial demonstrations, policy architecture, and training recipe. At 400 simulated trajectories per subtask, EmbodiRSI achieves 79.9% versus 59.3% simulation success across S1 and S3. The advantage remains after both policies receive ten real-world refinement trajectories per subtask: 82.1% versus 67.9%. This is a meaningful control, and it supports the argument that the collected experience matters, not just the number of retained trajectories.

Critical weaknesses
  1. The baselines do not sufficiently test the contribution beyond existing corrective-data and self-improvement methods.

IntervenGen already generates corrective experience by adapting a small set of human interventions to states reached by the current policy. PLD collects recovery behavior through hybrid rollouts and distills the resulting data back into a VLA. These methods differ from EmbodiRSI, particularly in how corrective actions are obtained, but they are relevant alternatives to static demonstration collection. The manuscript discusses related approaches, including PLD in Appendix F, but the experiments compare against static collection and real-only training. I see value in the proposed integration, yet these comparisons do not establish what its recovery design adds over simpler self-training or existing corrective-data generation. A success-only self-training baseline and a suitable adaptation of a prior recovery method would make the empirical contribution much clearer.

  1. The individual contributions of the two feedback mechanisms remain unclear.

The comparison in Table 1 changes both the use of rollout-derived corrective experience and the distribution of newly generated demonstrations. It therefore cannot distinguish the contribution of Collaborative Error Correction from that of Adaptive Data Collection. The paper acknowledges this limitation, but the coupling of these mechanisms is central to the proposed method, so I consider the missing component ablations important to the assessment. Appendix D.1 also groups autonomously successful and recovery-assisted episodes together as rollout-derived data, without reporting their separate counts. This makes it difficult to judge how much corrective experience actually enters training. Correction-only and adaptive-collection-only variants, together with this data breakdown, would help establish whether the two mechanisms provide complementary gains or whether one accounts for most of the improvement.

Questions for authors
  1. Could you isolate the two feedback mechanisms on the existing simulation tasks?

The most important comparisons for me are Correction Only, which retains recovery but keeps subsequent demonstration sampling at the original distribution, and Adaptive Collection Only, which uses targeted sampling without external recovery. Please keep the initialization, retained-trajectory count, and policy-training schedule matched, and explain how the data allocation changes when a component is removed. These results would directly address whether the proposed combination offers complementary benefits.

  1. How does the method compare with simpler self-training and prior corrective-data generation?

A useful first control would collect successful autonomous rollouts without recovery or adaptive sampling, alongside the same initial data and fixed-distribution demonstrations. Comparing this success-only self-training variant with Correction Only would help isolate the value of corrective supervision. I would also like to see a recovery baseline inspired by IntervenGen, using trajectory adaptation rather than the proposed state-conditioned replanning, with the same downstream policy learner and evaluation protocol. Please state any changes to the original method, including the source of the recovery examples.

  1. How is corrective experience used in policy training?

Appendix D.1 states that only ultimately successful episodes are retained, including episodes containing policy errors followed by corrective interventions. Could the authors report the proportion of retained rollout-derived episodes that required correction? Also, how does the advantage-conditioning procedure distinguish earlier policy mistakes from subsequent corrective actions within these episodes? A brief explanation of the labeling procedure, ideally with an example, would be helpful.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.