acceptodds
Under review as a conference paper at ICLR 2027

RoboRSI: Stable, efficient, and reusable robot self-evolution in complex real-world environments.

66 views

Abstract

A generalist robot should not only perform diverse tasks but also improve through experience, turning what it learns during execution into capabilities that later tasks can reuse. Robot agents that act through code can already repair programs from execution feedback, yet it remains a central challenge to organize this experience around the task structure that gives it meaning, so that each repair is attributed to the responsible capability, supported by execution evidence, and validated before it is reused. We introduce RoboRSI, a robot self-improvement system built on Top-Down Skill Refinement (TSR). TSR decomposes tasks into compound, atomic, and base skills with scoped responsibilities and explicit input–output contracts, attributes each execution outcome to the responsible branch, and confines revision to that branch. Building upon this structure, a Manager, Planner, Engineer, and Reviewer coordinate planning, execution, diagnosis, and the validated release of new skills, while people steer the process through objectives and corrections; stable skill sequences are further consolidated into reusable compound skills. On a mobile manipulator, RoboRSI develops multi-object household cleanup over 104 rounds. In simulation, it achieves the highest success rate on LIBERO, LIBERO-PRO, LIBERO-Plus, and RoboTwin, exceeding the strongest baseline by 2.7 to 11.0 percentage points.

Primary area:
applications to robotics, autonomy, planning
Keywords:
self-evolution, agentic robot

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_zq42OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces RoboRSI, a robot self-improvement framework designed for complex, real-world environments. RoboRSI aims to transform execution feedback into skills that can be sustainably accumulated and reused. RoboRSI achieves higher success rates than the baselines across four simulation benchmarks, and ablation studies validate the effectiveness of composite skills in enhancing success rates and reducing execution overhead.

Critical strengths

The paper features a clear overall design and offers valuable solutions to practical challenges encountered during long-term autonomous robot execution.

RoboRSI outperforms the baseline methods across all four simulation benchmarks, achieving an 11.0 percentage point increase in success rate on LIBERO-PRO.

Furthermore, The 104-round physical development study demonstrate the system's potential for practical deployment.

Critical weaknesses

Since RoboRSI simultaneously introduces post-condition verification, failure recovery, multi-agent collaboration, and online skill updates, it is difficult to determine whether the performance gains stem from TSR or from these other mechanisms.

Skill repair relies primarily on offline replay-based verification, lacking repeated testing and regression rate analysis after the skill library has been frozen.

Questions for authors

Could you include more rigorous ablation studies to clearly demonstrate the independent contribution of TSR? Specifically, please compare the complete RoboRSI system against versions that retain only the post-condition verification and recovery mechanisms, as well as a version that employs Flat Revision for online skill updates.

How should versioning, conflicts, and shared dependencies be managed as the skill library grows? How should the impact scope of a single revision be estimated and how does the Manager’s review cost change with library size?

2 comments

Thank you for the positive assessment of the structure, the practical motivation and the physical experiments, and for the concrete ablation design.

W1/Q1: Separating TSR from the other mechanisms.

We ran exactly the requested ablation on LIBERO-PRO under the protocol of Table 1. Variant (b) keeps postcondition verification and recovery but never revises skills; variant (c) adds online skill updates with flat revision; (a) is the full system with TSR. Success (%) (new Appendix K):

VariantCheck + recoveryOnline skill updateRevisionSuccess
OpenETA✗✗—38.5
(b) Check + recovery only✓✗—39.0
(c) Flat revision✓✓flat35.8
(a) Full RoboRSI✓✓TSR49.5

Postcondition checking and recovery alone add 0.5 pp over OpenETA, and online updates with flat revision do not improve on them. With TSR, the same online updates raise success to 49.5%, +10.5 pp over (b) and +13.7 pp over (c). A revision-level comparison on 24 failed tasks, with each library evaluated frozen on a held-out seed, gives the same ordering: TSR 11/24, flat revision 8/24, starting library 7/24 (our response to mGzr, W2).

W2: Frozen re-testing and regression.

The LIBERO library is evaluated frozen on the 840 perturbed LIBERO-Plus instances (Table 1: 42.1% vs. 36.4% for OpenETA), and each arm of the 24-task TSR comparison is evaluated frozen on a held-out seed (TSR 11/24 vs. 7/24 for the starting library). Regression is guarded at release time: a revision is released only after it passes replay on every recorded task whose call graph reaches the revised scope (the replay set HtH_t), so a change that breaks a previously solved task is not released. On the physical robot, validation replays the most recent runs (Appendix F.5).

Q2: Versioning, conflicts, dependencies and review cost as the library grows.

  • Versioning and conflicts. Each candidate is staged in an isolated copy of the library and released only after the release gate passes; the Manager maintains skill versions (Section 3.3).
  • Shared dependencies and impact scope. A revision may change only the nodes in its scope StS_t; if an interface must change, the scope is first expanded to its callers (Eq. 2). The impact of a revision is the set of tasks whose call graph reaches StS_t, which is exactly the replay set HtH_t used for validation.
  • Review cost. The Manager reviews only StS_t and its interfaces, so review cost grows with the revised scope and validation cost with HtH_t.
  • Growth in practice. In physical development, the number of skill entry points stayed at 83, while explicit behavior units grew from 112 to 118, task and recovery routes from 19 to 25, and stop gates from 3 to 11; Reviewer tokens per run fell from 3.0k to 2.1k.

Statistics. Paired over the 120 LIBERO-PRO tasks (5 seeds each), RoboRSI improves over OpenETA by +11.0 pp (95% bootstrap CI [+5.7, +16.3]; better on 54 tasks, worse on 21; Wilcoxon p = 2.2×10⁻⁴), over Maestro by +16.5 pp ([+11.7, +21.7]; p = 3.5×10⁻⁸) and over CaP-X by +31.0 pp ([+25.5, +36.5]; p = 1.7×10⁻¹⁵) (Appendix P).

Thank you again for the concrete suggestions.

Thank you for your rebuttal. I maintain my previous assessment.

Reviewer_mGzrOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces RoboRSI, a RSI system to make robot improve their skills (in code) from experience and reuse them in later tasks. The proposed key method is named Top- Down skill refinement (TSR), it organizes tasks into a skill hierarchy, traces each failure to the earliest failing skill, and revises and validates only that branch. RoboRSI shows steady skill growth over 104 rounds of real-robot household cleanup and gets the promising success rate improvments on four simulation benchmarks, compared to CAP-X, Maestro and OpenETA.

Critical strengths
  • Clear motivation, it makes sense to that execution experience becomes reusable only when each fix is tied to the task structure.
  • Clean method, TSR find the earliest failing and makes the minimal revision, as well as validates before release, which is simple and principled.
  • Comprehensive evaluation, evaluation is conducted not only on existing simulation-based benchmarks, but also the real devices. Besides that, the ablation study is helpful.
Critical weaknesses

I agree with the motivation and the main design of this work. My major concerns are about the evaluation.

  • Unfair baselines. RoboRSI keeps learning on the evaluated tasks, while the baselines do not. As a result, the evaluation shows the gain from self-improvement itself, but not RoboRSI's advantage over other RSI systems or approaches.

  • Weak support for TSR. I understand that there may not yet be a solid open-source robotic RSI system to compare against. Even so, a simpler alternative design, such as an RSI variant with flat revision, would help. Right now, the main test of TSR (Table 4) uses only one failed task and two seeds, which is not enough to support the paper's cool idea.

  • Hand-picked real-robot results. The ten runs in Figure 4 were chosen after the fact. The paper also does not report an overall success rate or explain when a run stops.

  • Missing learning cost. The paper reports the cost of running tasks, but not the cost of finding and fixing errors (diagnosis, review, and validation). Without this, the total cost cannot be compared with the baselines.

Questions for authors
  • If possible, could the authors compare RoboRSI with related robotic RSI systems (e.g., ASPIRE, ENPIRE, or EmbodiSkill), or with a simple RSI variant built on one of the baselines?

  • Because more than 2/3 failures of baselines are due to the agent declares a task done without checking the result. What happens if a simple confirmation step from a fresh observation is added to a baseline such as OpenETA? This would show how much RoboRSI gains beyond this simple fix.

  • Could the authors repeat the TSR vs. flat revision comparison (Table 4) on more failed tasks?

  • What is the full trace across all 104 real-robot runs? Which key revisions actually drove the improvement (with skill contents as the demonstrations)?

  • What is the total token and time cost of diagnosis, review, and validation, either on the simulation benchmarks OR over the 104 real-robot runs?

  • The Reviewer component (in Multi-agent System) works as an LLM-as-a-judge. How accurate are its judgments, for example its pass/fail calls and its choice of the earliest failing skill, compared with simulator labels or human labels? Under what conditions a human should be involved?

5 comments
AuthorsOpenReview ↗

Response to Reviewer mGzr (Part 1/3): Self-Improving Baselines and a Confirmation Step

We sincerely thank the reviewer for the detailed review, and for recognizing the clear motivation, the simple and principled design of TSR, and the comprehensive evaluation. We are glad that the reviewer agrees with the motivation and the main design; the concerns about the evaluation were concrete and very helpful, and we addressed each of them with new experiments. All new results are in the revised PDF (changes in blue).

W1 / Q1: "If possible, could the authors compare RoboRSI with related robotic RSI systems (e.g., ASPIRE, ENPIRE, or EmbodiSkill), or with a simple RSI variant built on one of the baselines?"

We added three self-improving baselines under exactly RoboRSI's protocol: the same backbone for every role, including reflection and repair; the same 120 LIBERO-PRO tasks in the same order; 80 tool calls per episode; the same simulator verdict; and online learning, in which whatever a method releases is reused by later episodes. Reflection sees the public evidence available to RoboRSI's Reviewer (instruction, tool trace, camera views) plus the final outcome.

  • ASPIRE (Lu et al., 2026), ported from its official code onto the CaP-X tool surface. Within an episode it iteratively debugs full programs; after the episode, an independent verifier must re-solve the task on 2 of 3 development seeds before a revision is merged. It runs serially, so each verified repair serves the following episodes.
  • OpenETA + flat reflection: after each failure, the model rewrites a persistent skill-notes store that is injected into later episodes.
  • OpenETA + EmbodiSkill reflection (Ju et al., 2026): the model classifies each failure as wrong skill content, a skill not followed, or a missing skill, and only the matching edit is applied.

Table A. Success (%) on LIBERO-PRO (new Appendix I).

MethodGoalObjectSpatialAll
CaP-X17.513.524.518.5
OpenETA29.036.550.038.5
OpenETA + flat reflection35.035.047.539.2
OpenETA + EmbodiSkill reflection30.035.047.537.5
ASPIRE (on CaP-X)32.552.550.045.0
RoboRSI37.054.557.049.5

ASPIRE is the strongest baseline: it raises CaP-X from 18.5% to 45.0% and still remains below RoboRSI on every suite.

To separate learning from evaluation, we split the 120 tasks into 60 training and 60 held-out tasks, stratified by suite; each baseline learns on the training tasks and is then frozen. On the held-out tasks, OpenETA + flat reflection reaches 29.3% and OpenETA + EmbodiSkill reflection 25.9%, below OpenETA without learning on the same tasks (37.9%): their learned notes do not transfer. RoboRSI's frozen LIBERO library, in contrast, transfers to the 840 perturbed LIBERO-Plus instances (Table 1). ENPIRE (Xiao et al., 2026) improves neural policies through physical reinforcement learning on a robot fleet rather than reusable code skills, so it is discussed in Section 2.

Q2: "What happens if a simple confirmation step from a fresh observation is added to a baseline such as OpenETA? This would show how much RoboRSI gains beyond this simple fix."

Before declaring completion, OpenETA re-observes the scene from all available views and checks the goal condition with the same backbone; if the check fails, it continues within the same budget. Each episode with confirmation is paired with the episode without it on the same task and initial state.

Table B. Effect of the confirmation step on OpenETA (Appendix J).

BenchmarkPaired episodesChange in successMcNemar p
LIBERO-PRO477+2.0 pp0.45
LIBERO-Plus (7 perturbation types)280−1.0 pp0.81

Neither change is significant, and both are far smaller than RoboRSI's +11.0 pp over OpenETA.

AuthorsOpenReview ↗

Response to Reviewer mGzr (Part 2/3): Stronger Support for TSR

W2 / Q3: "Weak support for TSR. [...] a simpler alternative design, such as an RSI variant with flat revision, would help. Right now, the main test of TSR (Table 4) uses only one failed task and two seeds [...]" and "Could the authors repeat the TSR vs. flat revision comparison (Table 4) on more failed tasks?"

We thank the reviewer for this suggestion, which led to the most direct test of TSR in the revision. We now test TSR at two levels.

System level. Replacing TSR with flat revision in the full online system lowers LIBERO-PRO success from 49.5% to 35.8%, below the variant without online updates (39.0%).

Revision level. We repeated the comparison of Table 4 on 24 failed tasks instead of one. TSR and flat revision start from the same code, traces, agents, budget and release gate, and each released library is evaluated frozen on a held-out seed not used for development.

Table C. Held-out tasks solved by each frozen library (Appendix K).

LibraryTasks solved
Starting library7 / 24
Flat revision8 / 24
TSR revision11 / 24

In the original study of Table 4, TSR places its changes in the skill responsible for the failed effect (grasp_object), whereas flat revision also modifies an unrelated skill (pull_drawer). The complete records of that study are in logs/ablations/tsr_controlled_repair/ of the supplementary material: the patches of the TSR candidate and both flat candidates relative to the common starting code, the layer selection of each candidate, the release-gate results, the Manager reviews and the study protocol.

Mechanism. We also ran an earlier, independent set of experiments on the same tidy-up task with the same robot (276 runs), separate from the 104 runs of Figure 4. Comparing its runs before (Early) and after (Late) the TSR structure matured shows the mechanism.

Table D. Earlier experiments on the same task.

Early (136 runs)Late (140 runs)
Skill entry points / explicit behavior units83 / 11283 / 118
Task and recovery routes / stop gates19 / 325 / 11
Completed runs420
Inference tokens per run84.9k43.2k
Retries per run0.850.41
Runs with substantive human intervention4712

Later in development the system handled more situations explicitly, while retries, model usage and human intervention all decreased.

AuthorsOpenReview ↗

Response to Reviewer mGzr (Part 3/3): Physical Record, Learning Cost and Reviewer Accuracy

W3 / Q4: "What is the full trace across all 104 real-robot runs? Which key revisions actually drove the improvement (with skill contents as the demonstrations)?"

The new Figure 15 (Appendix L) and rebuttal/runs_104.csv in the supplementary material show all 104 runs of Figure 4 without selection: how many of the four items were placed in each run, as confirmed on site, and how each run ended. A run is complete when every object in its manifest is placed and the final-state audit passes (arm in its safe pose, gripper empty, robot docked). It stops early when a safety check fails, or when an execution failure has no controlled recovery and the Reviewer marks the run as blocked. Operator stops and hardware, service or process faults are recorded separately.

Over the 104 runs, 31.7% of items were placed and 21 runs completed, all from run 73 on. The placement rate rises from 13.8% in the first 20 runs to 81.2% in the last 20, where 16 of 20 runs completed. The improvement follows three groups of skill revisions, listed in Appendix L.1 (Table 11). Their raw patches, before/after code and SKILL.md changes are in rebuttal/key_skill_revisions/ of the supplementary material, with the UTC timestamp and changed files of every patch in index.json and checksums in SHA256.json:

  • Before run 12: local adjustment of the grasp stance after a failed grasp plan, grasp strategy chosen by object shape and deformability, and a held-object check at the safe pose.
  • Before run 55: navigation that separates local motion convergence from the trustworthiness of map localization, docking verified by power telemetry, and bounded direct-dock recovery.
  • Before run 73: a centroid-anchored grasp point with sufficient interior clearance instead of the deepest mask pixel, and retries that re-segment the object instead of reusing pixels from a previous view.

The run-73 revision has the most direct evidence: offline replay of the original failure moved the grasp point off the raised fold of the bag, and in run 73 the bag was held on the first grasp and placed.

W4 / Q5: "What is the total token and time cost of diagnosis, review, and validation, either on the simulation benchmarks OR over the 104 real-robot runs?"

We measured the complete learning cost from the earlier, independent experiments on the same task (276 runs), counting every agent interaction: diagnosis, code revision, review, replay, validation and communication with the operator.

Table E. Cost per physical run (Appendix M).

Early (136 runs)Late (140 runs)Change
Full development tokens86.6M20.4M−76.4%
Inference tokens during the run84.9k43.2k−49.1%
Reviewer tokens / calls3.0k / 1.482.1k / 0.86−30% / −42%
Fallbacks / retries0.85 / 0.850.64 / 0.41−25% / −52%

Learning is front-loaded: released revisions and compound skills are reused, so the cost per run falls as the library matures. Compound skills further reduce execution tokens by 29.4% (Section 4.4).

Q6: "How accurate are its judgments, for example its pass/fail calls and its choice of the earliest failing skill, compared with simulator labels or human labels? Under what conditions a human should be involved?"

On 459 frozen LIBERO-PRO episodes, the Reviewer's pass/fail verdict agrees with the simulator label in 73.6% of episodes and recognizes 178 of 179 successes; its errors are failures accepted as done (42.9% of failed episodes). Such an error does not enter the skill library: a failure accepted as done only means that no revision is proposed from that episode, every candidate revision must pass replay on all affected tasks before release, and compound skills are consolidated only from simulator-verified successes.

In the earlier experiments, runs needing substantive human intervention fell from 34.6% (47/136) to 8.6% (12/140), and physical or scene interventions from 30.9% to 7.1%. Humans are involved in two cases: physical resets or safety stops the robot cannot recover from, and supplying a key insight, such as the responsibility boundary of a skill, which the agent then reuses for similar skills (Appendix N).

We hope these responses address the reviewer's concerns. We would be sincerely grateful if the reviewer would take them into account in the final assessment, and we would be glad to provide any further clarification during the discussion period.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response: Summary of Revisions

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and both reviewers for their careful and constructive reviews. We are encouraged that they found the motivation clear and TSR simple and principled (mGzr), the paper well structured and aimed at a practical problem of long-horizon autonomy (zq42), the TSR design systematic and interpretable (QwRr), and the evaluation broad (mGzr, zq42, QwRr).

We have uploaded a revised PDF (changes in blue). All new simulation experiments use exactly the protocol of Table 1: the same backbone (GPT-5.6-SOL) for every role, the same tool interface, the same budget of 80 tool calls per episode, the same task panels and the same simulator verdict. The table below lists where each concern is addressed in the revised PDF and in the supplementary material.

ConcernRaised byWhere
Comparison with self-improving baselinesmGzr, QwRrAppendix I
Confirmation step added to a baselinemGzrAppendix J
Independent contribution of TSRzq42, mGzr, QwRrAppendix K; logs/ablations/tsr_controlled_repair/
Complete record of the 104 physical runsmGzr, QwRrAppendix L; rebuttal/runs_104.csv, rebuttal/key_skill_revisions/
Full learning costmGzr, QwRrAppendix M
Reviewer accuracy and human involvementmGzr, QwRrAppendix N
Thresholds and release gateQwRrAppendix O
Paired statisticsallAppendix P; logs/main_results/

The main results are:

  1. Self-improving baselines (mGzr, QwRr). Under the same online protocol on the 120 LIBERO-PRO tasks, RoboRSI (49.5%) outperforms ASPIRE ported to the same tool surface (45.0%) and OpenETA with flat (39.2%) or EmbodiSkill-style (37.5%) reflection. When the baselines learn on 60 tasks and are frozen, their learned knowledge does not transfer to 60 held-out tasks.
  2. Confirmation step (mGzr). Adding a fresh-observation confirmation step to OpenETA changes success by +2.0 pp on LIBERO-PRO and −1.0 pp on LIBERO-Plus, neither significant, against RoboRSI's +11.0 pp over OpenETA.
  3. Contribution of TSR (zq42, mGzr, QwRr). With verification and recovery but no skill revision, success is 39.0%; with online revision but without TSR, 35.8%; with TSR, 49.5%. On 24 failed tasks, the TSR-revised library solves 11/24 held-out tasks, against 8/24 for flat revision and 7/24 for the starting library.
  4. Physical record (mGzr, QwRr). All 104 development runs are reported without selection, with run-ending definitions and the key skill revisions and their code. The placement rate rises from 13.8% in the first 20 runs to 81.2% in the last 20, and all 21 completed runs follow the key revisions.
  5. Learning cost (mGzr, QwRr). In our earlier, independent experiments on the same task (276 runs, separate from the 104 runs), counting diagnosis, revision, review, replay and validation, the full development cost per run fell from 86.6M to 20.4M tokens (−76.4%) as the TSR structure matured.
  6. Reviewer accuracy and human involvement (mGzr, QwRr). The Reviewer recognizes 178 of 179 simulator-confirmed successes, and no revision is released on its verdict alone. In the earlier experiments, the share of runs needing substantive human intervention fell from 34.6% to 8.6%.
  7. Thresholds and release gate (QwRr). The number of compound-skill candidates changes gradually as each threshold varies, and the release gate admits the same tasks across a range of success-rate thresholds.
  8. Statistics (all). Paired over the 120 LIBERO-PRO tasks, RoboRSI exceeds OpenETA by +11.0 pp (95% CI [+5.7, +16.3], Wilcoxon p = 2.2×10⁻⁴); all gaps in Table 1 are significant.
  9. Complete records (all). The revised supplementary material contains the records behind these results: rebuttal/runs_104.csv (every physical run), rebuttal/key_skill_revisions/ (raw patches of the key revisions with UTC timestamps in index.json and checksums in SHA256.json), logs/main_results/ (per-task and per-instance results used for Table 1 and the paired statistics) and logs/ablations/tsr_controlled_repair/ (patches, layer selection, release-gate results and Manager reviews of the TSR study). logs/README.md maps every file to the corresponding table or figure.

We are grateful for these suggestions, which have clearly strengthened the paper, and we warmly welcome any further questions during the discussion period.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.