acceptodds
Under review as a conference paper at ICLR 2027

RoboRSI: Evolving a guidance Strategy for Robotic Self-Improvement

24 views

Abstract

Robotic self-improvement requires using history experience to determine which system improvements are worth pursuing. Evolving the execution system alone does not establish persistent guidance for selecting and reviewing improvements across successive rounds. We introduce RoboRSI, a robotic self-improvement framework that learns and evolves an explicit system-level strategy: a continually updated set of rules for selecting and reviewing improvements to task-planning guidance and robot skills. RoboRSI learns this strategy from historical traces linking execution outcomes, improvement decisions and their effects, and validates strategy updates through the downstream improvements they induce on held-out cases. Across four robotic benchmarks, RoboRSI improves task success rate by an average of 15.4 percentage over its no-strategy counterpart. Controlled ablations show an 8.2 percentage gain from maintaining the strategy under matched access to historical experience on LIBERO-Pro, and a 10.6 percentage gain from continually updating the strategy over keeping it fixed on RoboCasa365.

Primary area:
applications to robotics, autonomy, planning
Keywords:
Robotic Agents, Self-Evolving agents, Agent Memory, Metacognition

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_o6Q7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RoboRSI studies whether a robotic system can improve the strategy that guides its own subsequent modifications. It separates the execution system, improvement strategy, and accumulated experience while keeping model parameters and execution backends fixed. An inner loop proposes and evaluates system changes; an outer loop updates the improvement strategy using matched, isolated downstream improvement branches. Experiments cover four robotic benchmarks and include controls for access to historical experience and for continued strategy updating.

Critical strengths
  1. A meaningful target for self-improvement. Separating the execution system from the strategy that selects and reviews modifications makes learning how to improve operational. Testing candidate strategies through isolated downstream improvement branches, while keeping validation traces branch-local, is a sensible design.

  2. Controls support a benefit beyond additional history. With full historical access, the LIBERO-Pro control reaches 68.4%, versus 76.6% with the explicit strategy (§3.3). Updating the strategy improves Atomic-Seen success from 57.4% to 68.0% relative to keeping it fixed (Table 1). These are consequential gains for the central strategy-learning claim.

Critical weaknesses
  1. The evaluated implementation is not clearly connected to the complete claimed mechanism. Proposal review and outcome analysis are central to the method, but Appendix E, p. 19, states that the released four-benchmark development backend does not directly dispatch PreCritic and Auditor. This does not prove their functionality was absent from the experiments. It does leave unclear which selection-and-review components generated the reported results, limiting attribution to the learned review strategy.

  2. The controlled advantage needs a traceable resource and uncertainty comparison. Every candidate strategy is validated through two branches over at least 20 decisions. Equation 3 already includes proposal/review calls, implementation, outcome analysis, and robot execution in branch cost; the unresolved issue is total strategy-generation/validation expenditure and the units used to match budgets. History matching alone does not distinguish strategy benefit from additional development computation. Independent improvement-run variability is also unclear. Figure 2's LIBERO-Pro values of 64.7/76.5 differ from §3.3's 63.6/68.4/76.6 and need a configuration-level explanation.

Questions for authors
  1. Which backend and configuration produced each main result and ablation? Could you provide the mapping from the paper's roles to the executed components, together with one complete improvement trace showing proposal review and outcome analysis?
  2. What exactly is included in the matched improvement budget? Please report total model calls/tokens and robot executions, including both strategy-validation branches, and explain the cost units. A comparison at a common total budget on the existing controlled setting would directly address the attribution concern.
  3. How many independent improvement runs produced the final systems? Please clarify the Figure 2/Figure 3 discrepancy and provide run-level variability where repeated runs exist, or qualify the conclusions if only one learned trajectory supports a comparison.
Reviewer_hMqZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

RoboRSI evolves explicit strategies for selecting and reviewing changes to robot planning guidance and skills. Four simulated benchmarks and controlled ablations show benefits from maintaining and continually updating these strategies.

Critical strengths

Strategies are validated through downstream robot improvements, with paired comparisons and regression guards. Matched-history ablations and frozen-strategy evaluations provide useful evidence beyond simply supplying richer historical information.

Critical weaknesses

Correspondence between the proposed pipeline and experiments is unclear: Appendix E states that the released benchmark backend does not directly dispatch PreCritic or Auditor, although their functions are central to Algorithm 2. Total development costs and variability across independent improvement runs are unreported, limiting assessment of resource efficiency and small baseline differences.

Questions for authors

Clarification: Which review, outcome-analysis, and strategy components were actually exercised in the reported benchmark runs? Clarification: Were comparisons matched for total development resources, including strategy validation, and were complete improvement runs independently repeated?

Reviewer_TkUEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents RoboRSI, a robotic self-improvement framework that treats the improvement strategy as a tunable object, separate from the execution system. In the inner loop, a set of explicit selection and review rules guides how the system diagnoses, proposes, reviews, and validates modifications to task-planning guidance and robot skills. In the outer loop, the strategy is updated from accumulated improvement traces. A candidate strategy is accepted only if it leads to better downstream modifications on held-out cases, under a cost-adjusted statistical criterion. Across LIBERO-Pro, Robosuite, RoboCasa365, and RoboTwin 2.0, RoboRSI improves over its no-strategy variant by 8.6 to 24.1 points. Ablations suggest that the gains come from the strategy itself rather than from richer history alone.

Critical strengths
  • Clean and well-motivated, separating the strategy from execution, together with traceable rules and retained negative evidence, makes improvement decisions easy to audit.

  • The method is principle and the evaluation is comprehensive.

  • The ablations are well designed,

Critical weaknesses
  • Baselines, the backbone models used for the external baselines are not reported, thus it is hard to judge the performance gain. What's more, Some baseline numbers also appear lower than those originally reported. For example, Harness VLA reports 82.4% on LIBERO-Pro, compared with 70.6% in this paper. It would help to clarify how the baselines were run, and ideally to include a comparison under a shared backbone.

  • Missing related work and baselines, the discussion of closely related work could be expanded, benefiting from a clearer comparison with Hyperagents, which also aims to improve how the system improves itself by making the improvement procedure editable. It would help to explain the key differences, such as RoboRSI's use of explicit natural-language rules and its downstream validation, and why this design suits robotic settings. In light of this work, the claim that prior methods do not maintain an explicit improvement strategy might also be stated more precisely.

  • Demonstration of strategy and its evolution, it would be helpful to report more details about the initial and tuned strategy, including the number of strategy updates proposed and accepted, examples of the learned rules, and the overall token and time cost. It would also help to clarify which strategy the "Fixed" setting uses.

  • Another minor concern might be that results appear to come from single runs, and the snapshot study uses a single seed. Since some margins over ASPIRE are small (around 0.8), reporting variance over multiple seeds would strengthen the conclusions.

Questions for authors

See weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.