acceptodds
Under review as a conference paper at ICLR 2027

AReaL-RSI: Adaptive Scaling and Coordination of Recursive Self-Improvement

Abstract

Scaling recursive self-improvement requires additional computation to produce useful research progress in reaching persistent objectives. Intuitively, more re- searcher agents can explore more hypotheses, but early evaluation scores might misrepresent their promise, shared findings could lose their applicability condi- tions, and useful procedures could remain confined to completed investigations. In this paper, we present AReaL-RSI, a framework for organizing a persistent research team of agents without updating backbone LLM weights. We formu- late the scaling problem in terms of evaluated outcomes and whole-system re- source expenditure. Concretely, a leader agent selects task-appropriate research orientations and guides ongoing investigations; conditional memory sharing dis- tinguishes unresolved problems from supported findings and enables recipient adaptation. During recursive evolution, after each campaign, evidence-based re- view revises researcher orientation, knowledge, and skills for later campaigns. Experiments compare AReaL-RSI with Codex-Goal on downstream harness optimization, GPU kernel optimization, and AI research tasks using two back- bones. Under the reported resource allocations, AReaL-RSI improves Terminal Bench performance by 2.2 percentage points for each backbone and increases mean KernelBench speedup by 32.2–36.8%; further scalability comparisons fa- vor the design of AReaL-RSI over native scaling: combining experience sharing with specialized researcher roles yields 1.9–27.5% lower kernel latency than in- dependent exploration and 1.3–33.9% lower latency than sharing alone across all seven comparable task–backbone pairs.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
recursive self-improvement, test-time scaling, multi-agent systems, large language models, shared agentic memory, skill evolution, autonomous research

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_xabAOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes AReaL-RSI, a multi-agent framework for agentic research that consists of a leader agent that assigns "harnesses" to a fixed team of researcher agents. AReaL-RSI routes findings between them under a conditional sharing protocol, where unresolved problems go to at most one additional researcher, while supported findings are shared more widely. After each campaign, the leader revises the harnesses for future campaigns. The paper shows that this framework improves against Codex's goal functionality on three task families.

Critical strengths
  • The paper's evaluation spans three quite different task families and two backbones. The result points the same way in all 12 comparisons against Codex's goal, which is consistent in direction (though see the compute concern below).

  • I appreciate that the conditional sharing protocol is a concrete and competent response to a real problem: shared failures homogenising exploration. The configuration study gives some support for combining sharing with specialised roles.

Critical weaknesses
  • I'm not quite convinced that the experiments test the paper's main claims. The recursive part of the method, which is the post-campaign harness revision (Section 4), has not yet been evaluated. The paper is framed around adaptive scaling but it fixes the team size (see my question below). As it stands, I think the paper should lean more into framing itself as a competent multi-agent research system that can improve upon Codex's goal, rather than adopting a framing around RSI.

  • My other main concern is the strength of the main comparison. The paper has Codex's goal as the only external baseline, however, Codex's goal is a single agent loop. For KernelBench, Codex was also limited to one effective hour in total, whereas each AReaL-RSI researcher receives one hour, not mentioning the leader and review costs. The paper itself notes that RQ1 does not establish equal-compute efficiency -- if so, the gains cannot be separated from the extra compute.

  • Following up on the above point, the paper discusses several research optimiser systems (AIDE, AlphaEvolve, DGM, ADAS) but does not include any of them as baselines. It's unclear to me how AReaL-RSI compares with methods designed for these tasks rather than a general-purpose agent loop.

  • The paper defines the objective in terms of (tokens, model charges, experiment compute) in Eq. 1 and Eq. 13, yet no cost is reported for any run.

Questions for authors
  • Could the authors please provide the costs for each run?

  • What was the team size in RQ1? Had the RQ1 harnesses been revised by earlier campaigns?

  • Could the author include a scaling curve? What happens if you vary the size of the agent team?

  • Could the authors include one dedicated optimiser system baseline (AIDE or OpenEvolve)? I'd be happy to revise my score if the result holds against these baselines. Additionally, could you please include a few repeated runs to account for run-to-run variance -- though I'm conscious of the potential run costs.

As a suggestion, especially for the Terminal-Bench experiment, a dedicated harness optimiser such as Meta-Harness that reports Terminal-Bench 2.0 results with released code would be a more informative comparison than Codex's goal.

1 comment

Thank you for recognizing the breadth of the evaluation and the conditional sharing mechanism, and for clearly identifying the gap between the paper’s framing and its evidence. Your compute concern is well founded: allocating time separately to each researcher is not equivalent to a single agent’s total budget. The Codex-Goal comparison therefore should not be interpreted as demonstrating equal-compute efficiency. Team sizes, harness versions, and prior revision histories also need clearer reporting.

We agree that comparable-compute baselines, varying team sizes, repeated runs, and direct evaluation of cross-campaign improvement would strengthen the study. Your suggestions of AIDE, OpenEvolve, and Meta-Harness provide concrete directions for more informative comparisons. This feedback helps us define the current contribution more precisely and identify the evidence needed to support broader RSI claims.

Reviewer_5akBOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes AReaL-RSI, a framework for test-time scaling of recursive self-improvement (RSI). It organizes a leader agent and KK researcher agents and does not update model weights.

  • Within a campaign: the leader picks a researcher harness (orientation, knowledge, skills) for each researcher from a library and coordinates the investigation. Each researcher works on the complete task. Findings are shared through a conditional memory mechanism. Unresolved problems go to at most one additional recipient. Findings relabeled as positive can be routed more broadly.
  • Across campaigns: a post-campaign review revises the harnesses and publishes them to a versioned library for later campaigns.

The experiments compare AReaL-RSI with Codex-Goal on Terminal Bench 2.1, seven KernelBench tasks, and four RSI Bench tasks, using two backbones. They also compare three parallel configurations on four KernelBench tasks.

Critical strengths
  • The problem is clearly motivated. The paper names three concrete failure modes of naive parallel scaling: early scores misrepresent promise, sharing without conditions misleads or homogenizes exploration, and procedures are not reused.
  • The design choices are interpretable and well reasoned: observations are separated from interpretations, problems and positive findings are routed asymmetrically, recipients are expected to verify findings locally before adopting them, and library updates are version-checked.
Critical weaknesses
  • The cross-campaign (RSI) component is not evaluated. Harness evolution is central to the RSI framing and to Contribution 2, but its effect is not measured. For the claimed self-improvement, the key question is whether revised harnesses help on tasks that were not used to produce them. As the authors themselves note (L313–316), assessing transfer requires later campaigns with declared task splits and library versions, which would amount to evaluating revised harnesses on held-out tasks. Such an evaluation could report not only final performance but also efficiency (e.g., time or cost to reach the same target score), compared with a setting where the library is kept fixed and with review cost included. Without it, the current results mainly support coordinated multi-agent search rather than recursive self-improvement.

  • Gap between the stated problems and the presented evidence. The introduction motivates the method with specific failure modes of naive parallel scaling (e.g., homogenized exploration, abandoning promising directions, and non-reusable procedures), and each mechanism is designed to address one of them. However, the experiments report only end-to-end outcomes, so it remains unclear whether and how these failure modes actually arise and are mitigated by the proposed method. At a minimum, a quantitative or qualitative report showing how each mechanism affects the failure mode it targets would make the connection between the problem formulation and the method more convincing.

  • The scalability claims lack scaling experiments. Results are reported for a single team size (four agents in Table 2; K is not reported for Table 1), and neither the number of agents nor the budget is varied. Describing RQ2 and the abstract's comparisons as "scalability" may therefore overstate the evidence.

  • Statistical reliability is limited. No repeated runs or variance are reported, and several of the reported differences are small, which makes it difficult to judge whether they exceed run-to-run variance.

  • Baseline comparisons could be strengthened. Since the budget for KernelBench and RSI Bench is allocated per researcher agent (L341–343), AReaL-RSI likely uses substantially more total compute than a single Codex-Goal run. Including a baseline with comparable compute, as well as at least one existing multi-agent or harness-based approach, would help clarify whether the gains come from the proposed coordination itself or from additional compute and parallelism.

  • The formalism adds limited information beyond the text. Most equations restate the prose in notation rather than enabling analysis or specifying an algorithm. For example, Eq. 1 is explicitly not optimized; Eq. 2, 3, and 10 express LLM decisions as sampling from policies (π_select, π_coord, π_review) without specifying how those decisions are made; Eq. 4 is an argmax over candidates; and Eq. 8 is a generic state update. No property, guarantee, or design choice is derived from the formulation. The paper might benefit from either using the formalism to support a concrete analysis or replacing parts of it with pseudocode and concrete implementation details.

  • The paper is written at a high level of abstraction, which makes it difficult to understand and reproduce. The method is described mainly through notation and high-level descriptions, while many concrete details are missing, such as prompts, harness contents, how the initial library was constructed, the number and order of campaigns, and code. Concrete examples (e.g., an actual harness, shared experience records, and a full campaign trajectory) would greatly help readers understand how the framework operates and would make the work reproducible.

Questions for authors
  • Compute and baselines. What is the total compute (tokens, GPU-hours, wall-clock) used by each method, including leader and review costs? Could the authors report results against a baseline with comparable compute (e.g., Codex-Goal with a KK× budget, or best-of-KK independent runs), as well as an existing multi-agent or harness-based approach?

  • Harness evolution. Was cross-campaign harness evolution active in the Table 1 experiments? How many campaigns were run, in what order, and with which library versions? Could the authors show performance on held-out tasks across library versions, compared with a setting where the library is kept fixed?

  • Failure modes. Is there any evidence (quantitative or qualitative) that the failure modes described in the introduction occur in practice and are mitigated by the corresponding mechanisms, for example a measure of exploration diversity?

  • Revision quality. Since harness revisions are accepted without empirical validation and the exclusion of evaluation information is only a semantic requirement, how do the authors ensure that the library does not degrade over time or absorb information from evaluation feedback?

2 comments

Thank you for the detailed review and your thoughtful assessment of the motivation and sharing design. We agree with the central distinction you raise: implementing cross-campaign harness revision does not itself demonstrate sustained self-improvement. The current results primarily support within-task coordination. Establishing transfer requires explicit task splits and library versions, comparison with a fixed library, and accounting for revision costs. Version control alone also does not establish revision quality or prevent degradation.

Your suggestions on evidence and presentation are helpful. Beyond final scores, concrete trajectories are needed to show how the proposed failure modes arise and how sharing and roles affect exploration. Pseudocode, actual harnesses, and examples of shared records would make the method easier to understand and reproduce. We also recognize the importance of repeated runs, complete cost reporting, and stronger baseline comparisons. Thank you for identifying these gaps so clearly.

Thanks for the response. I appreciate that the authors agree with the main concerns. However, since these concerns have not yet been addressed with new results or clarifications, I will maintain my original assessment.

Reviewer_3ZxuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper they address RSI as a scalable agent problem. Running many AI researcher agents in parallel doesn’t automatically help. Early scores can mislead everyone toward a fragile idea, shared findings lose their context, and lessons aren’t carried forward to the next task. AReaL-RSI fixes this without retraining the model. A leader agent assigns specialized roles and steers the team by evidence. Findings are shared with their conditions, and unresolved problems are kept from spreading to the whole team. After each task, the agents’ knowledge and skills are updated for future use. Against the Codex-Goal baseline, it won on every metric with two different models with +2.2 points on Terminal Bench and 32–37% faster GPU kernels.

Critical strengths
  • The authors identify three reasons why adding more agents doesn’t automatically help: misleading early scores, findings that lose their context when shared, and lessons that get lost between tasks. Although these reasons are not shown in their analysis.
  • The framework provides a concrete mechanism for retaining research experience through revisions to agent orientations, knowledge, and reusable skills, without updating the underlying language-model weights.
  • The comparison of independent exploration, sharing alone, and sharing plus specialized roles shows that the combined configuration wins in all seven comparable task–backbone pairs.
Critical weaknesses
  • They present the problem of RSI as a scalable coordination problem, but they do not show how their solutions scales with more agents. It would be good to see how performance increases with the number of agents when using their method, while it plateaus with other methods.
  • Cross-campaign self-improvement is a central component of the framework, but the experiments do not demonstrate that evolving researcher harnesses improves performance on subsequent, held-out tasks.
  • The configuration comparisons do not isolate the contributions of individual mechanisms
Questions for authors
  • How does the method scale with agents? I believe this is a core aspect of your method that you do not show.
  • The comparison with the other methods also do not compare with equal compute budget. Therefore, it is not clear if the gains come from access to more compute.
  • And finally, the paper does not show which components contribute most to the gains.
2 comments

Thank you for your careful review and for recognizing the value of retaining research experience and combining sharing with specialized roles. You identify important limitations: results at a fixed team size do not establish benefits from adding agents, the configuration comparisons do not isolate each mechanism, and comparisons with a single agent cannot rule out gains from additional compute.

Our results provide initial support for the coordination design, while scaling and cross-task self-improvement require more direct validation. Your suggestions clarify the priorities for further evaluation: varying team size, comparing methods under comparable total compute, and testing evolved harnesses on tasks not used to produce the updates. We appreciate these concrete suggestions.

Glad that you appreciate my suggestions. In the light of no additional information in the rebuttal process, I will keep the score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.