acceptodds
Under review as a conference paper at ICLR 2027

Looped-RSI: When the Whole is Greater Than The Sum of Its parts

Abstract

Recursive self-improvement (RSI) increasingly relies on branching search over populations of self-modified agents. We identify two structural limitations of this paradigm. First, useful progress can become fragmented across lineages: a population may accumulate complementary discoveries that are not jointly realized by any individual agent. Second, task capability and self-improvement capability are inherited together even though they require different forms of feedback. We introduce Looped-RSI, which augments branching self-improvement with periodic population-level consolidation. Rather than selecting only among existing agents, Looped-RSI reasons over the evidence accumulated across the population to construct a successor, while separately consolidating the improver according to the quality of the descendants it produces. Across interactive environments, Looped-RSI produces stronger self-improvement trajectories. We further show that consolidation can recover substantial progress from populations in which every individual agent receives zero reward, and that the resulting improvers transfer effectively to previously unseen environments. These results suggest that recursive progress need not reside in a single lineage or a single agent state, but can emerge across the search process and be made heritable through consolidation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
RSI, Recursive Self-improving AI, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2j18OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates recursive self-improvement (RSI) and identifies two limitations of existing population-based approaches: fragmented knowledge across different agent lineages and the coupling of task performance with self-improvement capability. To address these issues, the authors propose Looped-RSI, a framework that periodically consolidates knowledge from multiple agents, including unsuccessful ones, to construct stronger task agents. Experiments on BabyAI and Crafter demonstrate that Looped-RSI outperforms HyperAgents, achieving better task performance with fewer evolutionary expansions.

Critical strengths
  1. This paper effectively integrates complementary knowledge and valuable experiences from unsuccessful agents.

  2. The method separates task-solving capability from self-improvement capability, and comprehensive experiments demonstrate improved performance, more effective recursive improvement, and strong cross-environment generalization.

Critical weaknesses
  1. How do the authors determine which pieces of knowledge acquired by each agent are useful? This seems to be a prerequisite for the proposed consolidation process. From my understanding, the useful knowledge contributed by each agent is identified based on real-time feedback from the environment (e.g., Table 3 in Appendix A.3). However, how would the proposed method work in environments where such feedback is unavailable, such as the Paper Review and IMO-GradingBench benchmarks used in the baseline paper?

  2. The paper evaluates the proposed method on only two benchmarks, BabyAI and Crafter, which limits the comprehensiveness of the experimental evaluation. Moreover, since the proposed method builds upon HyperAgents, the authors should at least conduct comparisons on some of the benchmarks used in the original HyperAgents paper. In particular, I suggest evaluating the proposed method on Paper Review and IMO-GradingBench to investigate how much improvement it can achieve over the baseline in these more challenging settings.

Questions for authors

See the weakness

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for the constructive feedback. Your questions point to an important distinction that was not sufficiently clear in the original presentation: Looped-RSI does not require interactive environment feedback such as valid/invalid actions. What it requires is evaluation evidence about the agents generated during self-improvement. Following your suggestion, we also added experiments on the non-interactive Paper Review and IMO-GradingBench settings used by HyperAgent.

Q1. How is useful knowledge identified when real-time environment feedback is unavailable, as in Paper Review or IMO-GradingBench?

The evidence used by consolidation does not need to come from real-time environment interaction. Looped-RSI uses the same general type of signal required by the underlying self-improvement process: what changed in an agent and what happened when that agent was evaluated. The precise evidence naturally depends on the task. In BabyAI, evaluation can provide task success together with behavioral evidence such as rejected actions. In Paper Review, there is no action interface or valid/invalid feedback; self-improvement instead operates on the agent's reasoning and decision procedure. The consolidator reasons over the population's modifications, measured task outcomes, and self-improvement traces, and synthesizes changes supported by this evidence. It does not require a predefined notion of what the useful capability should be.

Paper Review provides a particularly clean demonstration. Looped-RSI and HyperAgent share exactly the same first 15 generations, and all 15 obtain zero validation accuracy. At the first consolidation boundary, task-agent consolidation is rejected, but improver consolidation is accepted. The consolidated improver then changes the subsequent trajectory: Looped-RSI reaches 68% validation accuracy by generation 20 and 90% by generation 29, while HyperAgent remains at zero through generation 30 and eventually reaches 84%. Thus, useful self-improvement can be recovered without action-level feedback, even when the current population contains no positive task-level reward.

The key requirement is therefore evaluation feedback, not interactive feedback. If there were no signal at all for evaluating generated agents, neither Looped-RSI nor a selection-based method such as HyperAgent would have a basis for identifying useful changes. This is a requirement of the underlying RSI setting rather than an additional assumption introduced by consolidation.

Q2. Does Looped-RSI also improve over HyperAgent on the non-interactive benchmarks used in the original HyperAgent paper?

Yes. Following your suggestion, we added both datasets. For Paper Review, we use the same task formulation, prompt, and evaluation protocol as HyperAgent. The first 15 generations are shared.

Paper ReviewLooped-RSIHyperAgent
Evolution budget60 generations100 generations
Total LLM calls13,649129,582
Best validation accuracy0.900.84
Held-out test accuracy0.884 ± 0.0220.802 ± 0.008

Looped-RSI therefore obtains higher validation and held-out test performance while using substantially less total inference. Importantly for Q1, this improvement occurs in a setting with no interactive environment or action-level feedback. For IMO-GradingBench, we also follow HyperAgent's transfer protocol rather than performing a new Looped-RSI consolidation run on the target task. The source-domain improver is frozen and used to improve the target-domain agent:

IMO-GradingBench transferLooped-RSI improverHyperAgent improver
imp@2053.0%51.0%

The improvement on IMO is modest, and we do not overinterpret it. It provides complementary evidence that the improvement procedure learned by Looped-RSI transfers to a substantially different non-interactive structured-reasoning task.

We originally emphasized BabyAI and Crafter because our central claim concerns fragmented progress across branches, which is particularly exposed in open-ended settings where useful interfaces, behaviors, strategies, and failure information may be discovered separately during search. However, the new Paper Review/IMO-GradingBench results shows that the benefit is not restricted to interactive control. Beyond the two benchmarks, we also broadened the evaluation across model families and environments. With Gemini, Looped-RSI reaches 49.1% versus 9.8% for HyperAgent on Crafter (5.0× higher) and 41.7% versus 4.2% on MiniHack (9.9× higher), while using lower total API cost in both cases.

Together, the revised evaluation now spans interactive and non-interactive tasks, open-ended and structured settings, direct self-improvement and transfer, and both open- and closed-source model families. We have revised the manuscript accordingly to make clear that Looped-RSI requires empirical evaluation evidence, not real-time or action-level environment feedback.

Dear Reviewer 2j18,

Just a gentle reminder as the discussion deadline is approaching. We hope our response and the additional clarifications&expirements addressed your concerns, and we would be grateful if you could reconsider your assessment in light of our revised response. We would also greatly appreciate any further feedback you may have before the deadline. Thank you again for your time and consideration.

Reviewer_G9a2OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper augments branching recursive self-improvement with periodic population-level "consolidation," which merges partial progress scattered across lineages and separates credit assignment for task agents vs. improvers. Experiments on BabyAI and Crafter compare Looped-RSI against a HyperAgent baseline, including a transfer study to BabaIsAI and Crafter.

Critical strengths

Fragmented progress across evolutionary lineages is a genuine problem in population-based RSI, and the paper targets it directly.

Critical weaknesses
  1. The baseline is implausible. HyperAgent scores exactly 0.0% on Crafter transfer across all 20 expansions (Table 2, Figure 4). A baseline that produces 0% for 20 consecutive attempts on a new domain is more likely broken or unfairly initialized than evidence of the method's superiority. Even completely without considering any of other AutoResearch's stronger baselines.
  2. The central claim is circular. The showcase all-zero population recovery (Figure 1, Appendix A) simply merges two nodes with complementary bugs: Node 4 has valid actions but no state awareness, while Node 12 has state awareness but invalid actions. Combining them is elementary debugging rather than population-level inference discovering latent structure, since the two nodes already contain the full solution, merely separated.
  3. Consolidation is not robust. Appendix A.1 reports 10 consolidation attempts from the same population yielding a bimodal split of 3 failures (≤4%) and 7 successes (≥76%). The paper does not state which consolidation was used in the main run or how failures were handled, and selecting the best would amount to p-hacking.
  4. The budget comparison is misleading. The paper claims fewer evolutionary expansions but actually gave HyperAgent more expansions (55 vs. 45 on BabyAI) to offset consolidation compute. Efficiency cannot be claimed while handing the baseline extra budget.
  5. The consolidation prompt functions as an oracle. The two-page prompt (Appendix D.3–D.4) encodes detailed, author-written domain expertise about inspecting code and merging changes. This is instruction-following rather than a learned algorithm, and it would need rewriting for each new domain.
  6. Domain leakage contradicts the transfer story. Appendix B.3 admits that Crafter action names leaked into the supposedly domain-general improver ("Noop appears three times"), which directly conflicts with the transfer claims.
Questions for authors

See Weakness.

6 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/4)

Thank you for the review. We agree that the original presentation did not make several distinctions sufficiently explicit. We address below all the Reviewer's concerns and have revisited several parts of the paper to improve the flow and clarify potential misunderstandings.

Q1. HyperAgent scores 0% across all 20 Crafter transfer expansions. Is the baseline broken or unfairly initialized?

Here we would like to clarify that the HyperAgent baseline was not broken. We inspected all 20 transfer expansions directly, and the HyperAgent baseline is executing normally.

OutcomeHyperAgentLooped-RSI
Evaluated descendants1519
Evaluation crashes00
Passed staged evaluation0 / 1514 / 19
Descendants with 100% invalid actions12 / 150 / 19
Best Crafter score0.0%19.09%

Both methods start from the same unseen Crafter task agent, use the same evaluator, admissibility policy, and self-improvement budget, and no population-level consolidation is performed during transfer. The only relevant difference is the improver evolved on BabyAI.

The HyperAgent improver does generate executable descendants: 15 are successfully evaluated and none crashes. Its failure is behavioral. In 12/15 evaluated descendants, every emitted action is invalid, so none passes staged evaluation. By contrast, all 19 evaluated descendants generated from the Looped-RSI improver have 0% invalid actions, and 14/19 pass the staged evaluation.

This is why the transfer experiment is useful for isolating improvability. The BabyAI task agents are discarded, and the two evolved improvers are placed on exactly the same target-domain task agent. The separation therefore cannot come from one method transferring a stronger source-domain task agent; it reflects a difference in the inherited mechanism for generating future agents.

This result directly targets the within-node credit-coupling problem identified in the paper: strong current task performance does not necessarily imply a strong improver. Looped-RSI therefore evaluates and consolidates the improver separately according to the descendants it produces. We added these transfer diagnostics in Appendix C.12 to make clear that HyperAgent's zero trajectory is a genuine self-improvement failure rather than an execution, evaluation, or initialization failure.

AuthorsOpenReview ↗

Official Comment by Authors (2/4)

Q2. Is the all-zero recovery example merely “elementary debugging,” making the central claim circular?

The BabyAI example is intentionally mechanistic: the individual fixes are easy to interpret once isolated, which allows us to trace exactly where useful progress resides in the population and how consolidation makes it jointly heritable. This example directly illustrates the paper's across-node fragmented-progress problem, but the successful consolidation is more than simply combining two complementary bugs. The central claim is not that each local fix is intrinsically difficult, but that useful partial progress can be distributed across individually unsuccessful branches without being jointly instantiated by ordinary lineage-based search.

The useful evidence is distributed across several unsuccessful branches. Node 4 discovers the valid BabyAI action vocabulary and action extraction/normalization, but largely fails to condition on the observation. Node 12 incorporates the current observation and interaction history, but emits an incompatible action vocabulary. Population-wide failures additionally reveal systematic vocabulary mismatches. Successful consolidation combines these complementary mechanisms and synthesizes additional translation, fallback parsing, and validation logic. The resulting 76%-80% agent therefore contains a capability combination that no individual node in the population instantiated.

Crucially, HyperAgent already has access to the same population history, yet it does not instantiate this combination at the all-zero boundary. Starting from the identical 15-node population, ordinary branching first reaches comparable performance only after further exploration (76% at generation 20), whereas our post-hoc analysis shows that consolidation of the frozen population itself, without generating any additional descendants, recovers 76-80% in 7/10 trials. Thus, the result is not simply that the relevant fixes become obvious once all nodes are available; the distinction is whether progress distributed across those nodes can be made jointly inheritable.

Our new ablation isolates this further:

MethodBest score within 45 expansions
Restart from best existing node, no consolidation0%
Consolidate, but append synthesized state to existing tree80%
Full Looped-RSI100%

Restarting from the strongest existing lineage does not recover the distributed progress within the same budget. Population-level synthesis does, and making the synthesized state the inherited starting point for subsequent search improves further from 80% to 100%.

The phenomenon is also not specific to elementary action-interface debugging. In our new Gemini experiments (Figure 2), all 15 shared first-round agents score zero on both Crafter and MiniHack, yet consolidation of those same populations immediately produces task agents scoring 45.0% and 29.2%, while HyperAgent reaches best scores of only 9.8% and 4.2%, respectively. On Paper Review, where there is no action interface or environment mechanics to debug, the shared first 15 generations also all score zero; after improver consolidation, Looped-RSI reaches 90% validation accuracy by generation 29.

Thus, the BabyAI case is useful because the distributed pieces can be inspected explicitly; the broader claim is that useful progress can already exist across unsuccessful branches before any individual lineage expresses it, and consolidation makes that progress heritable.

AuthorsOpenReview ↗

Official Comment by Authors (3/4)

Q3. Is consolidation unreliable, and was the successful BabyAI result selected from the 7/10 successful samples?

No cherry-picking occurred. The 10 consolidation samples in Appendix A were generated only as a post-hoc robustness analysis after freezing the population from the main run. They were never candidates from which the online Looped-RSI trajectory selected a result.

In the actual BabyAI trajectory, task consolidation was performed once at the first boundary and produced a task agent scoring only 4%, not one of the 76%-80% post-hoc samples. The 7/10 result instead measures the stochasticity of one direct task-synthesis operation: using exactly the same frozen population, 7 of 10 independent samples recover at least 76%, while 3 do not. We therefore do not interpret 7/10 as a universal success probability for consolidation.

More importantly, Looped-RSI does not require every task-agent consolidation to succeed. Task-agent and improver consolidation are performed separately. In the actual BabyAI run, although task consolidation reached only 4%, the independently consolidated improver was accepted, and its very next self-improvement expansion reached 80%. Thus, a weak task synthesis does not imply that the consolidation round failed: the same population may contain useful evidence about how the self-improvement mechanism itself should change.

Nor does the next round simply resample the same merge from the same evidence. The accepted improver generates a new population, so subsequent consolidation operates on newly generated evidence. This is also visible on Qwen Crafter: after consolidation, roughly 60%60\% of Looped-RSI expansions produce scoring descendants across several subsequent windows, and, holding the task-agent state fixed, consolidated improvers produce children scoring 6.4 vs. 2.7 and 5.5 vs. 0.9 relative to the current improver.

Finally, the new Gemini experiments show that successful recovery is not specific to the favorable BabyAI post-hoc samples. On both Crafter and MiniHack, all 15 agents in the shared first-round population score zero, yet the actual online task consolidations directly produce agents scoring 45.0% and 29.2%, respectively.

The 7/10 analysis therefore measures the stochastic reliability of one task-synthesis operation on one frozen population, not the reliability of the complete recursive Looped-RSI process.

Q4. Is the compute comparison misleading because Looped-RSI uses additional consolidation compute?

The performance difference is not explained by additional compute. We now provide direct compute accounting rather than relying on the number of evolutionary expansions.

On the Qwen BabyAI experiments:

Qwen BabyAILooped-RSIHyperAgent
Total LLM calls~25.8K~25.5K
Evaluation-normalized budget51.451.0
Best score100%84%

The total budgets are therefore essentially matched, while Looped-RSI reaches substantially higher performance.

The synthesis itself is also inexpensive. The consolidation synthesis sessions account for only 211 calls, approximately 0.8% of Looped-RSI's total BabyAI LLM calls. Most consolidation overhead comes from evaluating candidate task agents and probing candidate versus current improvers to verify that a proposed update is actually beneficial.

We additionally repeated the comparison using Gemini, where actual API expenditure can be measured directly:

Gemini settingLooped-RSIHyperAgent
Crafter49.1% @ $5369.8% @ $611
MiniHack41.7% @ $580.54.2% @ $793

Thus, on both Gemini environments, Looped-RSI reaches substantially higher performance while using less total API cost.

The separation also occurs immediately after the shared initial search. On both environments, the first 15 generations are identical between methods and all score zero. Consolidation of those same populations directly produces a 45.0% Crafter agent and a 29.2% MiniHack agent, while HyperAgent continues ordinary branching and remains far below these scores.

We have therefore replaced expansion-count-based efficiency claims with explicit compute and API-cost comparisons in the manuscript.

Reviewer_qPFZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Looped-RSI, a method that improves both task agents and the agents responsible for creating better agents. It combines useful changes, success and failure cases from different agents to build a stronger task agent, while separately improving the process used to create new agents.

Critical strengths
  • This paper provides a useful idea for self-improvement. Different agents may learn different useful things, even when they perform poorly on their own. Combining these ideas while separately improving the agent-generation process is a promising direction.

  • A helpful case study. In Appendix A, consolidation raises performance from zero to 0.76–0.80 in several trials. The authors also show examples of useful mechanisms being combined, which helps explain why consolidation can work.

Critical weaknesses
  • The paper does not explain clearly how capabilities are identified and combined. Agents may have overlapping abilities or changes that conflict with each other, but the method does not systematically detect these cases or explain how to handle them. It also does not study how agent size relates to capability, or whether assigning tasks of different difficulty based on agent ability would improve evolution.

  • The gains may partly come from sharing more information and using more compute. The paper does not clearly separate the benefit of its specific method from the general benefit of giving an LLM more agents’ code, experiences, and failure cases. Since consolidation also adds inference and evaluation costs, comparisons under the same total compute budget are needed.

  • Limited analysis of evaluation reliability. The evolution process relies on task scores and measures of child quality to decide which changes to keep. Although the paper includes held-out evaluations, it remains unclear whether these signals reliably reflect genuine improvements over multiple rounds or whether the process can overfit to the evaluation setup.

  • Limited evaluation across model sizes. The experiments use only one model backbone (Qwen3.6-35B-A3B), so it is unclear whether Looped-RSI also works well with smaller or larger models. Model size may affect the ability to combine knowledge across agents and generate useful improvements, but this is not investigated.

  • It is unclear how reliable consolidation really is. In Appendix A, 7 out of 10 trials reach at least 0.76, but the other 3 score only 0–0.04. The paper explains some failures through lost observation-processing logic, but does not show whether later rounds can recover from these failures or how often consolidation succeeds in general.

Questions for authors
  1. How reliable is consolidation? Appendix A shows a clear gap between successful and failed trials. Is this pattern common, and can later rounds recover from failed consolidations?

  2. How does the method identify and combine useful capabilities? How does it tell whether a capability actually works, avoid keeping duplicate mechanisms, and handle changes that conflict with each other?

  3. Could agent capabilities guide the evolution process? Does agent size relate to its capabilities or performance? Could assigning easier or harder tasks based on each agent’s ability improve the results?

  4. How much of the improvement comes from the proposed method itself? How does Looped-RSI compare with a simpler approach that gives all agents’ code and experiences to an LLM, when both methods use the same total compute budget?

4 comments
AuthorsOpenReview ↗

Official Comment by Authors (1/3)

We thank the reviewer for the thoughtful questions. In the revision, we clarify the consolidation mechanism and strengthen the empirical analysis with new ablations, matched-compute comparisons, and additional experiments across model families and domains.

Q1. How reliable is consolidation? Appendix A shows a gap between successful and failed trials. Is this pattern common, and can later rounds recover from failed consolidations?

We agree that a single LLM consolidation proposal is stochastic. Looped-RSI does not assume that every consolidation will successfully recover all useful information in the population; instead, robustness comes from proposal-and-test acceptance, separate task/improver consolidation, and repeated rounds of exploration and consolidation. To quantify the stochasticity directly, we froze the same first-round BabyAI population, in which all 15 agents score zero, and independently reran task consolidation 10 times. 7/107/10 runs recover at least 76% performance. Importantly, this is a post-hoc robustness analysis, not best-of-1010 selection. In the actual online trajectory, task consolidation was executed only once and produced only 4%. We therefore do not interpret 7/10 as a universal success probability; it measures the recoverability of useful information from this particular frozen population.

The online run also illustrates why Looped-RSI does not depend on successful task synthesis at every boundary. Although the first BabyAI task consolidation reaches only 4%, the independently consolidated improver is accepted, and its very next descendant reaches 80%80\%. Thus, a weak task-agent consolidation does not imply that the round has failed: the same population may contain useful evidence about how the self-improvement mechanism should change, even when that evidence is not successfully synthesized into the current task agent.

To further show the robustness of Looped-RSI, we now include new results with Gemini on Crafter and MiniHack. In both cases, the shared first 15-node population contains only zero-reward agents, yet task consolidation itself directly produces agents scoring 45.0% and 29.2%, respectively (Figure 2). In contrast, HyperAgent continues ordinary branching from the same initial populations and reaches best scores of only 9.8% on Crafter and 4.2% on MiniHack. On Paper Review (Appendix E.1), using the original HyperAgent prompt and protocol, task consolidation is rejected after 15 zero-scoring generations, but improver consolidation is accepted; Looped-RSI subsequently reaches 0.90 validation accuracy by generation 29. Thus, robustness does not require every task consolidation to succeed: useful population evidence can be recovered either directly in the task agent or indirectly by improving the mechanism that generates the next population. We therefore view consolidation reliability as depending on the evidence available in the population, the difficulty of synthesizing it, and the capability of the consolidation model, rather than as a fixed success probability. Later rounds operate on populations generated by the updated improver and can therefore contain qualitatively different and potentially richer evidence. We have revised Appendix A and the main text to make the distinction between one-shot synthesis reliability and system-level robustness explicit.

AuthorsOpenReview ↗

Official Comment by Authors (2/3)

Q2. How does the method identify and combine useful capabilities? How does it know whether a capability works, avoid duplicates, and handle conflicts?

Our post-factum analysis of the consolidation is done in the appendix A. We agree that the consolidation mechanism may be not sufficiently clear in the original presentation of Section 2.1. We clarify that consolidation is not a deterministic patch merge and does not rely on predefined capability labels. It is an agentic synthesis step. As detailed in Section 2.1 and Appendix C.3–C.4, the consolidator receives a working copy of the reference agent together with evidence from all nodes in the round: their code changes and ancestry, aggregate and per-item evaluation outcomes, behavioral traces, and the self-improvement traces that produced the changes.

A capability is therefore judged from what the modification actually did during evaluation, not simply from the score of the node containing it. The evidence includes, for example, whether outputs were valid, which outputs were rejected, whether newly introduced functionality was actually invoked, and the corresponding task outcomes. This is important because a useful mechanism can live inside an otherwise unsuccessful agent.

In Appendix A.2–A.4, the labels “action interface” and “observation processing” are our post-hoc descriptions; they are never given to the consolidator. For example, Node 4's traces showed that its actions were consistently accepted but that it largely ignored the observation, whereas Node 12 incorporated the observation and interaction history but emitted an invalid action vocabulary. The consolidator inferred this complementarity from their implementations and observed behavior and synthesized a successor combining the useful parts, together with additional logic inferred from population-wide failures.

Duplicates and conflicts are handled in the same agentic synthesis step rather than by mechanically unioning patches. The consolidator can retain one implementation, discard redundant or unsupported changes, repair an incomplete mechanism, or combine compatible mechanisms according to the observed evidence. The resulting successor is then evaluated using the same task protocol. Thus, Looped-RSI does not require a domain-specific capability detector: capability identification and conflict resolution emerge from reasoning jointly over implementations and their measured behavioral consequences.

Q3. Could capabilities guide the evolution process? Does agent size relate to capability or performance? Could easier/harder tasks be assigned according to ability?

Yes, these are interesting complementary directions, but they are not assumptions of Looped-RSI.

We do not use agent size as a proxy for capability, and our experiments do not support a claim that larger task-agent or improver code should monotonically imply better performance. Task-agent quality is measured through behavioral/task evaluation, while improver quality is measured through the descendants it produces.

Our new Gemini experiments (Figure 2) provide a related perspective on model capability rather than agent-code size. In addition to Qwen3.6-35B-A3B, we evaluate Looped-RSI with the substantially more capable Gemini model. Looped-RSI still provides large gains on Crafter and MiniHack. This suggests that the inheritance problem targeted by consolidation is not simply an artifact of a weaker underlying model: even stronger models can discover useful information along different branches without automatically making that progress jointly heritable.

Capability-aware evolution or adaptive curricula could also be useful—for example, assigning easier or harder tasks according to current competence, or using estimated capabilities to guide which branches to expand. However, these mechanisms address which experiences and branches the search should generate. Looped-RSI addresses a different problem: once branching search has produced multiple partially informative lineages, how should discoveries distributed across them become jointly heritable, and how should credit be assigned separately to the mechanism producing future improvements?

AuthorsOpenReview ↗

Official Comment by Authors (3/3)

Q4. How much of the improvement comes from the proposed method itself? How does Looped-RSI compare with a simpler approach that gives all agents' code and experiences to an LLM under the same compute budget?

We agree that this is the key attribution question. There are two interpretations of the proposed simpler baseline, and the revised paper now addresses both.

First, if “give the LLM all agents' code and experiences” means allowing the ordinary improver to inspect population history before modifying an agent, HyperAgent already has this information access. Its improver can inspect the read-only run archive containing previous generations' modifications, evaluation outputs, predictions, and self-improvement traces. The consolidation evidence in Looped-RSI is derived from these same underlying run artifacts (Appendix C.3). Thus, the distinction is not that Looped-RSI exposes otherwise hidden population information. HyperAgent can inspect evidence from other branches, but each descendant is still produced by modifying one selected parent; Looped-RSI introduces a many-to-one inheritance operation that can construct a new state from evidence distributed across several lineages.

Second, if the proposed baseline means asking an LLM to actually synthesize a new agent from the whole population, this is precisely the synthesis component that we now ablate. Starting from the identical first 15-node BabyAI population:

MethodBest score within 45 expansions
Restart from best existing node, no consolidation0%
Consolidate, but append synthesized state to existing tree80%
Full Looped-RSI100%

Population-level synthesis alone therefore explains a substantial part of the gain, but not all of it: making the synthesized state the inherited starting point for subsequent search further improves performance from 80%80\% to 100%100\%. This directly separates seeing the population, synthesizing across it, and making the synthesis heritable.

The gain is also not explained by additional compute. On Qwen BabyAI, Looped-RSI and HyperAgent use approximately 25.8K vs. 25.5K total LLM calls, while reaching 100%100\% vs. 84%84\% during self-improvement. The synthesis calls themselves account for only 211 calls, about 0.8%0.8\% of Looped-RSI's total; most of the additional overhead is spent evaluating proposed task agents and probing improvers.

The new Gemini experiments provide a second, directly monetary compute control. On Crafter, Looped-RSI reaches 49.1%49.1\% at a total API cost of $536, compared with 9.8%9.8\% for HyperAgent at $611; on MiniHack, it reaches 41.7%41.7\% at $580.5, compared with 4.2%4.2\% at $793. In both cases the methods share exactly the same first 15-node population, all of whose agents score zero, and diverge only at the first consolidation boundary. Thus, the improvement persists even when Looped-RSI uses less total inference cost.

Finally, the gain of Looped-RSI is not limited to synthesizing a better current task agent. The Qwen Crafter run provides a particularly clean isolation of the improver contribution: none of the six task consolidations changed the task agent, so accepted consolidation changes acted only through the improver. From the same task-agent state, the consolidated improver produced children with mean scores 6.46.4 vs. 2.72.7 in round 2 and 5.55.5 vs. 0.90.9 in round 3. The cross-environment transfer experiment provides complementary evidence: with the same target-domain starting agent, same self-improvement budget, and no population consolidation during transfer, the BabyAI-evolved Looped-RSI improver reaches 47.5%47.5\% on BabaIsAI and 19.09%19.09\% on Crafter, compared with 2.5%2.5\% and 0%0\% for the HyperAgent improver.

Together, these controls isolate the source of the improvement more precisely. Population access alone is insufficient; population synthesis produces a large gain; making that synthesis the inherited state improves it further; and separately consolidating the improver measurably improves the mechanism that generates future descendants. These effects persist for our additional experiments. We have added these ablations, compute accounting, Gemini experiments, and improver analyses to the revised manuscript.

AuthorsOpenReview ↗

Rebuttal Reminder

Dear Reviewer qPFZ,

Just a gentle reminder as the discussion deadline is approaching. We hope our response and the additional clarifications/experiments addressed your concerns, and we would be grateful if you could reconsider your assessment in light of our revised response. We would also greatly appreciate any further feedback you may have before the deadline. Thank you again for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.