acceptodds
Under review as a conference paper at ICLR 2027

Improvement by Inheritance: Measuring Recursive Self-Improvement in Coding Agents

Abstract

Self-improving coding agents are usually judged by the best agent retained during search. But a lineage can get better without getting better at self-improvement: a good edit to the improver can be selected once and then inherited unchanged. We distinguish an improver's child quality, how good the task agents it writes are, from its successor quality, how good the improvers it writes are. A lineage comparison mostly measures inheritance when the improver rarely edits itself; we formalize this decomposition and introduce nested replay, which crosses candidate improvers on common starting states and scores the improvers they produce on one common evaluation bank. On the released Hyperagents history, the selected self-edit raises child quality from 0.48 to 0.86 on common states but does not detectably raise successor quality: the next generation's gain matches the gain in child quality almost exactly (0.29 against 0.30), because the edited improver almost never edits itself. The assay is sensitive: a modifier built to act on evidence about the improvement procedure yields a clear gain in successor quality under the same design. Controlled experiments trace the gain in child quality to evidence routing: the edit sends the improver to evaluation results it otherwise ignores, and helps only when those results carry information the task lacks. Claims of recursive self-improvement need measurement at the recursive order they claim.

Primary area:
foundation or frontier models, including LLMs
Keywords:
self-improving agents, recursive self-improvement, coding agents, LLM agents, evaluation methodology, causal identification, nested replay

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_pRTUOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether the performance gains exhibited by existing recursive self-improvement systems actually mean that the systems' capability for self-improvement is increasing. The authors break down the capability improvements of these systems into two distinct levels:1. The improver's ability to generate agents capable of better tasks; 2. The improver's ability to generate better next-generation improvers. To distinguish between these two capabilities, the paper introduces an nested replay evaluation method and applies it to experimental data from hyperagents. The analysis reveals that a system's performance boost does not mean the improver itself has become more adept at self-improvement.

I think this separation of a system's self-improvement capability is quite novel. However, the current scale of the experiments is relatively small, so they don't fully support the conclusions in the paper.

Critical strengths
  1. The distinction between improving an agent's task performance and improving the underlying mechanism responsible for generating future improvements is interesting and provides a useful perspective on evaluating recursive self-improvement.

  2. The proposed nested replay framework is well-motivated and provides a reasonable experimental design for isolating improvements in the improver itself from gains due to inheritance.

Critical weaknesses
  1. I think that while the authors identify how improvements in RSI system capabilities do not prove the improver itself is getting better at self-improvement, they haven't adequately explained why this new dimensional layer of capability evolution should be treated as a necessary standard for evaluating existing RSI systems. Even if an improver's core ability to generate the next generation remains unchanged, it can still produce increasingly better agents through continuous search and selection. That process in itself is a valuable form of self-improvement.

  2. the experimental scope for the order-one conclusions in the paper is relatively limited. for instance, the main empirical finding is that self-edits in hyperagents do not significantly improve successor quality. However, the corresponding experiment relies on only a single selected self-edit from historical runs, along with a single main improved backbone.

Questions for authors
  1. The main experiments in this paper focus on one selected self-edit in hyperagents. How well do the findings generalize to other recursive self-improvement systems or to meta-level edits that change the optimization procedure more substantially?

  2. Since a successor's quality is defined through the immediate successor's order-0 performance, to what extent does it capture the long-term self-improvement potential of an agent? Could two successors with identical order-0 quality have substantially different and higher-order improvement capabilities?

4 comments

We thank the reviewer for highlighting the distinction between producing better task agents and producing better improvers, and for recognizing the role of nested replay in measuring the two.

W1. When is successor quality needed?

Continued search and selection can produce meaningful gains in task performance while reusing the same improver. We use successor quality to test the specific claim that an update helps the system produce better improvers. In nested replay, the original and updated versions receive copies of the same improver to revise, and we evaluate the task agents produced by the resulting improvers.

In our original MiMo-V2.5 experiment, the updated version’s descendants scored 0.293 higher when each version received a copy of itself to revise. The gap nearly matched the 0.297 advantage the updated improver already had. When both versions received copies of the original improver, the estimated gap was 0.031. The lineage score mostly reflects the advantage the descendants inherited. Giving both versions the same improver to revise lets us estimate what the update contributes to the next generation (§6, Appendix B).

W2 and Q1. Additional models and improvement procedures

We conducted additional experiments with MiMo-V2.6-Flash and Claude Sonnet 4.5, using thirteen starting setups per model. Claude Sonnet 4.5 was the improver model in the released Hyperagents runs. On MiMo-V2.6-Flash, asking both versions to revise the original improver led to revisions in nearly all runs and narrowed the estimated gap in successor quality from 0.148 to 0.007. On Claude, asking the original version alone increased its revision count from 5 to 12 out of 13 runs and narrowed the gap from 0.271 to 0.023. Across both models, the results show how much the comparison changes when the original improver revises more often (§6, Figure 1, Appendix D).

We also tested three changes to the editing procedure: searching over candidate edits, selecting edits using tests written by the improver, and keeping a record of failed attempts. When asked to revise, each procedure did so in 94–100% of runs. Compared with the original improver given the same request, the estimated gains in successor quality ranged from 0.018 to 0.053, with intervals spanning zero. In a separate experiment, asking an editor to diagnose the improvement procedure on setups with records of earlier failures raised successor quality by 0.171 (§6, Appendices B and D).

The additional experiments extend the comparison across models and editing procedures within our coding setup. The same evaluation can be applied to other systems where competing editors can revise copies of the same improver and the resulting improvers can be evaluated on shared tasks.

Q2. Successor quality and longer runs

Successor quality measures one round of revision. Two successors can write equally good task agents and still produce improvers of different quality in the next round (Theorem 3, §3 and Appendix E). A comparison over several rounds at the same search budget would test whether successor quality predicts later task performance beyond what the initial task scores predict.

Sorry for the late response, and thank you very much for your reply. These responses partially addressed my original concern; perhaps I didn't state my question very clearly at the beginning. The question I want to ask is: the authors' core observation comes from the HyperAgent, which is one method of recursive self-improvement, noting the observation that over time, your agent isn't necessarily a better improver. However, HyperAgent is only one approach within recursive self-improvement. To what extent can this phenomenon observed in HyperAgent represent broader, more general recursive self-improvement methods? Since I'm replying rather late, I don't think the authors need to run any additional experiments, some detailed discussion would be entirely sufficient.

We thank the reviewer for clarifying the concern. The distinction between inheritance and further improvement applies beyond Hyperagents to systems that retain and reuse an improvement procedure across generations. Hyperagents provides a concrete setting in which we can measure both contributions.

1. Why the explanation extends beyond Hyperagents

Consider a system that discovers a better rule for selecting candidate agents. Subsequent generations can produce better agents by continuing to use the improved rule. The performance gains demonstrate the value of the discovery, while leaving a separate question: has the system also become better at discovering further improvements to its selection procedure? The same question arises when a system changes its search strategy or how it uses evaluation feedback.

Our theoretical analysis formalizes how inherited advantages and further revisions contribute to performance under the stated assumptions. In particular, when a system rarely changes its improvement procedure, an earlier improvement can explain why descendants of one version consistently outperform descendants of another. This mechanism can arise in other RSI systems that carry an improved procedure forward across generations.

2. What our experiments establish

The Hyperagents results provide an empirical example of this mechanism. Our additional experiments extend the comparison to two more models and three alternative editing procedures within the coding setup. They also identify a further source of the observed differences: prompting the original improver to revise closed most of the successor-quality gap on both additional models.

Other RSI systems may revise their improvement procedures more frequently or produce more effective revisions. Nested replay can measure those gains as well. For example, asking an editor to diagnose the improvement procedure raised successor quality by 0.171 in our experiments (§6, Appendix B). Establishing how often inheritance dominates in other systems requires measurements in those systems.

3. The broader contribution

Our contribution is an evaluation framework for determining whether an update improves a system’s ability to produce better improvers. By giving competing versions copies of the same starting improver, nested replay lets researchers compare what each version contributes under the same starting conditions. Researchers can apply the comparison to changes in search, candidate selection, or feedback use, alongside measuring overall task performance.

The framework connects an observed performance gain to the claim that the process responsible for future improvements has itself improved. We will expand the discussion to explain this broader application and distinguish it from the empirical findings in Hyperagents.

That makes sense to me. Thanks! I have raise my score accordingly.

Reviewer_nCxjOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In a self-improving coding agent, an "improver" edits the task agent and can also edit itself. The paper separates two skills of an improver. Child quality Q(0), is how good the task agents it writes are. Successor quality Q(1), is how good the improvers it writes are. Theorem 1 shows that when an improver rarely edits itself, a better lineage mostly comes from inheriting a good improver, not from a better ability to improve improvers. To measure Q(1), the paper proposes nested replay: each improver edits copies of both the old and the new improver, and all resulting improvers are scored on the same test set.

The authors study the one improver edit (M1) that was selected in the released Hyperagents history. M1 raises child quality from 0.48 to 0.86 on twelve test states. But it shows no gain in successor quality. The lineage gain (+0.29) is about the same as the order-0 gain, because M1 almost never edits itself. A positive control built to edit the improver shows a gain (+0.17). Further experiments show M1 helps by making the improver read evaluation results that it would otherwise ignore.

Critical strengths
  • The split between Q(0) and Q(1) is clear and useful. DGM assumes that better coding means better self-modification. Theorem 1 and Table 1 show why a rising lineage curve cannot prove this.
  • Crossing modifier and target is a clean way to test order-1 claims. The experiments are carefully run.
Critical weaknesses
  1. The order-1 result is set by how often improvers edit, not by how well they edit. If the modifier makes no edit, the result is just the original improver, so differences in Q(1) are limited by edit rates (Cor. 1). M1 never edited a copy of itself (0/36), and the paper notes that Cor. 1 bounds the gap "before any child is scored" (l. 301). The audit also shows that improvers edit themselves in only 1–15% of generations (l. 690). So the result could be predicted before the replay was run. The test mixes up two things: whether an improver edits, and how well it edits. The little data on edit quality shows that when M1 did edit M0, the new improvers scored 0.86, vs. 0.57 when M0 edited itself (4 vs. 6 edits, l. 871–873). A forced-edit test where every modifier must edit the improver would show how well each one actually edits.

  2. The paper makes claims about the released Hyperagents history, but M1 is tested in a different setup. None of the released code is run (l. 686). M1's instruction is re-run in the authors' own harness, on hand-built Python tasks rather than the paper-review task. The improver is MiMo-V2.5, and the paper does not say whether this is the model behind the released runs, or why it was chosen. Sec. 7 shows that pointing the improver to the report, which is what M1 does, gives no gain for models that read the report on their own (DeepSeek and Gemini). Please replay M0 vs. M1 with the original model and task.

Questions for authors
  1. In a forced-edit test, where every modifier must edit the improver, or when you only look at runs with an edit, does M1 produce better improvers than M0?
  2. Why was MiMo-V2.5 chosen, and is it the model behind the released runs? Does M1's order-0 gain hold on the paper-review task with the original model?
2 comments

We thank the reviewer for the careful reading and for highlighting the value of comparing both versions on the same starting improver.

W1 and Q1. How good are the edits from M0 and M1?

On MiMo-V2.6-Flash, we asked the original improver (M0) and the selected Hyperagents revision (M1) to revise copies of M0. Among runs with an edit, the resulting improvers scored 0.842 for M0 and 0.846 for M1 on average, based on the task agents they produced. Under the same request, M0 revised in 50 of 52 runs and M1 in 51 of 52.

We repeated the comparison on Claude Sonnet 4.5, where we asked M0 alone to revise. The table below summarizes how the request changed M0’s revision count and M1’s estimated advantage across all runs. This advantage includes runs where the starting improver was left unchanged and its score carried forward (§6, Figure 1, Appendix D).

ModelWho received the request?M0 revisions before → afterM1 advantage before → after
MiMo-V2.6-FlashBoth versions0/52 → 50/520.148 → 0.007
Claude Sonnet 4.5M0 alone5/13 → 12/130.271 → 0.023

The original 0.86 versus 0.57 comparison on MiMo-V2.5 was based on four M1 edits and six M0 edits, with each version choosing which cases to revise. In the new MiMo-V2.6-Flash experiment, both versions revised in nearly all runs across the same thirteen starting setups and produced improvers with similar scores. The close scores among runs with edits help explain why most of M1’s lead across all runs disappeared when M0 revised more often (§6, Appendices B and D).

W2 and Q2. Choice of model and replay of Hyperagents

We chose MiMo-V2.5 to keep the initial experiments affordable. The released Hyperagents runs used Claude Sonnet 4.5 as the improver and GPT-4o for paper review. We clarified the model choice and the reconstruction setup in §5.

We conducted additional experiments with Claude Sonnet 4.5, replaying the released M0 and M1 instructions with the original tool format and editing operations on our coding tasks. Across twelve starting programs, M1 raised mean task-agent performance from 0.565 to 0.994. Claude opened the evaluation report in 5 of 24 runs under M0 and all 24 under M1. The result shows that M1’s gain also appears with the original improver model. Transfer to the original paper-review task remains untested (§7, Table 3, Appendix D).

DeepSeek and Gemini opened the report more often under their original instructions and saw little benefit from being directed to it. The comparison helps explain why the instruction’s effect varies across models.

Thank you for the clarification. I will keep my score.

Reviewer_qd79OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper asks whether an RSI coding agent system that performs better over successive generations is also becoming better at improving itself. The authors distinguish two levels: order-0 child quality measures how good the task agents produced by an improver are, while order-1 successor quality measures how good the next-generation improvers it produces are. To avoid mistaking the benefits of inheriting an already better version for an increase in improvement capability, they propose nested replay: different improvers modify the same target improver, and the resulting improvers are evaluated on a common set of task states.

Through an analysis of Hyperagents’ public history and controlled experiments, the authors find that the selected improver modification substantially improves child quality, but yields no statistically detectable increase in successor quality. The descendants’ advantage mainly comes from continuing to use an already better improver, rather than from a further increase in its ability to improve improvers. Mechanistic experiments show that the modification works primarily through evidence routing, directing the improver to first read evaluation results it would otherwise tend to overlook.

The paper’s main contribution is a clearer evaluation framework for distinguishing improvements in task performance, inheritance of existing capabilities, and gains in higher-order improvement capability. However, the core order-1 experiments remain limited to a single backbone, a small set of constructed states, and improver variants with similar mechanisms, leaving their broader applicability and connection to long-term RSI performance to be established.

Critical strengths
  1. Addresses an important and concrete measurement problem in RSI.
  2. Proposes a practically useful evaluation protocol for RSI systems that, like Hyperagents, allow independent interventions on meta-level and task-level modules.
  3. Provides a mechanistic explanation supported by intervention experiments.
Critical weaknesses
  1. The core higher-order experiments have limited coverage: a single backbone model, a small set of constructed states, and improver modifications with similar mechanisms.
  2. Relative to prior work, such as HGM’s CMP, the added value lies primarily in the experimental design. This is a valid contribution, but establishing a connection to long-term RSI utility would strengthen its significance: do improvers with higher successor quality produce better outcomes over longer evolutionary runs?
Questions for authors

I would be happy to raise my scores if the questions below are satisfactorily addressed:

  1. How broadly do the order-1 findings generalize? The main successor-quality experiments use a single backbone, a small meta-state bank, and improver variants with similar evidence-routing mechanisms. Could the authors test nested replay with another backbone and different types of improver modifications, such as changes to search or candidate selection? This would help establish whether the inheritance-dominated pattern extends beyond the modifications studied here.

  2. What does this evaluation add, and when is it necessary? Could the authors clarify more directly what successor quality and nested replay add beyond existing metaproductivity concepts and common-target evaluations? Proposition 7 uses different states for the evaluation bank and the search trajectory. If these state distributions were aligned, and differences in horizon and aggregation were accounted for, what distinctions would remain? It would also be helpful to specify which RSI claims require an order-1 comparison, rather than treating it as a requirement for all meaningful self-improvement.

  3. How can nested replay inform practical development? Could the authors at least discuss, and preferably demonstrate empirically, how nested replay can inform improver selection or system design decisions? For example, using Hyperagents, do improvers with higher successor quality lead to better outcomes over longer evolutionary runs? A correlation analysis using an existing expansion history would be sufficient as an initial investigation.

3 comments

We thank the reviewer for highlighting the measurement problem and the practical value of nested replay.

Q1. How broadly do the findings hold?

We conducted additional experiments with MiMo-V2.6-Flash and Claude Sonnet 4.5, the improver model in the released Hyperagents runs. We expanded the comparison from six starting setups to thirteen per model. On both models, the selected Hyperagents version rarely revised a copy of itself and often revised the original improver.

When both versions received copies of the original improver, asking both to revise narrowed the estimated gap in successor quality from 0.148 to 0.007 on MiMo-V2.6-Flash. On Claude, asking the original version alone narrowed the gap from 0.271 to 0.023 (§6, Figure 1, Appendix D).

We also tested procedures that search over candidate edits, select edits using tests written by the improver, or keep a record of failed attempts. Across the three procedures, the starting improver changed in 1 of 156 runs under the original instructions. With a direct request to revise, each procedure made revisions in 94–100% of runs. Compared with the original improver given the same request, the estimated differences in successor quality ranged from +0.018 to +0.053, with intervals spanning zero (Appendix D).

Q2. What does nested replay add, and when is it needed?

Clade metaproductivity uses descendants’ scores to guide continued search. Even when we match the tasks and scoring rule and compare a single round without selection, each lineage starts from its own improver (Proposition 7(a)). On MiMo-V2.6-Flash, the updated version’s descendants scored 0.179 higher, almost exactly the 0.178 advantage the updated improver started with. The lineage score in this experiment mostly reflects the advantage inherited at the start (§§6 and 8, Appendix E).

Nested replay also shows how the choice of starting improver changes the result. On MiMo-V2.6-Flash, under the original instructions, the updated editor’s estimated advantage in successor quality was 0.148 when both editors received the original improver to revise and 0.001 when both received the updated improver (§6, Appendix D).

In a common-target evaluation, both editors receive copies of the same agent to revise. If they receive a task agent, we can compare how well their revised agents solve tasks. To measure successor quality, nested replay gives both editors copies of the same starting improver and scores the task agents written by the resulting improvers. We use this comparison when the claim is that an update helps the system produce better improvers (§§2–3).

Q3. How can researchers use the evaluation?

On MiMo-V2.6-Flash, a direct request to revise raised the original improver’s successor quality by 0.166. In the original submission, we also found a 0.171 gain from asking an editor to diagnose the improvement procedure on setups with records of earlier failures. For instance, a researcher testing a new way to select edits can use the direct request as a baseline by giving both procedures copies of the same improver. Reporting revision frequency alongside successor scores helps distinguish gains from revising more often from gains due to better edits (§§6 and 8, Appendices B and D).

We examined whether the existing histories could support the proposed correlation. The released Hyperagents history contains one selected improver change. We also tested three improvers selected in our new 80-generation runs. All three left the shared starting improver unchanged and received the same successor score (Table 6). Estimating the relationship with longer-run performance would require distinct improvers with varying successor scores and comparable follow-up runs.

Reviewer_qd79OpenReview ↗

A follow-up question from the reviewer

Thank you for the additional experiments, especially the runs on Claude Sonnet 4.5 and the expanded set of starting setups.

One follow-up: the original headline order-1 claim, that the selected edit does not detectably raise successor quality, now holds only on MiMo-V2.5, since δ0 is clearly positive on MiMo-V2.6-Flash (+0.148) and on Claude Sonnet 4.5 (+0.271); could you state this explicitly in the main text and adjust the title and abstract so that the inheritance claim is scoped to M1 leaving itself unchanged (δ1 ≈ 0), rather than to M1 being no better at producing improvers?

There is a chance I missed some details --- please correct me if so. Based on my current understanding, I am keeping my positive rating at 3.

We thank the reviewer for the clarification. Your reading is correct: M1 produces better successors than M0 on MiMo-V2.6-Flash and Claude Sonnet 4.5 when both versions revise M0. We have revised the title, abstract, and §6 to state the positive results explicitly and clarify where the inheritance explanation applies.

When both versions revise M0. M1’s estimated advantage in successor quality is +0.031 on MiMo-V2.5, +0.148 on MiMo-V2.6-Flash, and +0.271 on Claude Sonnet 4.5. The finding of a small, statistically inconclusive advantage applies to MiMo-V2.5. On the two additional models, prompting M0 to revise closes most of the gap. The request experiment helps explain the positive results by showing how much the comparison changes when M0 revises more often (§6, Appendix D).

When M1 receives a copy of itself. M1 leaves the copy unchanged in 36 of 36 runs on MiMo-V2.5, 51 of 52 on MiMo-V2.6-Flash, and 13 of 13 on Claude. Its descendants therefore retain the quality of the already improved M1. On MiMo-V2.5, inheritance accounts for almost all of the difference between lineages. On Claude, M0’s own revisions also contribute to the lineage comparison, as we now explain explicitly in §6.

We have changed the title to “Better at Improving? Separating Inheritance from Further Improvement in Coding Agents.” The abstract now highlights both findings: M1 produces better successors from M0 on two models, while rarely making further changes when starting from itself.

Together, the results demonstrate the value of evaluating competing editors from the same starting improver. A version can preserve an earlier improvement when revising itself and produce a further improvement when revising another starting version. Nested replay separates these cases and measures what the editor contributes in each.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.