- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether the performance gains exhibited by existing recursive self-improvement systems actually mean that the systems' capability for self-improvement is increasing. The authors break down the capability improvements of these systems into two distinct levels:1. The improver's ability to generate agents capable of better tasks; 2. The improver's ability to generate better next-generation improvers. To distinguish between these two capabilities, the paper introduces an nested replay evaluation method and applies it to experimental data from hyperagents. The analysis reveals that a system's performance boost does not mean the improver itself has become more adept at self-improvement.
I think this separation of a system's self-improvement capability is quite novel. However, the current scale of the experiments is relatively small, so they don't fully support the conclusions in the paper.
Critical strengths
-
The distinction between improving an agent's task performance and improving the underlying mechanism responsible for generating future improvements is interesting and provides a useful perspective on evaluating recursive self-improvement.
-
The proposed nested replay framework is well-motivated and provides a reasonable experimental design for isolating improvements in the improver itself from gains due to inheritance.
Critical weaknesses
-
I think that while the authors identify how improvements in RSI system capabilities do not prove the improver itself is getting better at self-improvement, they haven't adequately explained why this new dimensional layer of capability evolution should be treated as a necessary standard for evaluating existing RSI systems. Even if an improver's core ability to generate the next generation remains unchanged, it can still produce increasingly better agents through continuous search and selection. That process in itself is a valuable form of self-improvement.
-
the experimental scope for the order-one conclusions in the paper is relatively limited. for instance, the main empirical finding is that self-edits in hyperagents do not significantly improve successor quality. However, the corresponding experiment relies on only a single selected self-edit from historical runs, along with a single main improved backbone.
Questions for authors
-
The main experiments in this paper focus on one selected self-edit in hyperagents. How well do the findings generalize to other recursive self-improvement systems or to meta-level edits that change the optimization procedure more substantially?
-
Since a successor's quality is defined through the immediate successor's order-0 performance, to what extent does it capture the long-term self-improvement potential of an agent? Could two successors with identical order-0 quality have substantially different and higher-order improvement capabilities?
4 comments
Sorry for the late response, and thank you very much for your reply. These responses partially addressed my original concern; perhaps I didn't state my question very clearly at the beginning. The question I want to ask is: the authors' core observation comes from the HyperAgent, which is one method of recursive self-improvement, noting the observation that over time, your agent isn't necessarily a better improver. However, HyperAgent is only one approach within recursive self-improvement. To what extent can this phenomenon observed in HyperAgent represent broader, more general recursive self-improvement methods? Since I'm replying rather late, I don't think the authors need to run any additional experiments, some detailed discussion would be entirely sufficient.
We thank the reviewer for clarifying the concern. The distinction between inheritance and further improvement applies beyond Hyperagents to systems that retain and reuse an improvement procedure across generations. Hyperagents provides a concrete setting in which we can measure both contributions.
1. Why the explanation extends beyond Hyperagents
Consider a system that discovers a better rule for selecting candidate agents. Subsequent generations can produce better agents by continuing to use the improved rule. The performance gains demonstrate the value of the discovery, while leaving a separate question: has the system also become better at discovering further improvements to its selection procedure? The same question arises when a system changes its search strategy or how it uses evaluation feedback.
Our theoretical analysis formalizes how inherited advantages and further revisions contribute to performance under the stated assumptions. In particular, when a system rarely changes its improvement procedure, an earlier improvement can explain why descendants of one version consistently outperform descendants of another. This mechanism can arise in other RSI systems that carry an improved procedure forward across generations.
2. What our experiments establish
The Hyperagents results provide an empirical example of this mechanism. Our additional experiments extend the comparison to two more models and three alternative editing procedures within the coding setup. They also identify a further source of the observed differences: prompting the original improver to revise closed most of the successor-quality gap on both additional models.
Other RSI systems may revise their improvement procedures more frequently or produce more effective revisions. Nested replay can measure those gains as well. For example, asking an editor to diagnose the improvement procedure raised successor quality by 0.171 in our experiments (§6, Appendix B). Establishing how often inheritance dominates in other systems requires measurements in those systems.
3. The broader contribution
Our contribution is an evaluation framework for determining whether an update improves a system’s ability to produce better improvers. By giving competing versions copies of the same starting improver, nested replay lets researchers compare what each version contributes under the same starting conditions. Researchers can apply the comparison to changes in search, candidate selection, or feedback use, alongside measuring overall task performance.
The framework connects an observed performance gain to the claim that the process responsible for future improvements has itself improved. We will expand the discussion to explain this broader application and distinguish it from the empirical findings in Hyperagents.
That makes sense to me. Thanks! I have raise my score accordingly.
We thank the reviewer for highlighting the distinction between producing better task agents and producing better improvers, and for recognizing the role of nested replay in measuring the two.
W1. When is successor quality needed?
Continued search and selection can produce meaningful gains in task performance while reusing the same improver. We use successor quality to test the specific claim that an update helps the system produce better improvers. In nested replay, the original and updated versions receive copies of the same improver to revise, and we evaluate the task agents produced by the resulting improvers.
In our original MiMo-V2.5 experiment, the updated version’s descendants scored 0.293 higher when each version received a copy of itself to revise. The gap nearly matched the 0.297 advantage the updated improver already had. When both versions received copies of the original improver, the estimated gap was 0.031. The lineage score mostly reflects the advantage the descendants inherited. Giving both versions the same improver to revise lets us estimate what the update contributes to the next generation (§6, Appendix B).
W2 and Q1. Additional models and improvement procedures
We conducted additional experiments with MiMo-V2.6-Flash and Claude Sonnet 4.5, using thirteen starting setups per model. Claude Sonnet 4.5 was the improver model in the released Hyperagents runs. On MiMo-V2.6-Flash, asking both versions to revise the original improver led to revisions in nearly all runs and narrowed the estimated gap in successor quality from 0.148 to 0.007. On Claude, asking the original version alone increased its revision count from 5 to 12 out of 13 runs and narrowed the gap from 0.271 to 0.023. Across both models, the results show how much the comparison changes when the original improver revises more often (§6, Figure 1, Appendix D).
We also tested three changes to the editing procedure: searching over candidate edits, selecting edits using tests written by the improver, and keeping a record of failed attempts. When asked to revise, each procedure did so in 94–100% of runs. Compared with the original improver given the same request, the estimated gains in successor quality ranged from 0.018 to 0.053, with intervals spanning zero. In a separate experiment, asking an editor to diagnose the improvement procedure on setups with records of earlier failures raised successor quality by 0.171 (§6, Appendices B and D).
The additional experiments extend the comparison across models and editing procedures within our coding setup. The same evaluation can be applied to other systems where competing editors can revise copies of the same improver and the resulting improvers can be evaluated on shared tasks.
Q2. Successor quality and longer runs
Successor quality measures one round of revision. Two successors can write equally good task agents and still produce improvers of different quality in the next round (Theorem 3, §3 and Appendix E). A comparison over several rounds at the same search budget would test whether successor quality predicts later task performance beyond what the initial task scores predict.