acceptodds
Under review as a conference paper at ICLR 2027

Beyond the Latest: Historical Capability Fragmentation and Recovery in Self-Evolving Agents

Abstract

Self-evolving language model agents improve through iterative data generation, feedback, and model updates, yet capabilities do not necessarily accumulate across iterations. Later iterations may lose previously acquired capabilities, while historical iterations retain complementary ones that no single iteration fully covers, a phenomenon we call historical capability fragmentation. Meanwhile, identifying what to preserve from model updates is unreliable: repaired and damaged instances exhibit highly overlapping responses across model parameter rows, making model-parameter response strength insufficient to distinguish beneficial from harmful update effects. We introduce CapDyn-Match, a lightweight response-level recovery framework that selects among frozen historical responses without modifying model parameters. It combines (1) historical candidate scoring, using query–response representations and cross-iteration agreement to identify useful responses, and (2) conservative selection, using a margin gate with a training-derived fallback to avoid harmful switches under insufficient evidence. Across distinct self-evolution frameworks and 14 math, general reasoning, and code benchmarks, CapDyn-Match improves question-micro accuracy over the latest iteration by 3.13 and 3.05 points on two three-iteration Qwen3-4B trajectories. On a six-iteration Qwen3-1.7B trajectory, it improves all 14 benchmarks by up to 7.40 points. Leave-one-dataset-out evaluation further shows transfer to unseen benchmarks. These results reveal a broader limitation of self-evolution: acquiring new capabilities does not guarantee preserving existing ones, leaving useful capabilities distributed across model history rather than consolidated in the latest iteration. Code: https://anonymous.4open.science/r/CapDyn-Match-4FC2/

Primary area:
foundation or frontier models, including LLMs
Keywords:
Self-Evolving Agents, Capability Dynamics, Model Selection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_PS76OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates capability fragmentation during the self-evolution of language model agents, where successive model updates may lose previously acquired capabilities, while different historical checkpoints retain complementary strengths. The authors analyze instance-level correctness transitions and parameter-response patterns across self-evolution iterations.

To recover these capabilities, the paper proposes CapDyn-Match, a response-level selection framework that chooses among outputs generated by frozen historical checkpoints. The method combines query–response representations, cross-iteration answer agreement, and a margin-based selection mechanism with a fallback to a training-derived prior.

Experiments are conducted on Agent0 and R-Zero self-evolution trajectories across 14 benchmarks covering mathematics, general reasoning, and code generation. The results show that CapDyn-Match generally improves accuracy over the latest checkpoint and several selection and model-merging baselines, with additional evaluations on held-out benchmarks and longer evolution trajectories.

Critical strengths
  1. An interesting observation about capability dynamics during self-evolution. The paper provides empirical evidence that improvements in aggregate performance can mask substantial changes in instance-level correctness. The observation that different historical checkpoints retain complementary capabilities is useful for understanding the limitations of iterative model self-improvement.

  2. Reasonably comprehensive empirical evaluation. The authors evaluate the proposed method across different self-evolution frameworks, multiple model configurations, and 14 benchmarks. The inclusion of baseline comparisons, ablation studies, and leave-one-dataset-out evaluations helps establish the practical effectiveness of historical response selection.

Critical weaknesses
  1. Limited methodological novelty. My main concern is that the proposed CapDyn-Match framework is relatively straightforward from a methodological perspective. It essentially performs instance-level selection among responses from historical model checkpoints, using learned representation-based correctness prediction, answer agreement, and a confidence-based fallback mechanism. These are largely established techniques in model selection and ensemble learning. While their application to self-evolving agents is reasonable, I find the resulting algorithmic contribution rather incremental, without a sufficiently novel technical insight.

  2. Limited incremental benefit from the proposed components and unclear practical significance. The ablation results in Table 3 show that the representation-only baseline already achieves 37.63% and 38.67% overall accuracy on Agent0 and R-Zero, compared with 37.98% and 38.76% for the full CapDyn-Match method. These relatively small improvements raise questions about the necessity and effectiveness of the additional agreement-based scoring and conservative selection mechanisms. Furthermore, the framework requires maintaining and generating responses from multiple historical checkpoints rather than consolidating the recovered capabilities into a single model. This limits its practical relevance as a solution to continual self-improvement.

Questions for authors

Please see Weaknesses

3 comments
AuthorsOpenReview ↗

Official Response to Reviewer PS76(Part I)

We sincerely thank you for your thorough and insightful feedback. We have carefully considered every comment and revised the manuscript accordingly. Due to space limits, the new and revised content (marked in blue) has been integrated not only into the relevant sections of the main text but also extensively into the appendix. We provide detailed point-by-point responses below.

C1: Positioning of Core Contributions

Response: We would like to clarify that the core contribution of this work does not lie in proposing a new ensemble algorithm. Rather, it lies in revealing an overlooked phenomenon in self-evolution, namely historical capability fragmentation, and in showing why such capabilities must be recovered at the response level rather than the parameter level. CapDyn-Match is a minimal sufficient instantiation for validating this finding, and its simplicity is a deliberate design choice. Specifically:

  • (1) A new problem finding: capabilities do not accumulate across iterations. Existing self-evolution methods adopt the latest checkpoint by default, implicitly assuming that capabilities accumulate across iterations. We find that similar aggregate accuracy conceals substantial capability turnover: 9.65%–12.58% of instances change correctness between adjacent iterations (Appendix, Table 6), and the historical union exceeds every individual checkpoint across all 14 benchmarks (Table 1). This phenomenon has not been systematically characterized before. It shows that the bottleneck of self-evolution lies not only in acquiring new capabilities but also in retaining the ones already acquired.

  • (2) A key technical insight: capability loss cannot be localized in parameter space. A natural approach is to identify and preserve beneficial updates at the parameter level. However, the overlap of highly responsive parameter rows between Gain and Damage instances (top-1%: 0.78–0.80) is comparable to that of random groupings (0.78–0.79), and the two groups also overlap in PCA projections (Appendix D.1, Figure 7). Beneficial and harmful updates are thus hard to distinguish at the parameter level, which directly motivates shifting recovery to the response level. This conclusion is not presupposed by existing model selection or ensemble methods; it emerges from our analysis.

  • (3) The design choices are not interchangeable: each component is empirically supported. If CapDyn-Match were merely a simple combination of existing techniques, replacing any component should make little difference. The experimental results show otherwise:

    • Query-only routing yields only +0.25 pp, whereas response-level representations yield +2.55 pp (Agent0), indicating that the correctness signal must come from the responses themselves;
    • Majority voting, which relies solely on answer agreement, underperforms CapDyn-Match by 2.62 pp;
    • Parameter-level merging methods are similarly limited: Model Soup only slightly exceeds the best checkpoint on Agent0 (+0.62 pp), and TIES improves on neither trajectory (Table 3).

    These comparisons indicate that the effectiveness stems from the insight of exploiting historical complementarity at the response level, rather than from any generic component.

Moreover, CapDyn-Match freezes all historical models and trains only lightweight probes. With parallel checkpoint inference, median latency increases only from 4.36 s to 5.52 s (Table 5), without retraining the base model or invoking a large judge model. We believe that validating a previously overlooked phenomenon with a simple mechanism, and achieving consistent gains across 14 benchmarks, itself demonstrates the value of the problem characterization.

AuthorsOpenReview ↗

Official Response to Reviewer PS76 (Part II)

C2: Component Contributions and Practical Significance

Response: We thank the reviewer for the careful reading. Note that Representation-only in Table 3 already includes gating and fallback, so its gap to the full method (+0.35/+0.09 pp) reflects only the agreement branch. Against Vanilla Linear, which has neither, the full method gains 0.85/1.06 pp in total (Agent0/R-Zero):

VariantAgent0R-ZeroComponent Added
Vanilla Linear37.1337.70–
Representation-only37.63 (+0.50)38.67 (+0.97)Gating & fallback
CapDyn-Match (Full)37.98 (+0.35)38.76 (+0.09)Agreement branch
  • (1) Both components yield statistically significant gains. With three iterations, query–response representations provide most of the gains, while agreement and gating add further improvements. Their paired bootstrap 95% CIs are [+0.18, +0.52]/[+0.02, +0.26] (agreement) and [+0.24, +0.76]/[+0.71, +1.23] (gating and fallback), all excluding zero.
  • (2) Agreement gains grow with more candidates. Three candidates yield few agreement patterns, yet these already help: when Iter1 and Iter2 agree but Iter3 disagrees, subset accuracy rises from 18.06% to 26.19% (Agent0) and from 19.59% to 26.14% (R-Zero) (Table 7). With six candidates (Qwen3-1.7B), agreement improves over Representation-only by +0.72 pp (nested CV) and +0.99 pp (LODO), with gains on all 14 held-out benchmarks (§4.4; Appendix F.7, Tables 15–16).
  • (3) Practical significance. Please see our response to C3 for a detailed discussion of inference cost and single-model consolidation.

In summary, agreement and gating each add significant gains on top of a strong representation-based scorer, and agreement becomes more useful as more candidates are available.

C3: Practical Significance and Single-Model Consolidation

Response: We thank the reviewer for this suggestion and agree that single-model consolidation is a valuable direction. Our work focuses on inference-time recovery: CapDyn-Match reuses existing historical checkpoints via a lightweight selection module, without modifying the base model. Parameter-level consolidation can follow as a subsequent stage; the two are complementary rather than contradictory.

  • (1) Practical value. CapDyn-Match requires no additional training beyond lightweight probes, yet outperforms the best single iteration by 2.90/2.34 pp (Agent0/R-Zero; Table 3). The cost is extra inference resources: 4 GPUs instead of 1, with median latency rising from 4.36 s to 5.52 s (Table 5). It thus suits settings where checkpoints are available and parallel inference is affordable.
  • (2) Comparison with parameter merging. CapDyn-Match outperforms Model Soup by 2.28/2.79 pp and TIES by 3.48/2.73 pp (Table 3). This matches our finding that Gain and Damage instances are hard to separate in parameter space (Appendix D.1, Figure 7), so direct merging cannot reliably keep beneficial updates while discarding harmful ones.
  • (3) Toward single-model consolidation. Response-level selection can also supply targeted training data for consolidation. Since CapDyn-Match needs no ground-truth labels at inference, it can select responses on unlabeled data, which can then serve for SFT or distillation into a single model.

We acknowledge that CapDyn-Match does not yet achieve single-model consolidation and relies on parallel inference resources. Nevertheless, it delivers immediate gains and lays a data foundation for future consolidation.

AuthorsOpenReview ↗

Kind Reminder: Discussion Period Ends Today

Dear Reviewer PS76,

We apologize for writing again. The discussion period ends today, so we wanted to send one final reminder in case our earlier message was missed.

In short, our responses address both of your main concerns:

  • Methodological novelty: Our core contribution is not a new ensemble algorithm. It is the finding of historical capability fragmentation: 9.65%–12.58% of instances change correctness between adjacent iterations. We also show that this loss cannot be localized in parameter space (Gain/Damage overlap 0.78–0.80, the same as random groupings, 0.78–0.79). This is what motivates recovering capabilities at the response level. The components are also not interchangeable: query-only routing yields only +0.25 pp versus +2.55 pp for response-level representations, and majority voting trails CapDyn-Match by 2.62 pp.
  • Component contributions: Representation-only in Table 3 already includes gating and fallback. Against Vanilla Linear, the full method gains +0.85/+1.06 pp, and the bootstrap 95% CIs for both the agreement and the gating components exclude zero. With six candidates, agreement adds +0.72 pp (nested CV) and +0.99 pp (LODO), with gains on all 14 held-out benchmarks.
  • Practical significance: CapDyn-Match outperforms the best single iteration by 2.90/2.34 pp and beats Model Soup and TIES by 2.28–3.48 pp. Median latency rises only from 4.36 s to 5.52 s. The selected responses can also serve as training data for future single-model consolidation.

If our responses have resolved your concerns, we would be sincerely grateful if you could consider updating your score. If any questions remain, we are happy to clarify right away before the deadline.

Thank you very much for your time. Best regards, The Authors

Reviewer_fFtTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper discuss self-evolving language model agents but how it may forget learning capabilities and no single iteration of the agent has full coverage (a problem they call historical capability fragmentation). To address this, they introduce CapDyn-Match which is a method that uses per-question selection to recover capabilities and evaluate across math, general reasoning, and code to improve question accuracy.

Critical strengths
  1. I think the motivation is quite strong and is demonstrated by the latest iteration not always having the best performance.
  2. Achieves strong performance when compared to the best individual iterations and majority votes. It doesn't need a judge which shows that instance-level selection can be very helpful.
Critical weaknesses

It performs recovery at inference time essentially using sets of supervised classifiers, requiring many iterations to run which seems to have a major scalability weakness. What demonstrates that further iterations wouldn't actually be better than this approach? Is there some sort of additional evolution vs comparative efficiency and their respective performance comparison (is there some point where performance can converge after self-evolution)?

Additionally, can improvements in the classifiers improve the performance? It's unclear if the contribution is really on capability recovery or just correctness prediction?

Questions for authors

I think it is not clear how you define a capability or loss of capabilities, especially at the beginning which could improve clarity.

Why does CapDyn-Match not have as good performance over highest iterations in some of the evaluation pools?

7 comments
AuthorsOpenReview ↗

Official Response to Reviewer fFtT (Part I)

We thank the reviewer for recognizing the motivation and performance improvements of our work. Below, we address your concerns regarding capability recovery, continued self-evolution, and deployment costs with additional experiments and discussion. All revisions in the manuscript are highlighted in blue.

C1: Scalability and Comparison with Continued Self-Evolution

Response:

  • (1) Continued evolution does not outperform recovery. In the six-iteration R-Zero 1.7B experiment (§4.4, Figure 5), 12 of 14 benchmarks peak before Iter6 (Appendix F.7, Table 15). On MATH500/MMLU-Pro, accuracy even drops from 68.60%/36.88% (Iter1) to 65.40%/33.82% (Iter6), while CapDyn-Match (LODO) reaches 71.60%/40.58%, i.e., +6.20/+6.76 pp over Iter6 (Table 16).
  • (2) Further iterations reshuffle rather than consolidate capabilities. Between adjacent iterations, 9.65%–12.58% of instances change correctness (Appendix, Table 6), and the historical union exceeds every individual checkpoint on all 14 benchmarks (Table 1). Additional iterations therefore keep redistributing capabilities across checkpoints instead of accumulating them in the latest one.
  • (3) Recovery is far cheaper. Each extra iteration costs 113–252 GPU-hours, whereas CapDyn-Match only fits lightweight probes. With parallel inference on 4 A100-40GB GPUs, median latency rises only from 4.36 s to 5.52 s per question(Table 5).
TrajectoryGPU ConfigurationGPU-hours per Iteration
R-Zero 4B8×H20-96GB252
Agent0 4B8×H20-96GB196
R-Zero 1.7B8×A100-40GB113

We acknowledge that inference resources grow with the number of retained checkpoints. Still, CapDyn-Match trades modest inference overhead for reliable recovery, avoiding costly retraining with no guaranteed gains.

C2: Recovery Mechanism and Selector Strength

Response: To separate the roles of historical capability recovery and correctness prediction, we compare CapDyn-Match with two LLM selectors, Qwen3-32B and Kimi-K3, on the same historical candidate pool (Appendix F.3, Table 13; best in bold):

BenchmarkAgent0 LatestAgent0 CapDyn-MatchAgent0 Qwen3-32BAgent0 Kimi-K3R-Zero LatestR-Zero CapDyn-MatchR-Zero Qwen3-32BR-Zero Kimi-K3
GSM8K91.2193.0393.7894.1691.8992.5292.9593.93
MATH50077.2081.1380.8082.2078.0080.0780.0082.40
Minerva59.1958.7058.8760.1357.3560.5460.6663.60
Olympiad44.5949.3848.7850.2642.5248.2547.4150.30
AMC2348.8352.6356.2257.5053.2056.3052.0857.50
AIME2412.4015.5215.2116.3318.2319.5518.3320.42
AIME256.3512.6716.9117.002.718.446.2512.45
Avg.48.5451.8752.9453.9449.1352.2451.1054.37
  • (1) Historical checkpoints retain recoverable capabilities. All three selectors outperform the latest checkpoint on nearly all benchmarks. On MATH500, the Oracle upper bound reaches 84.60%/85.40%, well above the latest 77.20%/78.00%, so the gains stem from genuine historical capabilities.
  • (2) Selector accuracy determines recovery. Kimi-K3 performs best everywhere, raising MATH500 to 82.20%/82.40%, close to the Oracle. Thus, the bottleneck is correctness prediction, and our framework directly benefits from stronger selectors.
  • (3) CapDyn-Match balances efficiency and performance. Its lightweight probes match Qwen3-32B, winning 8 of 14 combinations with a higher R-Zero average (52.24 vs. 51.10). They also recover about 60% of Kimi-K3's gain over the latest checkpoint (+3.33/+3.11 vs. +5.40/+5.24 pp) without any large LLM or retraining.

Kimi-K3 is accessed via the official Moonshot API, as it is a 2.8T-parameter MoE model (104B activated) that cannot be deployed on our hardware, and it costs $3/$15 per million input/output tokens, whereas CapDyn-Match only needs linear probes on frozen hidden states and calls no external model.

Stronger LLM selectors can further improve recovery, but at substantial computation and memory costs. Our goal is a practical balance between recovery and efficiency, not maximal accuracy at any cost. When resources allow, our candidate pool can also be paired with stronger selectors.

AuthorsOpenReview ↗

Official Response to Reviewer fFtT (Part II)

C3: Defining Capability and Its Loss

Response: We would like to clarify that "capability" in our paper is an operational definition, not an intrinsic property of the model: it means successfully solving an instance under a specified evaluation protocol (§2.2, before Eq. (1)). Building on this definition, we formalize the related concepts as follows:

  • Capability set. For greedy evaluation, let CtC_t denote the set of instances correctly solved by checkpoint tt.
  • Capability loss. Capability loss is defined as Ct−1∖CtC_{t-1} \setminus C_t, i.e., the instances solved by the previous checkpoint but no longer solved by the current one.
  • Historical capability fragmentation. This refers to the complementarity among the sets of correctly solved instances {Ct}\lbrace C_t \rbrace across checkpoints: capabilities acquired at different rounds are scattered across different checkpoints rather than accumulated in the latest one.
  • Extension to mean@32 evaluation. For mean@32 evaluation, we compare candidate groups by rollout index and then compute question-level weighted averages (Appendix F.6).

C4: Causes of Local Negative Gains

Response: We thank the reviewer for pointing out this phenomenon, which helps us clarify where the gains of CapDyn-Match come from and where its applicability ends. First, we would like to clarify that CapDyn-Match is not designed to outperform the latest or best single round on every evaluation combination. Instead, it improves overall performance through per-question selection that exploits the complementarity of historical responses across rounds. Overall, CapDyn-Match outperforms the best single round by 2.90 and 2.34 percentage points on Agent0 and R-Zero, respectively, and outperforms majority voting by 2.62 and 2.19 percentage points (Table 3).

For the five combinations in Table 2 that fall below the best single round, we analyze them as follows:

  • Combinations with clear gains over the latest round (2/5). On Agent0 BigCodeBench-Hard and R-Zero Minerva, CapDyn-Match improves over the latest round by 1.57 and 3.19 percentage points, respectively, leaving gaps of only 0.45 and 0.12 percentage points to the best single round. This indicates that complementary historical responses are effectively exploited in these two combinations.
  • Combinations with negative local net gains (3/5). The performance drop in the remaining three combinations is mainly due to small pools of recoverable questions, where a small number of routing errors can offset the recovery gains. This does not mean that historical candidates are unhelpful. Rather, it shows that routing errors have an amplified impact when the recoverable pool is small. Detailed statistics are given below:
Self-Evolution FrameworkBenchmarkRecoverable PoolRecoveredCollateral Damage (Instances)
Agent0Minerva291112
Agent0BBEH1368
R-ZeroMBPP+725

Thank you for your response and most of my questions have been resolved. Please include the definition of capability earlier in the paper so readers understand what you are describing.

Reviewer_9smeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the problem of capability fragmentation during the self-evolution of LLM agents. During self-evolution, models gradually acquire the ability to solve new problems while potentially losing the ability to solve previously mastered ones. Through instance-level analysis, the authors find that instances exhibiting capability gains and losses show substantial overlap in highly responsive parameter rows. To address this issue, the authors propose CapDyn-Match, an inference-time response selection framework that does not require modifying model parameters. Specifically, the method first generates candidate responses using frozen historical checkpoints, then scores these candidates based on query–response representations and cross-iteration answer agreement, and finally selects the response through a margin-based gating mechanism. The authors further demonstrate the effectiveness of their approach across multiple benchmarks and model configurations.

Critical strengths
  1. Relatively In-depth Empirical Analysis. Although the analysis largely remains at the instance level without further investigating which specific capabilities have changed, the parameter-response analysis provides empirical evidence of the difficulty in distinguishing beneficial from harmful model updates, offering insights into capability changes during self-evolution.

  2. Simple and Practical Approach. CapDyn-Match leverages the complementary capabilities of different historical checkpoints to improve overall performance without modifying model parameters. The method is relatively simple and has practical value.

  3. Comprehensive Main Experimental Evaluation. The main experiments cover different model sizes and various types of tasks, providing relatively comprehensive evidence of the effectiveness of the proposed method.

Critical weaknesses
  1. Lack of Fine-Grained Analysis of Historical Capability Fragmentation. The paper characterizes capability gains and losses across different iterations through instance-level correctness transitions but does not sufficiently investigate the underlying causes of these changes. For example, it remains unclear whether changes in correctness result from improvements or degradation in reasoning ability, computational errors, response formatting, or other behavioral factors. Although the authors conduct a parameter-response analysis, it does not directly explain the causes of changes in generated responses.

  2. Insufficient Analysis of the Effectiveness of Last-Token Representations. The authors use the hidden state of the last token to represent the entire query–response pair. Although this representation can incorporate contextual information through attention, it remains unclear whether it is the most effective representation.

  3. Lack of Appropriate Metrics for Evaluating Historical Capability Recovery. The paper primarily demonstrates the effectiveness of the proposed method through improvements in overall accuracy. However, overall accuracy alone cannot directly quantify how much of the historically lost capability is actually recovered. More appropriate metrics are needed to directly evaluate the effectiveness of historical capability recovery.

Questions for authors
  1. Section 2 analyzes capability gains and losses across different iterations through instance-level correctness transitions. However, what are the underlying causes of these changes? Could the authors provide a more fine-grained analysis to distinguish whether these changes result from improvements or degradation in reasoning ability, intermediate computational errors, response formatting, or other factors?

  2. Section 3.1 uses the hidden state of the last non-padding token to represent the entire query–response pair. Although this hidden state can incorporate preceding contextual information through self-attention, it remains unclear whether this is the most effective representation for predicting response correctness. Could the authors compare different representation strategies, such as Last-Token Pooling, Mean Pooling, and Answer-Level Representation, to validate the effectiveness of the current design?

  3. CapDyn-Match requires maintaining multiple historical checkpoints and generating candidate responses from each of them. If the self-evolution process extends beyond 10 iterations, how should historical checkpoints be selected? The paper could further investigate this direction.

  4. The authors use an additional Shared Reference Encoder to re-encode responses generated by historical checkpoints to obtain unified response representations. However, the historical checkpoints already compute hidden states during response generation. Given that these checkpoints originate from the same self-evolution trajectory, their representation spaces may remain relatively well-aligned. Could the authors explain the necessity of using an additional Reference Encoder and compare the performance and computational overhead of directly using the hidden states from the generating checkpoints versus those obtained from the Shared Reference Encoder? Additionally, why is Iter2 selected as the Reference Encoder rather than other iterations?

4 comments
AuthorsOpenReview ↗

Official Response to Reviewer 9sme (Part I)

We thank the reviewer for the careful reading and positive feedback. We have revised the manuscript (changes marked in blue) and added four sets of evidence to clarify the nature of capability loss, its distinction from sampling instability, the limitations of parameter-level localization, and the recoverability of historical capabilities.

C1: Underlying Causes of Correctness Changes

Response:

(1) Degradation Stems from Reasoning and Computation, Not Formatting

We analyzed 473 correct-to-incorrect transitions across four mathematical benchmarks for Agent0 and R-Zero (Appendix C.3, Table 8):

Failure CategoryAgent0R-Zero
Intermediate calculation errors141135
Reasoning-path changes7779
Generation truncation56
Answer formatting errors34
Grader misjudgments1112
Total237236
  • Reasoning and computation dominate. Excluding grader misjudgments, intermediate calculation errors and reasoning-path changes together account for 96.5%/95.5% of genuine degradations (Agent0/R-Zero).
  • Formatting plays a negligible role. Answer formatting errors account for only 3/4 cases. These results indicate that capability loss mainly reflects reduced reliability in reasoning and computation, not superficial formatting issues.

(2) Degradation Is Not Merely Sampling Randomness

  • Resampling does not recover lost capabilities. For instances degraded from Iter2 to Iter3, four extra samples from Iter3 (temperature = 0.6) reach only 0.39 accuracy, versus 0.96 for consistently correct ones. Excluding grader misjudgments, 34.4%/40.7% (Agent0/R-Zero) stay wrong in all samples.
  • Historical checkpoints beat resampling at equal budget. With three responses per question, the historical pool reaches Pass@3 of 81.49%/80.77%, +4.24/+3.87 pp above resampling the latest checkpoint (77.25%/76.90%).
  • Manual inspection reveals systematic errors, including concept misuse, geometric reasoning errors, and consistently repeated wrong answers (Appendix C.3, Table 9):
TrajectoryHistorical Pool Pass@3Latest (Iter3) Pass@3Gain (pp)Paired Bootstrap 95% CI
Agent0 4B81.49%77.25%+4.24[3.40, 5.54]
R-Zero 4B80.77%76.90%+3.87[2.68, 4.42]
ProblemHistoricalLaterError
Given odd f(x)f(x) and even g(x)g(x), determine the parity of f(f(g(f(g(f(x))))))f(f(g(f(g(f(x)))))).Iter2: evenIter3: oddTreats the composition as odd, ignoring the even inner expression.
SS, PP, QQ are collinear with PP between SS and QQ; ray PRPR forms an acute angle with PQPQ. Given sin⁡∠RPQ=725\sin\angle RPQ = \frac{7}{25}, find cos⁡∠RPS\cos\angle RPS.Iter2: −2425-\frac{24}{25}Iter3: 725\frac{7}{25}Confuses cosine with sine.

Thus, degradation reflects genuine behavioral changes rather than sampling variability.

(3) Parameter-Level Diagnostics

We further analyzed parameter-response differences between Gain (incorrect-to-correct) and Damage (correct-to-incorrect) instances. On MATH500, the overlap between their top 1% most responsive parameter rows is 0.78–0.80, comparable to randomly grouped instances without correctness labels (0.78–0.79). Figure 7 in Appendix D.1 visualizes row-wise gradient responses and CTR activation changes for Agent0 on MATH500 and Minerva. Gain and Damage exhibit substantial overlap in both spaces, with clustering primarily reflecting dataset differences rather than correctness transitions.

C2: Last-Token Representations vs. Alternative Pooling Strategies

The last-token representation at Layer 23 performs best among the evaluated strategies(Appendix F.2, Table 11).We compared last-token pooling, mean pooling over all tokens, answer-level mean pooling, and representations from different layers. The Layer-23 last-token representation improves over Latest by +3.87/+4.07 pp (Agent0/R-Zero), outperforming all-token mean pooling (+3.34/+2.74 pp), answer-level mean pooling (+3.47/+2.34 pp), and other evaluated layers. These results support our current representation design.

The ablation was conducted on MATH500, GSM8K, Minerva, and Olympiad, with each representation strategy trained and evaluated separately, without gating or fallback. Results report improvements over Latest (pp) with 95% confidence intervals.

Representation StrategyAgent0 (pp [95% CI])R-Zero (pp [95% CI])
Layer 23, Last Token+3.87 [1.89, 5.28]+4.07 [1.90, 6.41]
All-Token Mean Pooling+3.34 [1.27, 4.61]+2.74 [1.40, 4.08]
Answer-Level Mean Pooling+3.47 [1.47, 5.61]+2.34 [0.20, 4.54]
Layer 18, Last Token+2.34 [−0.07, 4.81]+3.27 [1.07, 5.48]
Layer 32, Last Token+3.14 [0.87, 5.48]+2.74 [0.47, 5.08]
Layer 35, Last Token+3.51 [1.34, 5.82]+2.54 [0.27, 4.81]
AuthorsOpenReview ↗

Official Response to Reviewer 9sme (Part II)

C3: Checkpoint Selection Beyond 10 Iterations

Response: In short, retaining all checkpoints yields the most complementarity, but a small complementary subset already recovers most of it.

We extended R-Zero 1.7B to 10 iterations and compared four retention strategies:

  • Full-10: all checkpoints;
  • Greedy-3: three checkpoints greedily selected by coverage on an independent validation set;
  • Recent-2 + Best: the two latest checkpoints plus the historically best one;
  • Last-3: the three latest checkpoints.

Results.

  • Full-10 performs best overall, leading on 11 of 14 benchmarks and improving over Latest by +5.75/+8.50/+7.91 pp on MATH500/MMLU-Pro/HumanEval+.
  • All reduced strategies beat Latest on all 14 benchmarks.
  • Greedy-3 is the strongest reduced strategy, winning on 12/14 benchmarks (including all seven math benchmarks) and even slightly surpassing Full-10 on Minerva and LiveCodeBench.

Recommendation. Greedy-3 offers the best performance–cost trade-off, keeping only three checkpoints without retraining. Full-10 suits performance-critical settings, and Last-3 is a simple option when no validation data is available. Hence, CapDyn-Match scales to longer trajectories without retaining every checkpoint.

CategoryBenchmarkFull-10Greedy-3Recent-2 + BestLast-3
MathMATH500+5.75+5.47+3.73+4.47
MathGSM8K+6.24+5.71+5.63+4.60
MathAMC23+7.13+6.71+5.49+3.35
MathMinerva+3.59+4.67+2.59+4.55
MathOlympiad+6.24+5.09+4.69+3.71
MathAIME24+3.75+2.54+1.32+1.63
MathAIME25+5.54+3.91+3.53+3.84
GeneralBBEH+1.35+1.00+1.06+1.46
GeneralMMLU-Pro+8.50+7.49+6.64+5.39
GeneralSuperGPQA+6.24+5.88+5.83+5.20
CodeHumanEval++7.91+5.98+6.81+7.42
CodeMBPP++4.05+3.53+2.61+3.14
CodeBigCodeBench-Hard+2.14+1.90+1.13+1.35
CodeLiveCodeBench+1.71+1.75+1.65+1.72

C4: Choice and Overhead of the Shared Reference Encoder

Response: In short, the shared encoder improves performance but is not essential. We trained and evaluated each representation source separately (with gating and fallback) on MATH500, GSM8K, Minerva, and Olympiad(Appendix F.2, Table 12). The table reports gains over Latest (pp [95% CI]):

Representation SourceAgent0R-Zero
Shared Iter2+4.78 [2.27, 6.95]+4.57 [1.74, 6.59]
Shared Iter1+4.28 [2.14, 6.61]+3.34 [1.20, 5.54]
Shared Iter3+4.18 [2.20, 6.55]+3.87 [1.60, 6.21]
Checkpoint-Specific Hidden States+3.84 [1.54, 6.19]+3.34 [1.14, 5.54]
  • Not essential. Checkpoint-specific hidden states already yield +3.84/+3.34 pp with no extra encoding (latency: 5.29 s/question).
  • Modest overhead. Shared Iter2 raises the gains to +4.78/+4.57 pp, adding only 222 ms of encoding (latency: 5.51 s/question).
  • Why Iter2? As the middle checkpoint, Iter2 serves as a fixed reference so that all responses are compared in a unified space; we do not assume it is optimal. Empirically, it performs best, but all choices yield significant gains with overlapping CIs, showing that the method is robust to the choice of encoder.
AuthorsOpenReview ↗

Official Response to Reviewer 9sme (Part III)

Additional Question: Quantifying Historical Capability Recovery

Response: We agree that overall accuracy alone does not directly quantify recovery. We therefore introduce two instance-level metrics. Let CTC_T be the instances solved by the latest checkpoint, R=(⋃t<TCt)∖CT\mathcal{R} = \left(\bigcup_{t<T} C_t\right) \setminus C_T the recoverable instances, and SS the instances solved by CapDyn-Match:

  • Historical Recovery Rate (HRR) = ∣S∩R∣∣R∣\frac{|S \cap R|}{|R|}: the fraction of recoverable instances that are actually recovered.

  • Collateral Damage Rate (CDR) = ∣CT∖S∣∣CT∣\frac{|C_T \setminus S|}{|C_T|}: the fraction of latest-correct instances lost due to routing errors.

On the six-iteration R-Zero 1.7B trajectory, CapDyn-Match recovers 37.9%–61.1% of recoverable instances across all 14 benchmarks, while keeping CDR within 0%–4.7%. For example, on MATH500 it achieves an HRR of 49.0% with a CDR of only 3.6%, improving accuracy by +5.80 pp. Thus, CapDyn-Match recovers historical capabilities while largely preserving those of the latest checkpoint.

BenchmarkIter6 (%)CapDyn (%)Δ Latest (pp)HRR ↑CDR ↓
Math
MATH50065.4071.20+5.8049.0%3.6%
GSM8K80.5284.86+4.3456.8%1.5%
AMC2338.8344.06+5.2349.1%3.4%
Minerva41.1843.87+2.6937.9%4.7%
Olympiad29.4835.70+6.2245.7%3.6%
AIME2411.5615.42+3.8652.9%2.2%
AIME255.008.44+3.4454.1%0.0%
General
BBEH9.9111.16+1.2538.4%3.1%
MMLU-Pro33.8241.22+7.4061.1%3.4%
SuperGPQA17.5122.25+4.7449.9%3.3%
Code
HumanEval+59.7661.77+2.0148.5%2.6%
MBPP+57.9459.99+2.0543.9%1.9%
BigCodeBench-Hard4.736.91+2.1846.7%0.0%
LiveCodeBench15.0715.98+0.9153.3%2.2%
AuthorsOpenReview ↗

Kind Reminder: Discussion Period Ends Today

Dear Reviewer 9sme,

We apologize for writing again. The discussion period ends today, so we wanted to send one final reminder in case our earlier message was missed.

In short, our responses address all of your concerns, and the new results are in the revised manuscript (marked in blue):

  • Causes of correctness changes: Over 95% of genuine degradations come from calculation errors and reasoning-path changes, not formatting. They also cannot be recovered by resampling (Appendix C.3).
  • Representation choice: The Layer-23 last-token representation outperforms mean pooling, answer-level pooling, and other layers (Appendix F.2, Table 11).
  • Checkpoint selection beyond 10 iterations: Keeping only three greedily selected checkpoints recovers most of the gains and beats Latest on all 14 benchmarks.
  • Shared Reference Encoder: It is helpful but not essential, and it adds only about 222 ms per question (Appendix F.2, Table 12).
  • Recovery metrics: With the new HRR/CDR metrics, CapDyn-Match recovers 37.9%–61.1% of lost capabilities while keeping collateral damage within 4.7%.

If our responses have resolved your concerns, we would be sincerely grateful if you could consider updating your score. If any questions remain, we are happy to clarify right away before the deadline.

Thank you very much for your time.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers for their constructive feedback. We are encouraged that the reviewers recognized the value of our observations on historical capability dynamics (fFtT, PS76, 9sme), the simplicity and practicality of CapDyn-Match (9sme, fFtT), and the comprehensive evaluation across self-evolution frameworks and 14 benchmarks (9sme, PS76).

We have revised the manuscript (changes highlighted in blue) and conducted additional experiments to address the reviewers' concerns. The main findings are summarized below.

1. Understanding Capability Degradation (9sme, fFtT). We clarify the instance-level definition of capability loss and analyze 473 correct-to-incorrect transitions. Most genuine degradation events involve reasoning-path changes or intermediate calculation errors (95.5%–96.5%), rather than formatting issues. Under an equal three-response budget, historical checkpoints achieve higher Pass@3 than repeated sampling of the latest model (+4.24/+3.87 pp), indicating that historical complementarity cannot be explained by sampling variability alone.

2. Historical Capability Recovery and Method Design (9sme, PS76, fFtT). We introduce Historical Recovery Rate (HRR) and Collateral Damage Rate (CDR) to directly quantify recovery and harmful routing. Additional ablations show that query–response representations account for most of the gains, while agreement scoring and conservative gating provide further improvements, particularly with more historical candidates. Stronger LLM selectors further improve recovery, confirming that historical checkpoints retain correct answers whose utilization depends on selection quality.

3. Scalability and Computational Efficiency (9sme, fFtT, PS76). Experiments extending self-evolution to 10 iterations show that retaining only three complementary checkpoints can recover substantial gains. Further self-evolution does not guarantee improvement and requires 113–252 GPU-hours per additional iteration. In contrast, CapDyn-Match requires no foundation model retraining and increases median latency from 4.36 to 5.52 s/question with parallel inference, trading additional inference resources for capability recovery.

4. Contributions and Limitations (PS76, 9sme). Our contribution is not a new ensemble algorithm, but revealing historical capability fragmentation and demonstrating its practical recoverability. Parameter-response analyses, including random-label controls and PCA visualizations, show that beneficial and harmful changes exhibit highly overlapping response patterns, motivating response-level recovery. We also clarify that CapDyn-Match does not consolidate historical capabilities into a single model; SFT or distillation-based consolidation remains future work.

Detailed results and corresponding manuscript revisions are provided in our responses below.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.