acceptodds
Under review as a conference paper at ICLR 2027

Agent Plasticity: Measuring Self-Improvement Through Experience

18 views

Abstract

AI agents are increasingly being used in systems requiring them to self-diagnose and improve through experience, yet evaluations largely measure what they can do, not how effectively they learn. We study persistent self-improvement with frozen model weights, where agents learn by constructing and refining tools, skills, memories, and strategies inherited by future instances. Despite comparable learning opportunities, frontier models differ sharply in their ability to self-improve with experience, and gains on held-out in-distribution games often carry over only partly to harder, out-of-distribution games. We introduce agent plasticity, the efficiency of an agent in improving future performance as it gets more experience. Tracing failures through the improvement loop, we find that weaker agents often fail to reuse relevant artifacts, while highly plastic agents often fail despite reusing them, highlighting a shift in the bottleneck from reusing artifacts to building generalizable artifacts and applying them effectively. These results establish plasticity as a dimension of long-lived agent capability that static evaluations overlook.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Self-Improvement, Plasticity, Agents

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_opQbOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates how effectively language-model agents improve through experience while their model weights remain fixed. It introduces agent plasticity, which measures performance improvement relative to cumulative learning expenditure, together with a saturation-based metric for summarizing learning efficiency. The evaluation protocol alternates between acting and reflection: agents use training experience to update a persistent inventory of tools, code, instructions, and memory, while subsequent acting episodes begin with fresh contexts. In the board-game experiments, held-out evaluation trajectories and scores are withheld from reflection. Experiments across chess, Go, Hex, and NetHack reveal substantial variation in improvement across models and tasks, including pronounced gains, stagnation, and regressions following inventory updates. Beyond reporting learning curves, the paper analyzes persistent artifacts and execution traces to distinguish failures without a covering artifact, failures where an available artifact is unused, and failures despite artifact use. These analyses help characterize how agents translate accumulated experience into subsequent behavior. The main contribution is an evaluation framework and empirical study of persistent agent adaptation, emphasizing learning efficiency and adaptation dynamics alongside initial task performance.

Critical strengths
  1. A clear and well-controlled protocol for evaluating persistent agent improvement. The paper addresses the important question of how effectively agents improve across interactions while their model weights remain fixed. Fresh acting contexts and explicit inventory inheritance establish a clear boundary between transient interaction history and persistent adaptation. Withholding held-out trajectories and scores from reflection strengthens the board-game evaluation, while including rejected reflection attempts in learning costs makes the expenditure accounting more complete. Together, these choices provide a coherent framework for examining improvement beyond initial capability (§§2.1–2.2 and §3.1).
  2. Informative longitudinal experiments supported by inspectable behavioral evidence. The experiments across chess, Go, Hex, and NetHack reveal substantial model and task dependence, including stagnation and harmful updates. Reporting full learning curves exposes these dynamics, such as the late Go regression following a tool rewrite (§3.3). The artifact analysis carefully distinguishes relevance, failure coverage, and observed use, grounding its judgments in execution traces and engine evidence. The guard-removal experiment in Figure 15 additionally provides concrete evidence that a particular learned artifact changes a subsequent decision.
Critical weaknesses
  1. The specific contribution of training feedback remains unquantified. The experiments measure improvement under the complete interaction-and-reflection process. As acknowledged in §§3.2 and 4, they do not separate the incremental benefit of training feedback from additional artifact-development computation. Figure 15 provides useful local evidence of a feedback-derived correction, but its contribution to the aggregate improvement remains unclear. This limits conclusions about the specific value of experience. The distinction between improvement under the complete protocol and the contribution of feedback should remain explicit throughout the framing; a targeted comparison with the same development budget would strengthen the latter interpretation.
  2. The statistical reporting supporting the headline plasticity comparisons is incomplete. Section 2.2 specifies the curve family, fitting criterion, and decision thresholds, but leaves important implementation details unclear, including the construction of the binomial likelihood, calibration of the flat-curve test, and sources of variability captured by the parametric bootstrap. Under A=0A=0, C50C_{50} and hh are unidentified, making the test calibration relevant to interpreting the significance gate. Figure 1 also presents plasticity estimates without uncertainty intervals. These omissions limit assessment of the precision of the reported comparisons, although the broad differences visible in the learning curves remain informative.
Questions for authors
  1. Beyond the local intervention in Figure 15, is there evidence quantifying the incremental contribution of training feedback relative to additional artifact-development computation? Has a comparison with the same development budget but without trajectory or outcome feedback been examined in any representative setting?
  2. How is the stated binomial likelihood constructed for chess scores that include half-point draws and for the equally weighted combined-game curve? How is the likelihood-ratio test against a flat curve calibrated, given that C50C_{50} and hh are unidentified when A=0A=0?
  3. How many independent adaptation lineages were evaluated for each board-game condition, and what sources of variation does the parametric bootstrap capture? Could the corresponding uncertainty intervals be reported for the plasticity estimates in Figure 1?
6 comments
AuthorsOpenReview ↗

Author Response (Part 1/3)

Thank you for your careful review and for recognizing how our protocol isolates persistent adaptation, accounts for rejected reflection attempts in learning cost, and demonstrates the causal effect of a learned artifact through the guard-removal experiment (Fig. 15). We address your concerns with the following new results:

Summary

(W1/Q1) Contribution of feedback relative to artifact-development compute: Across three Gemini models on chess and Hex, training feedback yields higher held-out performance than reflection alone (+3.8 to +21.7 points). Qualitative analysis shows that feedback directs reflection toward specific game-losing mistakes, whereas reflection alone predominantly optimizes code efficiency.

(W2/Q2) Statistical validity: We clarify the fractional-score fitting objective and replace the asymptotic χ² reference with simulation-based calibration. All 26 plasticity classifications remain unchanged.

(W2/Q3) Uncertainty intervals: For Figure 1, we replace the original bootstrap with 2,000 resamples that preserve draws, held-out-game identity across checkpoints, and independent lineages where available. Figure 1 now reports 90% bootstrap intervals, and we qualify model rankings accordingly.


W1/Q1: Contribution of training feedback

We ran the suggested reflection-only control. Both conditions start with empty artifact inventories and run 20 update rounds.

  • With feedback: The agent plays training games and reflects on trajectories, outcomes, and move-quality feedback, as in our original protocol.
  • Reflection only: No training games or task feedback are supplied during reflection. We restrict the model to see only its persistent artifacts (its inventory, including earlier notes).

We evaluate three recent Gemini models (Flash 3.8, Argon, Carbon; released recently, accessed via Google AI Studio) on chess and Hex, with two independent Argon runs per game.

Held-out ID score over checkpoints 0–20 and PsatP^{sat} (points per 1k USD), averaged across chess and Hex (and both Argon runs):

ModelWith feedbackReflection onlyPsatP^{sat}, with feedbackPsatP^{sat}, reflection only
Argon91.0%81.3%1,357396
Carbon92.5%88.8%3,0451,383
Flash 3.848.3%26.7%8630

All three models reach higher performance with feedback (3.8–21.7 points), although the benefit varies across models and tasks. Feedback also gives higher plasticity (PsatP^{sat}): 1,357 vs 396 points per 1k USD for Argon, 3,045 vs 1,383 for Carbon, and 863 vs 0 for Flash, whose reflection-only runs show no reliable rise.

What does feedback change?

1. Both conditions construct and reuse strong tools. Our failure/reuse analysis (§3.5, Appendix H), pooled over checkpoints 1–20 of the original Argon chess runs, shows:

Pooled over checkpoints 1–20With feedbackReflection only
Relevant-move reuse98.3%99.4%
Failures per relevant move0.0620.064
Failures with artifact used (FusedF_{used})88.3%90.3%
No covering artifact (FabsentF_{absent})10.6%9.2%
Covering artifact unused (FmissedF_{missed})1.2%0.5%

Argon resembles the strong improvers in our original study (Fable 5, Opus 5, Opus 4.8, GPT-5.6 Sol): 94–98% reuse, with 83–99% of remaining failures occurring despite artifact use. Unlike Sol and Opus 4.8, which begin below 20% reuse at checkpoint 1, surprisingly, Argon already reaches approximately 99%.

In both conditions, nearly all reuse involves executable tools; strategies and memory notes are used in approximately 1% of relevant moves. Thus, tool construction and reuse are not exclusive to learning from training games.

2. Feedback directs reflection toward task failures. We manually inspected reflection summaries from the two original Argon chess runs:

  • With feedback: 13/15 non-empty summaries address specific game mistakes, including missed mates, unsound sacrifices, and threefold repetitions while ahead. The agent also discovers task-specific rules, such as reaching 300 plies counting as a loss.
  • Reflection only: 9/14 summaries focus on profiling and speedups; the resulting engine is almost twice as large (260 versus 138 KB).

Both conditions develop capable engines. However, supplied feedback directs reflection toward mistakes that actually lose games, consistent with the higher chess score with feedback (95.8% versus 81.3%). Figure 15 provides complementary intervention evidence that a feedback-derived fix can affect subsequent behavior.

Conclusion: Supplied training feedback is associated with additional improvement and promotes reflection toward observed failures, a core requirement for agents that can learn from experience.

AuthorsOpenReview ↗

Author Response (Part 2/3)

W2/Q2: The fitting objective, draws, and combined curve

Draws. At checkpoint tt, we define the total score kt=Wt+Dt/2k_t=W_t+D_t/2 across ntn_t games. With pt=S^(ct)/100p_t=\hat S(c_t)/100, the fitted objective is

ℓ(θ)=∑t[ktlog⁡pt+(nt−kt)log⁡(1−pt)],\ell(\theta)=\sum_t\left[k_t\log p_t+(n_t-k_t)\log(1-p_t)\right],

together with weak N(0,32)\mathcal N(0,3^2) penalties on the baseline and asymptote logits. The generalized binomial coefficient, defined using gamma functions for fractional ktk_t, is constant with respect to the fitted parameters.

We agree that this is a fractional-score objective, not the true likelihood of win/draw/loss outcomes. For game score Y∈{0,12,1}Y\in\lbrace 0,\tfrac12,1\rbrace with mean pp,

Var(Y)=p(1−p)−14Pr⁡(D),\text{Var}(Y)=p(1-p)-\tfrac14\Pr(D),

so a Bernoulli likelihood does not correctly represent score variance when draws occur. The objective remains well-defined for fitting the mean performance curve, but we do not use its likelihood-based standard errors for uncertainty estimation.

Combined curve. Chess first averages independent lineages within each difficulty and weights Easy and Hard equally. Chess, Go, and Hex then receive equal weight.

The combined ktk_t equals this weighted mean score multiplied by the total number of games. This preserves our intended equal-task performance measure, but treating the weighted aggregate as a single binomial does not generally yield correctly calibrated uncertainty, since task weights need not match game-count proportions and outcomes include draws.

We therefore separate the fitting objective from uncertainty estimation and use outcome-level resampling for the latter. We clarify these choices in §2.2.


W2/Q2: Simulation-calibrated flatness test

We agree that under the null hypothesis of no rise (A=0A=0), the parameters C50C_{50} and hh are unidentified.

We therefore recalibrated the test by simulation. For each of the 26 board-game curves, we estimate each held-out game's win/draw/loss probabilities from its 21 checkpoint outcomes and hold these probabilities fixed across checkpoints. We generate 1k no-learning histories preserving the observed evaluation structure, game counts, learning costs, and task weights.

Each simulated history is refitted using the full procedure, including 15 initialization starts, parameter bounds, regularization, and dispersion adjustment. The simulated p-value is

psim=1+count(Tsim≥Tobs)1+Nsim.p_{sim}=\frac{1+\text{count}(T_{sim}\geq T_{obs})}{1+N_{sim}}.

All 26 final plasticity classifications remain unchanged when combining the recalibrated test with our separate 10-point minimum-rise criterion, although individual p-values change:

  • Combined Luna: pp: 0.235 → 0.077, but the fitted 90% rise is only 2.6 points, below the required 10 points; hence Psat=0P^{sat}=0.
  • Chess GPT-5.5: pp: 0.35 → 0.043, but the fitted rise is only 1.8 points, also below the minimum-rise criterion.

Thus, the classification depends on both statistical evidence and a minimum improvement magnitude, rather than significance alone.

Scope. The reference distribution is built from each game's own observed outcomes and reproduces the actual evaluation design (game counts, draws, learning costs and task weights), rather than relying on an asymptotic approximation; its independence assumption is stated in Appendix G.2. Under this calibration all 26 classifications are unchanged, showing that they are robust to how the no-rise test is calibrated. Where independent lineages exist, their variation is captured by the bootstrap intervals (W2/Q3).

AuthorsOpenReview ↗

Author Response (Part 3/3)

W2/Q3: Independent lineages and bootstrap uncertainty

Independent lineages. In chess, Sol, Opus 5, Opus 4.8, GPT-5.5, and Gemini 3.1 Pro have three independent lineages per difficulty; Fable, Opus 4.6, and Luna have one. Go, Hex, and NetHack each have one lineage per model. In the new feedback controls, each Argon condition has two lineages, while Flash and Carbon have one.

Limitations of the submitted bootstrap. Our original procedure sampled each checkpoint score from Binomial$(n_t,\hat p_t)$, held learning costs fixed, and refitted the curve 100 times. This captured only checkpoint-level game-outcome noise. It did not preserve draws, the same held-out game's outcomes across checkpoints, or independent-lineage variation; 100 resamples also provided coarse estimates of interval tails.

Revised bootstrap (Figure 1). We now report 90% intervals from 2,000 resamples. Each resample:

  • Resamples independent lineages where available (three per chess difficulty for five models).
  • Resamples held-out games by identity, preserving each sampled game's win/draw/loss outcome sequence across checkpoints.
  • Refits the full estimation pipeline, including the saturation rule.

This preserves the longitudinal evaluation structure and accounts for the variability represented by the available lineages and held-out games. For models with one lineage, the intervals reflect variation across held-out games.

The main conclusions remain: GPT-5.6 Sol and Opus 5 have the highest reported saturation-plasticity point estimates, with overlapping bootstrap intervals. Both intervals lie clearly above those of Fable 5 and Opus 4.8. We therefore describe Sol and Opus 5 as leading models without asserting a statistically established ordering between them. GPT-5.5 is reported separately using plasticity to date, which is not directly the same estimand as saturation plasticity.

As an additional robustness check, varying the saturation threshold to 85% or 95% leaves the combined ordering unchanged.

We have revised the ranking language in §3.2–3.4 and Finding 2 accordingly.


Paper revisions (in blue)

All changes in the revised manuscript are marked in blue:

  • Appendix M, §3.2, Discussion: Feedback-versus-reflection-only experiments, Argon failure/reuse analysis, and qualitative reflection comparisons (W1/Q1).
  • §2.2: Exact fractional-score fitting objective, regularization, combined-task weights, and clarification that likelihood-based uncertainty is not used (W2/Q2).
  • Appendix G.2, §2.2: Simulation-calibrated flatness test, its assumptions, and the separate 10-point minimum-rise criterion (W2/Q2).
  • Figure 1, §2.2: New 2,000-resample bootstrap intervals preserving held-out-game identity and independent lineages where available, with explicit lineage counts (W2/Q3).
  • §3.2–3.4, Finding 2: Qualified model rankings where bootstrap intervals overlap (W2/Q3).

We are grateful for the reviewer's careful and constructive feedback. We hope these additions answer the questions raised, and we would appreciate it if the reviewer would take them into account in the final assessment. We are happy to provide further analyses during the discussion period.

Reviewer_N1HvOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies how effectively agents improve through experience while keeping model parameters fixed. The paper introduces agent plasticity, which measures performance improvement per unit of learning cost. Agents play games, reflect on their training trajectories, and construct persistent tools, strategies, and memories that are inherited by future instances. Experiments on chess, Go, Hex, and NetHack show substantial differences across models and tasks. The paper further analyzes whether failures arise because useful artifacts are absent, ignored, or used unsuccessfully. The results suggest that endpoint performance and the efficiency of acquiring improvements capture different aspects of agent capability.

Critical strengths
  • The paper studies an important aspect of agent capability: whether performance improves with experience, and at what cost. The distinction between endpoint performance and learning efficiency is useful and supports the value of the proposed evaluation perspective.

  • The board-game protocol uses fresh contexts, frozen model weights, and held-out evaluation. Together with experiments across multiple models and four environments, including regressions and unsuccessful adaptation, this provides useful evidence beyond selected success cases.

Critical weaknesses
  • It’s not clear whether the gains establish learning from experience. Reflection receives both task feedback and additional computation that can be spent developing a stronger game-playing program. Section 4 acknowledges this issue, but the experiments do not separate the two effects. Without a budget-matched artifact-development control that omits trajectory feedback, the central attribution remains uncertain.

  • The reliability of the plasticity comparisons is not sufficiently established. Hard chess uses only 12 held-out ID games per checkpoint, and NetHack has one lineage per model with changing game difficulty. Moreover, the metric fits a monotone saturation curve and permits extrapolation to three times the observed cost despite observed regressions. These choices could materially affect the model ordering, so the current scalar rankings appear stronger than the evidence supports.

Questions for authors

Please refer to the weaknesses.

4 comments
AuthorsOpenReview ↗

Author Response (Part 1/2)

Thank you for your thoughtful review and for recognizing our held-out evaluation protocol and experiments across four environments. We have addressed both concerns with new experiments and statistical analyses:

Summary

(W1) Experience vs. reflection-only: Across three Gemini models on chess and Hex, experience yields higher held-out performance (+3.8 to +21.7 points). Our analysis further shows that experience directs reflection toward game-losing mistakes, whereas reflection alone primarily optimizes code efficiency.

(W2) Reliability of Plasticity comparisons: All 26 board-game plasticity classifications remain unchanged under simulation-calibrated statistical tests, and the combined model ordering is stable across 85%/90%/95% as saturation thresholds for performance improvement curves. We additionally report 90% bootstrap intervals and qualify rankings where intervals overlap.


W1: Learning from experience versus additional development compute

We ran the suggested reflection-only control. Both conditions start with empty artifact inventories and run 20 update rounds.

  • With experience: The agent plays training games and reflects on trajectories, outcomes, and move-quality feedback, as in our original protocol.
  • Reflection only: No training games or task feedback are supplied during reflection. We restrict the model to see only its persistent artifacts (its inventory, including earlier notes).

We evaluate three recent Gemini models (Flash 3.8, Argon, Carbon; released recently, accessed via Google AI Studio) on chess and Hex, with two independent Argon runs per game.

Held-out ID score over checkpoints 0–20 and PsatP^{sat} (points per 1k USD), averaged across chess and Hex (and both Argon runs):

ModelWith experienceReflection onlyPsatP^{sat}, with experiencePsatP^{sat}, reflection only
Argon91.0%81.3%1,357396
Carbon92.5%88.8%3,0451,383
Flash 3.848.3%26.7%8630

All three models reach higher performance with experience (3.8–21.7 points), although the benefit varies across models and tasks. Experience also gives higher plasticity (PsatP^{sat}): 1,357 vs 396 points per 1k USD for Argon, 3,045 vs 1,383 for Carbon, and 863 vs 0 for Flash, whose reflection-only runs show no reliable rise.

What does experience change?

1. Both conditions construct and reuse strong tools. Our failure/reuse analysis (§3.5, Appendix H), pooled over checkpoints 1–20 of the original Argon chess runs, shows:

Pooled over checkpoints 1–20With experienceReflection only
Relevant-move reuse98.3%99.4%
Failures per relevant move0.0620.064
Failures with artifact used (FusedF_{used})88.3%90.3%
No covering artifact (FabsentF_{absent})10.6%9.2%
Covering artifact unused (FmissedF_{missed})1.2%0.5%

Argon resembles the strong improvers in our original study (Fable 5, Opus 5, Opus 4.8, GPT-5.6 Sol): 94–98% reuse, with 83–99% of remaining failures occurring despite artifact use. Unlike Sol and Opus 4.8, which begin below 20% reuse at checkpoint 1, surprisingly, Argon already reaches approximately 99%.

In both conditions, nearly all reuse involves executable tools; strategies and memory notes are used in approximately 1% of relevant moves. Thus, tool construction and reuse are not exclusive to learning from training games.

2. Experience directs reflection toward task failures. We manually inspected reflection summaries from the two original Argon chess runs:

  • With experience: 13/15 non-empty summaries address specific game mistakes, including missed mates, unsound sacrifices, and threefold repetitions while ahead. The agent also discovers task-specific rules, such as reaching 300 plies counting as a loss.
  • Reflection only: 9/14 summaries focus on profiling and speedups; the resulting engine is almost twice as large (260 versus 138 KB).

Both conditions develop capable engines. However, supplied experience directs reflection toward mistakes that actually lose games, consistent with the higher chess score with experience (95.8% versus 81.3%).

Conclusion: Supplied task experience is associated with additional improvement and promotes reflection toward observed failures, a core requirement for agents that can learn from experience.

AuthorsOpenReview ↗

Author Response (Part 2/2)

W2: Reliability of plasticity comparisons

We agree that performance estimates and model comparisons require explicit uncertainty quantification. We have added 90% bootstrap intervals, simulation-calibrated no-rise tests, saturation-threshold sensitivity checks, and additional trajectory-level measures.

1. Evaluation size and independent lineages

Hard chess evaluates 12 held-out games per checkpoint, making individual scores noisy. Figure 1 additionally aggregates Easy chess, Go (20 games), and Hex (40 games), fitting all 21 checkpoints.

Five models (Sol, Opus 5, Opus 4.8, GPT-5.5, Gemini 3.1 Pro) have three independent chess lineages per difficulty. Fable, Opus 4.6, and Luna have one due to evaluation cost (for Fable) or weak performance; Go and Hex also have one per model, to manage cost.

Figure 1 now includes 90% bootstrap intervals over independent lineages and held-out games. We clarify the resampling procedure and lineage counts (§2.2), and qualify rankings wherever intervals overlap (e.g., “GPT-5.6 Sol and Opus 5 lead” in §3.2–3.4 and Finding 2). The intervals of GPT-5.6 Sol and Opus 5 overlap each other but lie clearly above those of all other models.

Where models have three lineages, the intervals also capture run-to-run variation.

2. Robustness of plasticity classifications

We recalibrated the no-rise test using 1k simulated no-learning histories per curve, each refitted with the complete procedure. None of the 26 board-game classifications changes.

Varying the saturation threshold to 85% or 95% leaves the combined ordering unchanged.

Combined-curve results:

ModelNo-rise test p: χ² → simulatedClassification (unchanged)PsatP^{sat} at 85% / 90% / 95%
GPT-5.6 Sol<0.001 → 0.001Saturated351 / 298 / 261
Opus 5<0.001 → 0.001Saturated233 / 233 / 233
Fable 5<0.001 → 0.001Saturated68 / 57 / 43
Opus 4.80.042 → 0.003Saturated (extrapolated)14.4 / 14.4 / 14.0
GPT-5.5<0.001 → 0.001Rising, not saturatednone (plasticity to date: 34)
GPT-5.6 Luna0.235 → 0.077No reliable rise (2.6 points)0 / 0 / 0

The minimum attainable simulated p-value is 0.001. Luna's fitted rise remains below our 10-point minimum despite its simulated p-value of 0.077.

These checks show that the plasticity classifications are robust to how the no-rise test is calibrated, and that the combined model ordering is robust to the choice of saturation threshold.

3. Extrapolation and non-monotonic learning

Only two combined-curve fits have saturation crossings beyond observed costs:

  • Opus 4.8: The fitted crossing (1.07k USD) is close to the observed 819 USD, within our permitted extrapolation range.
  • GPT-5.5: The crossing exceeds the three-times-cost limit; we report plasticity to date (34), not PsatP^{sat}.

Because monotone fits summarize overall improvement but cannot represent late regressions, we retain full raw curves and additionally report endpoint and late-checkpoint plasticity (Appendix G.2, Table 6).

For example, Opus 5 in Go falls from 50% to 0% at checkpoint 20. Manual inspection traces this to an engine rewrite calling a function unavailable in the actor sandbox.

4. NetHack seed sensitivity

Each model has one lineage. All six models play the same 10 seeds within each checkpoint, enabling paired model comparisons, while seeds change across checkpoints (§3.3). The character remains a lawful dwarf female Valkyrie; dungeon generation and random events vary.

We tested whether particular seeds are consistently harder across models. Seed identity explains 18% of within-checkpoint score variance, compared with 17% under permutation (p = 0.24); cross-model per-seed scores are essentially uncorrelated (mean Spearman ρ = 0.02).

We therefore find no detectable common seed-difficulty effect, although 10 seeds per checkpoint cannot exclude smaller effects or fully characterize learning-trajectory variance.

Our original submission shows that surprisingly, only Opus 5.5 (the most recent model when we performed evaluations) improves; we do not rank the other five models.


Paper revisions (in blue)

All changes are marked in blue:

  • Figure 1, §2.2, §3.2–3.4: 90% bootstrap intervals, explicit lineage counts/resampling, and qualified rankings (W2).
  • Appendix G.2: 85/90/95% threshold sensitivity and simulation-calibrated no-rise tests; §2.2 and Fig. 1 clarify saturation/extrapolation (W2).
  • Appendix M, §3.2, Discussion: Experience-versus-reflection control, Argon failure/reuse diagnostics, and qualitative analysis (W1).
  • Appendix G.2, Table 6: Endpoint and late-checkpoint plasticity, with §2.2 pointer (W2).
  • §3.3: NetHack seed limitations and seed-effect analysis (W2).

Thank you again for the constructive review. We hope the new experiments address your concerns and would be grateful if you would consider them in your final score. We are happy to run further analyses during the discussion.

AuthorsOpenReview ↗

Follow-up on Author Response

Dear Reviewer N1Hv,

Thank you again for your thoughtful review. As the author–reviewer discussion period is coming to an end, we wanted to follow up on our response. In brief, we ran the reflection-only control you suggested, which shows that training experience gives higher held-out performance and higher plasticity for all three models tested. We also strengthened the statistical analysis with 90% bootstrap intervals in Figure 1 and simulation-calibrated testing, under which all 26 board-game plasticity classifications remain unchanged. Our general response, "Summary of Revisions", gives an overview of all changes.

Please let us know if any concerns remain; we would be happy to clarify or provide further analysis. If our response addresses your concerns, we would be grateful if you would consider updating your score.

Best regards,
The Authors

AuthorsOpenReview ↗

Follow-up and New Results

Dear Reviewer N1Hv,

Thank you again for your review. We wanted to share a further result and to follow up on our response.

New results (updated Figure 1). Your thoughtful comments prompted additional experiments during the discussion period, and these gave us new results that we have added to Figure 1: three recent Gemini models, evaluated on the same four games (Easy and Hard chess, Go, Hex). These are early results (two independent runs per game for Gemini 4 Argon, one for the other two models); complete three-run results will be presented in the camera-ready version, and the paper will be updated accordingly.

ModelPsatP^{sat}
Gemini 4 Carbon634
Gemini 4 Argon486
GPT-5.6 Sol298
Opus 5233
Gemini 3.8 Flash213

Figure 1 also has two new panels, showing plasticity by provider and by release date. Surprisingly, for all three providers (OpenAI, Anthropic, and Google), plasticity increases with release date across the models shown (the panels omit GPT-5.6 Luna, which shows no reliable rise). With few models per provider, we present this as a descriptive trend. However, for all three providers and the models we tested, plasticity is improving as time passes, whether it is emergent or explicitly optimized for. This itself makes plasticity an important metric for tracking progress in self-improvement.

These additions complement our earlier response to your two concerns: the reflection-only control (Appendix M), in which training experience gives higher held-out performance and higher plasticity for all three models tested, and the 90% bootstrap intervals and robustness checks for the plasticity comparisons (Figure 1, Appendix G.2).

As the discussion period is closing, we wanted to check whether our response has addressed your concerns, or whether there are other clarifications or experiments you would like to see. If your assessment of the paper has changed after the rebuttal, we would be grateful if you could update your review.

Thank you again for your time. We would be very happy to engage further on any questions or comments.

Best regards,
The Authors

Reviewer_QgfpOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper defines agent plasticity as the efficiency with which an agent turns experience into future performance gains through persistent artifacts. Experiments on Chess, Go, and Hex show substantial differences in plasticity across frontier models, with limited transfer to harder out-ofdistribution games. Weaker agents struggle to reuse relevant artifacts; more plastic agents reuse them, but their gains remain limited by how well the artifacts generalize and how effectively they are applied.

Critical strengths
  1. Shifting agent evaluation from "static capability" to "learning efficiency" addresses an important and underexplored dimension.
  2. Distinguishing between "failure to reuse artifacts" and "reusing artifacts that lack generalizability" provides a useful framework for understanding bottlenecks at different capability levels.
Critical weaknesses
  1. Evaluation is limited to three board games, all of which are highly structured (perfect information, deterministic rules, self-play possible). This severely limits the generalizability of conclusions. "Agent plasticity" as a general concept requires broader task coverage to support claims of universality.
  2. P = Delta S/C is essentially "performance gain per unit cost." Is this sufficient to capture the richness of "plasticity"? Does the paper analyze learning curve shape (e.g., whether there are phase transitions), artifact quality evolution, or the relative contributions of different artifact types (tools vs. memories vs. strategies)?
Questions for authors
  1. What is the criterion for determining the saturation point? Is a statistical test or preset threshold used? For the claim that "GPT-5.5 is not yet saturated," what is the evidence?
  2. Does the learning cost (USD) calculation account for differences in token pricing across models? If GPT-5.6's per-token cost is twice that of Claude Opus 5, does this affect the fairness of cross-model comparison?
  3. Have you attempted to validate the plasticity concept on non-board-game tasks (e.g., code generation, web navigation)?
  4. What is the concrete form of the artifacts? Has there been any manual inspection of artifact quality?
6 comments
AuthorsOpenReview ↗

Author Response (Part 1/3)

Thank you for your thoughtful review and for recognizing learning efficiency as an important, underexplored dimension of agent evaluation, as well as the value of our artifact-reuse and failure analyses. We have addressed your concerns with the following new results:

Summary

(W1/Q3) Generalization beyond games: We extend our evaluation to web navigation across three WebArena websites and multiple Gemini models. We measure plasticity, and our failure analysis reveals a distinct bottleneck: 74–85% of web-navigation failures lack a covering learned artifact, unlike the missed-reuse bottleneck prominent in weaker chess agents.

(W2/Q4) Learning and artifact dynamics: We expand and point to our analysis of learning-curve shapes, artifact reuse, effectiveness, and evolution. We show that increased artifact reuse does not necessarily translate into performance improvement, and provide additional manually inspected examples of successful, unsuccessful, and repaired artifacts.

(Q1/Q2) Metric robustness and cost fairness: We verify that the combined model ordering remains unchanged across 85%/90%/95% saturation thresholds and that all 26 board-game classifications remain unchanged under simulation-calibrated testing. We also introduce token-normalized plasticity, showing that the top two models reverse order relative to dollar-based efficiency.


W1 / Q3: Generalization beyond board games

Our original study already includes NetHack (§3.3, Fig. 5, Appendix D), a long-horizon, partially observed, procedurally generated single-player environment. Notably, only the recent Opus 5.5 exhibits positive plasticity on NetHack, further motivating plasticity as a task-dependent evaluation metric.

Why games? Chess, Go, and Hex allow agents to play against fixed engines (Stockfish, GNU Go, and a native Hex engine), providing controlled in-distribution/out-of-distribution held-out splits, adjustable opponent strength, and engine-graded mistakes. Together with partially observable environments such as NetHack, these enable a controlled study of learning efficiency and its failure modes. As with the historical role of games in RL for designing evaluation and training algorithms, we view this as a foundation for studying broader real-world agentic tasks for measuring learning from experience.

New web-navigation experiments (Appendix L). Following your suggestion, we evaluate three Gemini models on GoBrowse tasks across three WebArena websites: ShoppingAdmin, GitLab, and Reddit. We retain the paper's improvement protocol, running 10 updates per model/site and measuring performance gain and learning cost. Web agents typically show smaller rises within ten updates, so we drop the 10-point minimum-rise condition to compare plasticity across models.

ModelGainPlasticity
3.5 Flash-Lite+6.8724
3.6 Flash+2.160
3.8 Flash+3.385

Flash-Lite, the weakest model, achieves the highest plasticity on every site. Although absolute improvements are modest, these results demonstrate that the protocol and efficiency metric extend beyond games, and that learning efficiency need not track initial capability.

New failure and artifact-reuse analysis. We additionally apply our failure-analysis protocol (Appendix H) to Flash-Lite and 3.6 Flash:

MetricFlash-Lite3.6 Flash
Updates changing artifacts29/3021/30
Success on analysed tasks, checkpoints 0→5→1051→51→57%82→79→82%
Decision-level reuse, checkpoints 5→1011→10%29→49%
Missed reuse, checkpoints 5→1089→90%71→51%

Failure attribution (checkpoints 5 and 10):

Failure categoryFlash-Lite3.6 Flash
No covering artifact (FabsentF_{absent})195 (85%)70 (74%)
Covering artifact unused (FmissedF_{missed})33 (14%)19 (20%)
Artifact used, failure persisted (FusedF_{used})0 (0%)6 (6%)

The most frequent failures across both models and all checkpoints were repeated ineffective actions (44%, e.g., entering a search without submitting), premature termination or false success claims (21%), incorrect clicks/inputs (13%), navigation errors (9%), and incorrect final answers (5%).

Three key findings emerge:

  • More reuse does not necessarily yield improvement. 3.6 Flash's reuse increased from 29% to 49% without improving success, whereas Flash-Lite improved despite approximately 10% reuse.
  • Most failures lack covering artifacts. For Flash-Lite and 3.6 Flash, respectively, 85% and 74% of failures had no applicable artifact. Across relevant decisions, successful outcomes were observed in 70% of cases with reuse versus 49% with missed reuse. This association does not establish causality.
  • Systematic failures persist. Ineffective repeated actions and premature termination account for 65% of failures, with little reduction in overall failure counts.
AuthorsOpenReview ↗

Author Response (Part 2/3)

W1 / Q3: Generalization beyond board games (continued)

Significance beyond games. Performance curves alone reveal whether an agent improves; our analysis helps explain where improvement breaks down. In chess, missed reuse is prominent among weaker agents, while more capable frontier agents can fail even after reuse, potentially reflecting poor artifact generalization. On web tasks, the dominant bottleneck is instead failure to construct artifacts covering encountered errors. This suggests that improving reflection and the coverage/generalizability of learned artifacts may be more important than retrieval alone. The framework identifies these distinct bottlenecks across environments.


W2: Is performance gain per unit cost sufficient?

We agree that a single scalar cannot fully characterize learning. We propose PsatP^{sat} as a measure of learning efficiency to be evaluated alongside raw capability, complete learning curves (Figs. 1 left, 2b, 3, 9), and systematic artifact/failure diagnostics. The revision expands these analyses (Appendix G.5):

1. Learning-curve shape. Raw trajectories exhibit rapid early gains and, in some cases, late regressions. The fitted curves also differ substantially in steepness: on the combined curve, the reported factors for costs spanning 10%–90% of the fitted improvement are 1.7 (Opus 5), 5.3 (GPT-5.6 Sol), and 7.8 (Fable 5). These dynamics are not captured by a single gain-per-cost number.

2. We studied artifact-reuse dynamics. For every model in chess, Go, and Hex, we track decision-level reuse, identified failures, and their composition across checkpoints (Fig. 7; Appendices I, J).

  • Reuse can precede improvement: In Hard chess, GPT-5.6 Sol's reuse rises from 19.5% at checkpoint 1 to 99.7% at checkpoint 3, when its score is still only 4.2%. Performance peaks at 41.7% at checkpoint 12.
  • Reuse can increase despite regression: In Go, Opus 4.8's successful-reuse share rises from 59–65% to 93%, while held-out ID performance drops from 40% initially to 10–25% at checkpoints 15–20.

3. Artifact effectiveness over time. Appendix Fig. 33 (chess) and Figs. 41–42 (Go/Hex) track reuse outcomes. The proportion of artifact uses labeled successful on held-out chess games changes as follows:

Model / environmentCheckpoints 1–5Checkpoints 16–20
GPT-5.6 Sol, Hard chess82.0%89.1%
GPT-5.6 Sol, Easy chess85.4%92.0%
GPT-5.5, Hard chess77.1%86.4%
Opus 4.8, Hard chess89.0%85.6%

Effectiveness increases for GPT-5.6 Sol (within both tools and strategies) and GPT-5.5, but decreases for Opus 4.8. Figure 8 additionally demonstrates fixes persisting after failures. These measures track observed reuse effectiveness.

4. Artifact composition. Among moves involving artifacts (training and held-out, checkpoints 1–20), tools appear in 89.6% of Hard chess moves, 99.4% of Go moves, and 99.3% of Hex moves; strategies appear in 68.0%, 9.2%, and 19.3%, respectively. Memory appears in at most 3%, while instructions appear in 8.8% of Hard chess moves. Categories overlap because decisions may use multiple artifact types.

Together, these analyses distinguish improvement speed, artifact adoption, effectiveness, and failure modes: dimensions that PsatP^{sat} alone cannot capture.


Q1: Saturation criterion and GPT-5.5

We fit Eq. 4 to held-out scores versus cumulative learning cost. Saturation cost C∗C^* is the cost at the first observed checkpoint at which the fitted curve reaches 90% of its total improvement. We set Psat=0P^{sat}=0 if the fitted improvement is below 10 points or statistically indistinguishable from flat (p≥0.10p\geq 0.10). If the crossing is not observed, extrapolation is permitted up to three times the final observed cost; beyond that, saturation is not established.

GPT-5.5 is not considered saturated. Its combined fitted curve reaches 90% at approximately 40k USD, versus only 414 USD observed learning cost and a 1.2k USD extrapolation limit. Its performance is still rising at checkpoint 20. We therefore report plasticity to date (34 points/1k USD) rather than PsatP^{sat} (lighter bar, Fig. 1). The 40k USD figure is an extrapolation of the fit, not a predicted training budget. Opus 4.8 also requires extrapolation, but only from 819 USD to approximately 1.07k USD.

Robustness checks (Appendix G.2). Repeating the analysis at 85%, 90%, and 95% preserves the combined saturated-model ordering (GPT-5.6 Sol > Opus 5 > Fable 5 > Opus 4.8 > GPT-5.6 Luna, with GPT-5.6 Luna at zero); GPT-5.5 remains unsaturated throughout. Recalibrating the flatness test using 1,000 refitted no-change simulations per curve leaves all 26 board-game classifications unchanged.

AuthorsOpenReview ↗

Author Response (Part 3/3)

Q2: Provider prices, fairness, and token-normalized plasticity

Our dollar-denominated plasticity intentionally measures economic learning efficiency, incorporating the recorded token usage of both training games and reflection at each provider's rates. Dollar cost is a practical, although imperfect, proxy for resource consumption: in real-world deployments, including coding agents, improvement achieved per dollar is directly relevant to users working with these agents.

New per-token analysis (Appendix G.1). We refit the learning curves using token consumption instead of dollar cost:

ModelImprovement per 100M learning tokens
Opus 5146.5
GPT-5.6 Sol125.0
Fable 571.4
Opus 4.87.3

The top two models reverse order relative to dollar-normalized plasticity, demonstrating that pricing affects the ranking. We therefore provide both views. Token counts are not equivalent measures of compute across architectures, so neither denominator should be interpreted as architecture-independent compute efficiency.


Q4: Artifact contents and manual inspection

Artifacts are persistent files written by the learning agent and loaded by subsequent instances. They include executable Python tools (e.g., self-written chess/Go move-playing tools and helpers executing multiple moves per call), strategies/skills, textual memory, and instructions. In NetHack, the agent additionally modifies persistent coding agent files such as CLAUDE.md, AGENTS.md, and SKILL.md. Each artifact is registered by kind, path, and entry point (Appendix K.3).

Yes, we manually inspected the artifacts and their effects. Appendices E–F provide line-level code and textual examples across Figs. 12–19, including successful and unsuccessful artifacts.

  • Controlled intervention (Fig. 15): Removing a learned avoid-list entry causes subsequent agents to repeat the original mistake, providing direct evidence that the artifact affected behavior.
  • Go regression: Opus 5's held-out Go score fell from 50% to 0% at checkpoint 20; we traced this to an engine rewrite that calls a function unavailable in the actor's sandbox.
  • Self-repair in Hex: In a new Gemini Flash run, a game-playing helper stopped after one move at checkpoint 6; the agent repaired it at checkpoint 7. Later instances reused the repaired helper in both wins and losses.
  • Web-navigation failure: A substring-matching artifact incorrectly routed a credit-memo review task toward a product-review page. The acting agent ignored the advice and completed the task successfully.

These cases illustrate both beneficial and harmful artifact behavior. The broader reuse and failure classifications combine LLM judgments with execution evidence (Appendix H), rather than relying solely on manual examples.


Paper revisions (in blue)

All changes are marked in blue in the revised manuscript:

  • Appendix L: Web-navigation plasticity, failure attribution, and artifact-reuse analysis (W1/Q3).
  • Appendix G.5: Learning-curve shape, reuse dynamics, artifact effectiveness, and composition (W2).
  • §2.2, Fig. 1 caption, Appendix G.2: Pointer to the saturation checks, explicit Opus 4.8 extrapolation, threshold sensitivity, and simulation-based checks (Q1).
  • Appendix G.1: Token-normalized plasticity curves and corresponding version of Fig. 1 (Q2).

We thank the reviewer again for the constructive feedback. We hope these additions address the points raised, and we would be grateful if the reviewer would take them into account when finalizing the score. We are happy to run further analyses during the discussion period.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Summary of Revisions

We thank all reviewers for their constructive feedback and for recognizing the importance of evaluating agents' ability to learn from experience, our controlled held-out evaluation protocol across four environments, and our artifact-reuse and failure analyses.

We address all four main points raised across the reviews with new experiments and analyses:

  • Role of experience (N1Hv W1, opQb W1/Q1; Appendix M). In the suggested reflection-only control with three recent Gemini models on chess and Hex, all three models reach higher held-out performance (+3.8 to +21.7 points) and higher plasticity with training experience (PsatP^{sat} of 1,357 vs 396 for Gemini 4 Argon, 3,045 vs 1,383 for Gemini 4 Carbon, and 863 vs 0 for Gemini 3.8 Flash, in points per 1k USD). Experience thus yields more improvement per dollar than reflection alone. Both settings build and reuse strong tools, and experience directs reflection toward the specific mistakes that lose games.
  • Generality beyond games (Qgfp W1/Q3; Appendix L). Beyond the board games and NetHack in the submission, we add web navigation across three WebArena websites, where the protocol, plasticity metric, and failure analysis carry over directly. Flash-Lite, the least capable model, shows the highest plasticity on every site, and the analysis identifies a distinct bottleneck: 74–85% of web failures lack a covering artifact, whereas weaker chess agents mainly miss reuse.
  • Characterizing learning (Qgfp W2/Q2/Q4; Appendices E–F, G.1, G.5). We add learning-curve shapes, artifact-reuse dynamics, effectiveness over time, artifact composition, and manually inspected artifact examples. These show, for example, that reuse can precede improvement: in Hard chess, GPT-5.6 Sol reaches 99.7% reuse by checkpoint 3 and its score peaks at checkpoint 12. We also introduce token-normalized plasticity: the same two models, GPT-5.6 Sol and Opus 5, lead under both dollar and token cost.
  • Statistical reliability (Qgfp Q1, N1Hv W2, opQb W2/Q2/Q3; §2.2, Fig. 1, Appendix G.2). Under simulation-calibrated testing, all 26 board-game plasticity classifications remain unchanged, and the combined ordering is stable across 85%/90%/95% saturation thresholds. Figure 1 now reports 90% bootstrap intervals (2,000 resamples over held-out games and independent lineages): GPT-5.6 Sol and Opus 5 lead, with intervals clearly above those of the other saturated models. We also detail the fitting objective and saturation extrapolation, and add endpoint plasticity and a NetHack seed analysis.

Overall, these additions strengthen both central contributions of our work: measuring agents' efficiency in learning from experience, and diagnosing why their self-improvement succeeds or fails.

All revisions are marked in blue in the updated manuscript, and the reviewer-specific replies provide full details.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.