acceptodds
Under review as a conference paper at ICLR 2027

Certified Long-Horizon Code Agent Evolution via Validation-Gated Skill Optimization

Abstract

Long horizon agent self-evolution without model weight updates is essential for enabling deployed agents to accumulate reusable skills and improve over time. Prior self-evolution work has focused primarily on short-horizon tasks, while repository-level software engineering remains unexplored despite being an ideal testbed for long-horizon adaptation. In this setting, agents are required to solve streams of sequential tasks, navigate complex dependencies with evolving repositories and persistently store and reuse experience. Text-based skill optimization offers an efficient, non-parametric approach for such adaptation. However, existing methods often suffer from unstable updates, performance drawdown, and agent collapse over extended deployments. % In this paper, we formalize the concept of in-context self-evolution and introduce , a validated-gated framework for long-horizon skill optimization. We establish finite convergence, provide theoretical guarantees for future-task gain and drawdown, and derive the validation and evaluation holdout sizes required for a prescribed tolerance, with leading-order scaling . Empirically, our pipeline, achieves stable self-improvement over evolution horizon spanning more than SWE tasks, with average final and peak gains of and points across three frontier models (GPT-5.5, Claude-4.6 and MiniMax-M2.7). The validation gate reduces average drawdown by 75% and produces an more compact skill bank than ungated evolution. We further present extensive ablations identifying the design choices most critical to long-horizon skill evolution.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, self-evolving agents, skill optimization, test-time adaptation, agent memory, code agents, lifelong learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_V313OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes VALVE, a validation-gated method for long-horizon test-time evolution of repository-level code agents without weight updates.

Critical strengths

The unusually long evaluation horizon is a meaningful empirical contribution. The study follows roughly one thousand task arrivals per repository, reports both final and peak performance, and shows that ungated memory accumulation can collapse after an early peak. VALVE improves all twelve reported model–repository cells and substantially reduces both drawdown and bank size.

Critical weaknesses

The title and framing overstate what is certified in the realized experiments. The guarantees concern same-distribution expected task success under i.i.d. sampling, disjointness, a prespecified checkpoint family, and valid variance upper bounds. They do not certify software correctness, security, robustness to repository drift, or monotonic population improvement. More importantly, the 50-task validation panel is smaller than every per-update sample size shown in Table 4 for even a 10-point population-loss tolerance. The paper does not report the actual run-specific lower bounds, so the operational certificate may be weak or vacuous.

Questions for authors
  1. What are the actual Theorem 3 lower bounds for every run at n = 50, using the realized gate-query count, accepted-update count, measured margin, and chosen confidence level? Are any nontrivial?

  2. How can the prospective certificate use a gate horizon of 60 when observed runs reach 85 gate queries?

  3. Can the empirical variance estimates be replaced with independently justified high-probability upper bounds?

4 comments
AuthorsOpenReview ↗

Part 1/2: Scope of certification and validation-panel size (W1)

We thank the reviewer for the positive assessment of the long-horizon study and for the precise questions about what the guarantees mean in practice. Below we clarify the role of the theory in the paper and give the run-specific numbers.

W1. The title and framing overstate what is certified; the 50-task panel is smaller than Table 4 requires.

Thank you for this point. The guarantees address a question we consider central for self-evolving agents: whether improvements carry over to the future in-distribution tasks the agent will face, beyond the panel used to accept them, and how much held-out data is needed to establish this. That is what "certified" refers to.

  • Scope. In the paper, "certified" has a specific meaning: high-probability bounds on the expected solve rate for future tasks drawn from the same repository distribution, under Assumption 1. Concretely, final gain and drawdown are certified via a disjoint evaluation set (Thm 2), and the per-update population loss is bounded under adaptive panel reuse (Thm 3). This is the guarantee self-evolution needs: the improvement holds on tasks the agent has not yet seen, not only on the panel used to accept it. It concerns task-success rates and does not extend to properties such as patch correctness, security, or robustness to repository drift. We will polish the wording in the introduction and add a short "Scope of certification" paragraph so that this meaning is clear from the start.
  • Why a 50-task panel, and what the theory guarantees at this size. The theory assigns different guarantees to different held-out sets, and this determines their sizes. (i) The panel only needs to keep harmful updates out. Thm 3 bounds the population loss of every accepted update, and Thm 1 caps the number of accepted changes at K≤⌊(1−F^0)/ρ⌋K\le\lfloor(1-\widehat F_0)/\rho\rfloor. Table 4's ~136–155 tasks would tighten this per-update loss bound to 10 points; at 50 tasks it is looser (Table R7 below), but the gate's decisions already saturate (Fig. 4: 85.0 / 92.0 / 92.3 at 10 / 30 / 50 tasks). (ii) Improvement is certified on the accumulated gain via the disjoint evaluation set, and here the theory is most effective. Because the gate keeps the number of distinct banks small (A≤K+1A\le K+1), the drawdown certificate pays only log⁡A\log A rather than a log of the ~1,000-task horizon. Because the accumulated gain is large (+15 to +20 points), 400 tasks certify ≥ +11.5 / +12.3 (Table R5 in our response to fyRY). Certifying each small (3–5 point) update as positive would instead need hundreds to over a thousand panel tasks, since the required sample size grows with the inverse square of the effect. The sizing rule makes this explicit, and it is what directs the improvement guarantee to the accumulated gain rather than to each individual update. SkillOpt receives the same 50-task panel.
  • Empirically, retention is close to monotone at the task level. On held-out tasks, only 2.1% get worse from step 0 to the final VALVE bank, versus 8.5% without validation (Table R4 in our response to fyRY).
AuthorsOpenReview ↗

Part 2/2: Theorem 3 bounds (W1/Q1), query horizon T (Q2), and variance bound (Q3)

W1/Q1. What are the actual Theorem 3 lower bounds for every run at n = 50? Are any non-trivial?

We computed them for every accepted update in all 12 runs, using the realized acceptance index kk, the measured margin, T=100T=100 and δ=0.05\delta=0.05:

Table R7: Certified per-update population loss γ\gamma at n=50n=50 (empirical variance, as in the paper)

BackboneAccepts per runγ\gamma, 1st acceptRuns with all accepts at γ≤20\gamma\le20 ppShare of accepted gain with γ≤20\gamma\le20 ppNon-vacuous
GPT-5.51 / 1 / 15 / 17.0–15.4 pp3 / 455%18 / 18
Claude-4.62 / 7 / 6 / 56.3–11.8 pp1 / 456%20 / 20
MiniMax-M2.75 / 15 / 14 / 1414.2–15.2 pp0 / 411%48 / 48

Every accepted update has a non-trivial bound, and the bounds are tight where it matters. The first update in each run is controlled to within 6–15 pp. For GPT-5.5 and Claude-4.6, every update is within 15 pp in 4 of the 8 runs, and updates with γ≤20\gamma\le20 pp carry over half of the accumulated panel gain. Bounds widen for many small late updates, because the union over (Tk)\binom{T}{k} accept paths grows like klog⁡(T/k)k\log(T/k), which is the price of allowing proposals to react to earlier verdicts. With the variance-assumption-free bound of Q3, the first-update bound widens to 37–46 pp.

The pattern across backbones also follows the theory. MiniMax-M2.7, whose panel outcomes are about twice as variable, makes the most accepts (12 per run) with the smallest margins (median 3.3 points), so few of its updates are tightly bounded, which is exactly the case where the sizing rule calls for a larger panel.

We view this as the analysis working as intended: Thm 3 guards each update against large losses, while the improvement guarantee behind the reported results is placed on the accumulated gain (Table R5), where it is non-trivial even without a variance assumption. We will add Table R7 to the appendix and describe the two theorems in these roles.

Q2. How can the prospective certificate use T = 60 when observed runs reach 85 gate queries?

Thank you for this careful check. In Thm 3, TT is an a-priori upper bound on the number of gate queries, not a typical value. The T=60T=60 in Table 4 was an illustrative choice near the average number of queries per run, and pairing it with the observed runs was an oversight on our part, since some runs exceed it. The correct horizon follows from the protocol itself: the gate is queried at most once per mixed-outcome task, and each run stops after 100 such tasks (Sec. 4.1), so T=100T=100 is fixed before any run starts. It covers all 12 runs (15–97 queries; see our response to fyRY). Because TT enters the bound only through ln⁡Nk\ln N_k, the change is mild: the 10-point / K≤7K\le7 panel size in Table 4 goes from 136 to 155. All numbers in Table R7 already use T=100T=100, and we will update Table 4 and App. A.9 accordingly.

Q3. Can the empirical variance estimates be replaced with independently justified high-probability upper bounds?

Yes, and we have now done so. With Y∈[−1,1]Y\in[-1,1], Var(Y)≤E[Y2]\mathrm{Var}(Y)\le\mathbb E[Y^2], and E[Y2]\mathbb E[Y^2] has a closed-form upper confidence bound from the lower-tail Bernstein inequality for Y2∈[0,1]Y^2\in[0,1] (formula in our response to fyRY). It uses only the sample and Assumption 1, at the cost of a δ/2\delta/2 split. The resulting certified final gains are ≥ +11.5 (GPT-5.5) and ≥ +12.3 (Claude-4.6) at 95% (Table R5). For reference, the trivially valid Var≤1\mathrm{Var}\le1 would give a 15.5-point radius at nH=400n_{\mathcal H}=400, about twice the data-driven one. We will also use this bound in the sizing rule, so the recommended holdout sizes are prospectively valid.

Dear Reviewer,

Thank you so much for your thoughtful initial reviews!

With only two days left before the discussion window closes, I wanted to send a friendly reminder to check out the authors' rebuttal. They have put a lot of effort into answering the specific questions you raised.

Whenever you have a spare moment today, please drop a brief reply in the thread to let the authors know if their responses have addressed your concerns, or if any questions remain. Your timely feedback makes a huge difference in ensuring a fair and constructive review process!

Warm regards

Thanks for the detailed rebuttal. I maintain my score to support this paper.

Reviewer_fyRYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces VALVE, a framework for the self-evolution of long-term code agents. It combines contrastive skill extraction, validation-gated updates, and skill retrieval. The authors provide theoretical guarantees and evaluate VALVE using 3 LLMs on four SWE-smith codebases. VALVE reports an average final gain of 14.9 points, with reduced regression and a smaller skill library.

Critical strengths
  1. This paper investigates the skill evolution process across approximately 1000 tasks in each repo. This goes beyond the short-term scenarios commonly found in previous studies.
  2. VALVE improved performance in all 12 model-repo runs. Compared to ungated evolution, it also reduced the magnitude of backsliding and lowered memory consumption.
Critical weaknesses
  1. Validation and deployment may use different contexts. Appendix B.6 evaluates these skills by adding new skills to the currently retrieved context. However, during deployment, the updated knowledge base is reordered, and only the top 10 items per category are retrieved (Appendix B.5). These contexts may differ. Consequently, validation scores may not reflect actual performance during deployment. This also weakens the link between the implementation and the theoretical guarantees.
  2. The proof appears to use a one-sided concentration bound with incorrect orientation (Appendix A.5). The argument for the variance bound is also insufficient.
  3. Several numerical values are inconsistent. Appendix A.9 reports a maximum of 7 updates, yet Table 3 includes a knowledge base with 38 skills, even though updates are limited to 3 skills per update. The reported 492 validation queries also imply an average of 41 queries per run across 12 runs, instead of 61.5. These issues undermine trustworthiness in the results.
Questions for authors
  1. Was the same top-k retrieval algorithm used in the validation phase as in the deployment phase? If not, how can the performance improvement observed in the validation phase reflect the actual performance after deployment?
  2. Can the authors explain the inconsistencies regarding "accepted updates" "skill repo size" and "number of validation queries"?
  3. Can the authors clarify the proof of "backout" and explain the rationale for the variance bounds used in the certification?
4 comments
AuthorsOpenReview ↗

Part 1/2: Validation vs. deployment context (W1/Q1)

We are grateful to the reviewer for the very careful reading of both the implementation and the proofs. The questions concern how the theory connects the gate to performance on future tasks. We address each one below with quantities recomputed from the run logs. None of them changes the empirical results (Tables 1–2, Figs. 1–4), which are measured directly on the held-out evaluation set.

W1/Q1. Validation and deployment may use different retrieval contexts. Do validation scores reflect deployment performance?

Thank you, this is a good observation. The gate compares R(x,B)R(x,\mathcal B) with R(x,B)∪ΔR(x,\mathcal B)\cup\Delta, while deployment uses R(x,B∪Δ)R(x,\mathcal B\cup\Delta) (top-10 per category). We clarify three points, plus direct evidence on deployment behavior:

  1. Every reported gain is measured under the deployment context. Tables 1–2 and the held-out certificate (Thm 2) evaluate the final bank with the ordinary top-10 retriever on the disjoint set H\mathcal H. Thm 2 assumes nothing about how the gate picked that bank (Assumption 1 allows arbitrary, history-dependent proposals and verdicts). The gate decides which bank we keep; deployment performance is always measured separately.
  2. Thm 3 holds for the gate's actual map. Its proof only needs the retriever to be fixed independently of VV. With RV(x,B,m):=R(x,B)∪ΔmR_V(x,\mathcal B,m):=R(x,\mathcal B)\cup\Delta_m, the theorem certifies the gain under RVR_V verbatim. We will state this precisely, so that Thm 3 is not read as a statement about the top-10 context.
  3. The two contexts are identical whenever the updated bank has at most 10 principles and 10 skills, because top-10 then injects the whole bank. From the logged bank snapshots, 36 of the 86 accepted updates (42%) are in this regime, as are all updates in 4 of the 12 runs (GPT-5.5 deepdiff / pygments / sqlfluff, Claude-4.6 deepdiff).
  4. What the gate accepts transfers to deployment at the task level. On the held-out set, comparing each run's final bank with its own step-0:

Table R4: Per-task changes from step 0 to the final bank (held-out tasks, deployment retriever; GPT-5.5 and Claude-4.6 runs with per-task logs)

Tasks that got worseTasks that got betterSuccessful rollouts gained : lostAlways-solved tasks kept
VALVE w/o val8.5%23.2%3.6 : 190.8%
VALVE2.1%31.6%25.9 : 197.1%

So updates accepted by the gate rarely break anything once deployed: about 26 successes are gained for each one lost. Without the gate the ratio is 3.6.

A deployment-consistent gate (rerank the candidate bank with the deployment retriever before validating) is a one-line change, and we will expose it as an option in the released code.

AuthorsOpenReview ↗

Part 2/2: Proof orientation and variance bound (W2/Q3); corrected statistics (W3/Q2)

W2/Q3. The proof seems to use a one-sided bound with the wrong orientation. The argument for the variance bound is insufficient.

(a) Orientation: the reviewer is right for the drawdown bound, and we thank them for catching it. Proving Dj≤D^j+bD_j\le\widehat D_j+b needs the lower tail of D^j\widehat D_j. The correct argument applies Lemma 1 to Z=final−checkpointZ=\text{final}-\text{checkpoint}, whose mean is −Dj-D_j. Its one-sided range is 1+Dj1+D_j rather than 1, so with the peeling of Remark 1 the effective range is M=1+D^j+bM=1+\widehat D_j+b, or M=2M=2 unconditionally. The theorem and its rate are unchanged; at the paper's settings the radius changes from 6.5 to 6.6 points. The gain, separation and reused-panel certificates bound the upper tail of candidate-minus-reference and are correctly oriented.

(b) Variance. In the paper, the bound comes from the baseline decomposition σ2≤(1+2r) Ex[p(1−p)]+2εvar\sigma^2\le(1+\tfrac2r)\,\mathbb E_x[p(1-p)]+2\varepsilon_{\mathrm{var}} (Sec. 3.3), instantiated with an empirical proxy. We agree an empirical value is not a guaranteed upper bound. Two clarifications:

  • the proxy was computed on candidate-vs-current validation differences (and on one run, not 24,600 pooled differences as App. A.9 states, which we correct);
  • the held-out final-vs-base differences used by Thm 2 have a larger empirical variance (≈0.05 averaged over repositories).

We therefore replace the proxy with a bound that requires no variance assumption. Since Y∈[−1,1]Y\in[-1,1], Var(Y)≤E[Y2]\mathrm{Var}(Y)\le\mathbb E[Y^2], and since Y2∈[0,1]Y^2\in[0,1], the lower-tail Bernstein inequality for non-negative variables (Maurer, 2003) gives, with probability ≥1−e−L\ge1-e^{-L}, E[Y2]  ≤  (L/2n+L/2n+Y2‾)2.\mathbb E[Y^2]\;\le\;\Big(\sqrt{L/2n}+\sqrt{L/2n+\overline{Y^2}}\Big)^2 . This bound is plugged into Bennett (M=2M=2, δ\delta split between the two events).

Table R5: Certified final gain at 95% (jointly over backbones), no variance assumption, pooled over 4 repositories

BackbonenHn_{\mathcal H}Final gain (Table 1)RadiusCertified gain
GPT-5.5400+19.07.5≥ +11.5
Claude-4.6400+19.67.3≥ +12.3

For MiniMax-M2.7, the measured gain is smaller (+6.3) and its outcomes are about twice as variable (unstable-task share h≈0.22h\approx0.22 vs. 0.12–0.13), so 400 held-out tasks are not enough to certify a positive gain without a variance assumption. This is exactly the trade-off the sizing rule quantifies: both a smaller target gain and a larger variance call for a larger evaluation set. We will replace the numerical instantiation in Sec. 4.1 and App. A.9 with Table R5 and recompute Table 4 using this variance bound.

W3/Q2. Inconsistencies: maximum of 7 updates vs. a 38-skill bank; 492 queries vs. a mean of 61.5.

Thank you very much for checking these. Both stem from the same reporting error: the App. A.9 statistics were computed when only the GPT-5.5 and Claude-4.6 runs (8 of 12) were complete, and were not updated after the MiniMax-M2.7 runs were added. Corrected from the logs:

Table R6: Corrected appendix statistics (12 runs)

QuantityPaper (8 runs)Correct (12 runs)
Gate queries492 total, mean 61.5, max 85860 total, mean 71.7, max 97
Accepted updates KKmean 4.75, max 7mean 7.17, max 15

492 / 8 = 61.5, which explains the mismatch the reviewer noticed. The 38-skill bank is GPT-5.5 / pydicom, which made 15 accepted updates (each adds ≤3 skills), ending at 38 skills and 28 principles, so it is consistent with the corrected KK. Thm 1's cap K≤⌊(1−F^0)/ρ⌋K\le\lfloor(1-\widehat F_0)/\rho\rfloor holds in all 12 runs (e.g., 15 ≤ 33 for that run). We will also present the localized refinement ⌊h/ρ⌋\lfloor h/\rho\rfloor as conditional on its extra assumption, since some runs exceed it. All main-text results are unaffected.

Thank you for the detailed rebuttal and for your efforts to address the theoretical concerns. I still have some reservations about the overall contribution. While the framework brings together several useful techniques, I am not fully convinced that their combination offers sufficiently distinct methodological insights. The theoretical guarantees also seem to provide limited practical assurance, as the per-update bound is not meaningful under the actual experimental settings and does not directly apply to the deployed retrieval procedure. While the additional analyses are helpful, I remain unconvinced that the overall contribution is sufficient and will maintain my current rating.

We believe that controlling collapse over long-horizon self-evolution and sizing the validation panel to support future-task guarantees are important questions for the community at current stage and we sincerely appreciate your thoughtful engagement throughout the rebuttal which helped us make this paper better! Many thanks : )

Reviewer_evYWOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how code agents retain useful experience over long task streams. The model weights and solver harness remain fixed. VALVE combines contrastive skill extraction, validation-gated memory updates, and selective retrieval. It also analyzes stability and holdout requirements. Experiments cover three models and four SWE-smith repositories. The reported average final gain is 14.9 percentage points. Average drawdown falls from 6.1 to 1.5 points compared with ungated evolution.

Critical strengths
  • The paper directly studies whether gains last. It reports final performance and peak-to-final drawdown. These measures show whether accumulated experience remains useful. The model and solver harness stay fixed, making the effect of memory easier to interpret.

  • The results are consistently positive. VALVE beats both the base agent and ungated evolution in all twelve model–repository combinations. Its average final gain is 14.9 points, compared with 7.1 for SkillOpt. Average drawdown is also lower (Tables 1–2).

  • Selective retention produces compact, useful memory. Validation reduces the bank from 4,446 to 396 items. Final performance also improves. The bank-growth analysis and learned examples support a useful finding: storing more experience does not always help.

Critical weaknesses
  • The method builds on existing validation gates. SkillOpt and SkillOpt-Lite already use such gates. VALVE adds a combined memory design and a long-stream study. A clearer comparison would help show what these additions contribute.

  • The benefit at a fixed computation budget remains unclear. Average runtime is 122.1 hours for VALVE, 96.1 for SkillOpt, and 50.6 without validation. The paper reports these costs. Comparing performance at similar budgets would help assess the practical tradeoff.

  • The ablations do not directly measure retained gains. Figure 3 reports the best score during evolution for one model–repository pair. Final scores would better match the paper’s main objective. Some variants also change several mechanisms, making component effects harder to separate (Section 5.1).

Questions for authors
  • Section 4.1 reports a longer average runtime for VALVE than for SkillOpt. Were the total rollout budgets comparable, including validation? Please explain the main source of the runtime difference. This would help assess whether the extra cost is justified by the retained gains.

  • Figure 3 uses the best score reached during evolution. Could you summarize the final scores of the existing ablation runs, if available? Please also clarify which variants change more than one mechanism. This would help distinguish effects on peak performance from effects on retained gains.

4 comments
AuthorsOpenReview ↗

Part 1/2: Contributions beyond SkillOpt (W1)

We sincerely thank the reviewer for the thoughtful and encouraging review, and for highlighting that VALVE measures whether gains last. That question is in fact the starting point of the paper, and the reviewer's comments helped us state the paper's purpose more clearly.

W1. SkillOpt and SkillOpt-Lite already use validation gates. A clearer comparison would show what VALVE's additions contribute.

We agree that the gate itself is not our contribution, and will make that explicit. The paper sets out to answer three questions that, in our view, need answers before self-evolving agents can be relied on in real long-term use, such as co-developing a codebase with an agent over months. VALVE is the pipeline that results from answering them.

(1) Does gated self-evolution actually hold up over a long horizon? Existing skill- and context-optimization methods are evaluated in bounded settings: a fixed training split revisited for a few epochs. SkillOpt, for example, defaults to 4 epochs with 40-task batches (39 training tasks on ALFWorld), and SkillOpt-Lite reports reaching near-final validation within 2–3 steps. Whether the gains survive when one persistent memory must serve a single stream of new tasks had not been tested. We run exactly this: a single pass over streams of up to ~1,000 sequential task arrivals per repository, with up to 100 gated evolution steps, across 3 backbones × 4 repositories. The answer is not obvious:

Setting (12-run average)Peak gainFinal gainDrawdown
No gate+7.0+0.96.1
SkillOpt (gated)+10.8+7.13.6
VALVE+16.5+14.91.5

Without a gate, almost all of the gain is lost by the end. Even with SkillOpt's gate, a third of the peak gain is lost. Bounded evaluations cannot reveal this, and we see such long-horizon evidence as a prerequisite for studying self-evolution.

(2) Which design choices actually matter? Each self-evolution method ships its own bundle of extraction, storage, retrieval and gating, so the contribution of each part has been unclear. We isolate them with controlled ablations: one component changed at a time, everything else, including the gate, held fixed (Fig. 3). Both sides of every comparison correspond to choices in published methods (App. Table 6). The findings directly define VALVE's pipeline. For the four components where SkillOpt makes the other choice:

Table R1: Replacing one VALVE choice with SkillOpt's choice, gate held fixed (GPT-5.5 / pydicom; base 38.2)

ComponentSkillOpt's choiceFinal avg@3Δ vs. VALVEDrawdown
—VALVE76.3—0.7
Retrievalinject the whole bank72.7−3.70.3
Training streamextract from all tasks67.3−9.01.3
Storageone evolving document65.3−11.00.7
Extraction granularitybatch of 463.3−13.02.0

Each SkillOpt-style choice costs 3.7–13.0 points of retained performance even with the gate unchanged. This explains much of the main-table gap. The remaining ablations add further lessons:

  • Structured memory matters for retention. Without principles or with sentence-format skills, drawdown rises to 3.3 (Q2 below).
  • Validation size matters. Final avg@3 is 80.0 / 85.0 / 92.0 / 92.3 at 0 / 10 / 30 / 50 panel tasks (Fig. 4).
  • Validating joint bank moves pays off. Every retained item helps on its own (+0.3 to +10.3), but the validated bank as a whole gives +13.8 (App. G).

(3) Can the gain be guaranteed on future tasks, and how much held-out data does that take? A gate only shows improvement on its own panel. For self-evolution to be useful, the improvement must hold for the new in-distribution tasks the agent meets afterwards. This question also matters to practitioners, who need to know how large a validation set to collect. Our theory addresses it:

  • Thm 2 certifies final gain and drawdown on future tasks via a disjoint evaluation set.
  • Thm 3 bounds the population loss of every accepted update despite adaptive reuse of the panel.
  • Cor. 1 inverts these bounds into holdout sizes for a target tolerance, N≳s2L/γ2N\gtrsim s^2L/\gamma^2.

On our runs, the resulting certificates show a future-task gain of at least +11.5 (GPT-5.5) and +12.3 (Claude-4.6) at 95% confidence (Table R5 in our response to fyRY). The sizing rule also says when a panel is too small for tight per-update control (see our response to V313). MiniMax-M2.7 illustrates this: its panel outcomes are about twice as variable, and with the same 50-task panel it accepts more, smaller updates and retains the smallest gain, as predicted (Q1 below).

AuthorsOpenReview ↗

Part 2/2: Compute comparison (W2/Q1) and final ablation scores (W3/Q2)

W2/Q1. Were total rollout budgets comparable, including validation? What drives the runtime difference?

Thank you for this question. A per-backbone breakdown gives a clearer picture than the 12-run average:

Table R2: Compute and final gain per backbone (train + validation; held-out evaluation excluded)

BackboneWall-clock (h) VALVE / SkillOptTrain+val rollouts VALVE / SkillOptFinal gain VALVE / SkillOpt
GPT-5.5120.6 / 123.9≈57.7k / 80.0k+19.0 / +6.2
Claude-4.6113.6 / 44.5≈43.0k / 44.2k+19.6 / +10.2
MiniMax-M2.7131.9 / 120.0≈68.2k / 29.2k+6.3 / +5.0

VALVE rollouts are computed from the logged gate-query and accept counts: 3 per stream task, 150 per gate query (candidate), and 150 per distinct bank (the current-bank result is reused until an update is accepted). SkillOpt counts come from its saved history.

  • GPT-5.5 is effectively compute-matched. Wall-clock is within 3% and VALVE uses ~28% fewer rollouts, while retaining 3× the gain.
  • Claude-4.6 is matched in rollouts (43.0k vs. 44.2k), the load-independent measure of compute. For API-served models, wall-clock is mostly set by the serving capacity available under concurrent load (rate limits and concurrency caps). That capacity varied across our many simultaneous experiments, which explains the wall-clock gap despite near-identical rollout counts.
  • MiniMax-M2.7 needs more gate queries, and its behavior matches what the theory predicts (Table R2b). Served locally at temperature 1.0, MiniMax produces mixed outcomes far more often. Since the gate runs once per mixed-outcome task, it is queried about 5× more often per stream task than for Claude-4.6, and the run reaches the 100-step cap after 300–500 tasks. The same outcome variance shows up on the validation panel, where MiniMax has about twice the share of unstable tasks. By Cor. 1, the required panel size grows with this variance, so a fixed 50-task panel resolves less for MiniMax. As expected, its gate accepts more and smaller updates, and its retained gain is the smallest.

Table R2b: Evolution behavior per backbone (VALVE runs; per-run averages or ranges over the 4 repositories)

BackboneMixed-outcome tasksGate queries / runStream tasks seenUnstable panel tasks hhAccepted updates / runMedian accepted marginFinal gain
GPT-5.58–14%62–85524–1,0590.134.55.0+19.0
Claude-4.62–8%15–69753–1,0670.125.03.7+19.6
MiniMax-M2.720–34%89–97294–5060.2212.03.3+6.3
  • Main cost driver: the gate accounts for 75–92% of VALVE's rollouts, and it is the whole difference to VALVE w/o val (122.1 vs. 50.6 h). The validation-size sweep shows that a 30-task panel already achieves 92.0 vs. 92.3, a 40% cheaper operating point for practitioners.

We will add Tables R2 and R2b to App. E.

W3/Q2. Figure 3 uses the best score during evolution. Could you summarize final scores of the existing ablation runs?

Certainly. Final scores for all ablation runs:

Table R3: Peak and final scores for all component ablations (GPT-5.5 / pydicom; base 38.2)

VariantPeakFinalΔ peakΔ finalDrawdown
Full VALVE77.076.3——0.7
Inject all73.072.7−4.0−3.70.3
All-task stream68.767.3−8.3−9.01.3
No principles68.064.7−9.0−11.73.3
Sentence format67.063.7−10.0−12.73.3
Single document66.065.3−11.0−11.00.7
Batch size 465.363.3−11.7−13.02.0
  • All conclusions hold on final scores. The full pipeline remains best, and the ordering of variants is nearly unchanged.
  • For 5 of 6 components, the effect on retained performance is at least as large as on peak.
  • Structure matters most for retention. Removing principles or switching to sentence-format skills raises drawdown from 0.7 to 3.3.

We will redraw Fig. 3 with final score as the primary bar and peak as a marker.

W3/Q2 (cont.). Which variants change more than one mechanism?

Thank you for pointing this out. Three variants are coupled by construction:

  • Single document changes storage and removes item-level retrieval, since a single evolving document cannot be ranked top-k.
  • Batch of 4 changes extraction granularity and, as a consequence, makes 4× fewer gate decisions.
  • All-task stream changes the extraction trigger and therefore how often extraction and the gate run.

These match how prior methods actually package each choice (App. Table 6). We will label them as operational alternatives in Fig. 3. The single-mechanism comparisons are inject-all, no-principles, and sentence-format.

Dear Reviewer,

Thank you so much for your thoughtful initial reviews!

With only two days left before the discussion window closes, I wanted to send a friendly reminder to check out the authors' rebuttal. They have put a lot of effort into answering the specific questions you raised.

Whenever you have a spare moment today, please drop a brief reply in the thread to let the authors know if their responses have addressed your concerns, or if any questions remain. Your timely feedback makes a huge difference in ensuring a fair and constructive review process!

Warm regards

Thank you for the detailed rebuttal, which has addressed my major concerns. I maintain my recommendation for this paper: weak acceptance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.