acceptodds
Under review as a conference paper at ICLR 2027

When Is Enough Enough in Self-Evolving LLM Systems?

Abstract

Self-evolving large language model (LLM) systems repeatedly propose, evaluate, and incorporate updates to prompts, skills, or other persistent artifacts. Despite their growing effectiveness, these systems typically operate under a predetermined iteration or compute budget, without a principled criterion to determine when further evolution is no longer worthwhile. This can lead to two undesirable consequences: unnecessary computation after performance has saturated and the risk of returning late updates that overfit or exploit the evaluation signal. These issues motivate us to study two fundamental questions: when should a self-evolving system stop, and what should it output once it stops? We address the first by formulating an online sequential testing problem and constructing an anytime-valid restart detector using the per-item paired evaluation outcomes already produced by self-evolving LLM systems. We address the second by formulating a change-point estimation problem and using the estimated transition to select an earlier artifact for output. The resulting procedure is plug-and-play and requires no modification of the underlying self-evolving algorithms. Across two self-evolving frameworks, three LLM model families, and five benchmarks, our method substantially reduces computation costs while maintaining comparable unseen-test performance. For example, on SearchQA with SkillOpt and DeepSeek V4 Flash, our method stops at round 4 rather than the full budget of 40, reducing token usage by 91.6% while achieving 82.43% unseen-test accuracy versus 82.00% under the full-budget run.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Self-evolving LLM agents, early stopping, sequential hypothesis testing, change-point estimation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wfm2OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies two decisions in self-evolving LLM systems: when to stop optimizing persistent artifacts such as prompts or skills, and which historical artifact to return. The proposed method constructs restart-based sequential tests from paired candidate–incumbent evaluation outcomes and uses the strongest restart process to estimate a change point for output selection. Experiments cover two self-evolving frameworks, three model families, and five benchmarks. The main results show 48.4%–91.6% token savings relative to full-budget runs, with relatively small observed differences in unseen-test accuracy.

Critical strengths
  1. A useful problem formulation with a practical interface. Separating stopping time from output selection is well motivated: additional rounds may be needed to establish that progress has stalled, while an earlier artifact may be preferable for deployment. The method reuses existing paired evaluation outcomes and can be integrated without changing candidate generation or acceptance, making the proposed approach practically relevant.

  2. Promising computational savings supported by a statistical formulation. Table 1 demonstrates substantial reductions in token consumption while retaining much of the improvement obtained by the underlying evolution process. The explicit assumptions and false-alarm run-length guarantee provide a principled foundation, while the experiments with shared default parameters across models and frameworks offer encouraging initial evidence of applicability.

Critical weaknesses
  1. The comparisons do not establish an advantage over simpler stopping strategies. The main baseline is running the underlying system to a fixed budget. There are no direct comparisons with patience-based stopping, development-set-calibrated short budgets, or simple gain-threshold rules. SkillOpt and GEPA serve as host frameworks rather than competing stopping methods, and aGRAPA versus the fixed mixture is an internal ablation. Consequently, the reported savings do not isolate the benefit of the proposed statistical construction over ordinary early stopping. Moreover, the paper does not compare change-point-based output selection with returning the incumbent at the same alarm time. These comparisons are essential to substantiate the contributions of both components.

  2. Evidence for robust performance preservation and transfer remains limited. Cross-model and cross-framework experiments concentrate on SearchQA and GSM8K; the more complex tool-oriented tasks receive substantially narrower coverage. Table 7 reports stopping times and change-point estimates, but not the corresponding accuracy–cost trade-offs across parameter settings, and the procedure used to choose the shared defaults is insufficiently documented. Multiple independent evolution runs are also not reported. The paired confidence intervals quantify uncertainty across test examples, not variation across evolution trajectories, and intervals containing zero do not establish non-inferiority—particularly given the wide OfficeQA interval. These limitations weaken the evidence that the method reliably preserves performance across tasks and search dynamics.

Questions for authors
  1. Comparative evidence: How does the method compare with a tuned patience rule and a fixed short-budget baseline under a matched development and evaluation protocol? Does it retain an advantage at comparable test accuracy or computational cost?

  2. Component attribution: At the same alarm time, how does change-point-based output selection compare with returning the current incumbent? What proportion of the benefit can be attributed specifically to retrospective selection?

  3. Clarification and robustness: How were the default parameters selected, and were they fixed before evaluating the reported tasks? Can repeated runs, particularly on OfficeQA and SpreadsheetBench, establish the variability of token savings and test-performance changes?

5 comments
AuthorsOpenReview ↗

Response to Reviewer wfm2 (1/2)

Thank you for the careful review, insightful questions and suggestions. We address your comments and questions below.

Q1. Comparative evidence

How does the method compare with a tuned patience rule and a fixed short-budget baseline under a matched development and evaluation protocol?

Both baselines require a number fixed before the run: the patience rule needs the number of rounds without improvement to tolerate (pp), and a fixed budget needs the total number of rounds (N′N'). To compare them with our method, we check which value works on each benchmark.

For the patience rule, the loop stops once the incumbent's validation accuracy has not increased for pp consecutive rounds, and the incumbent is returned. We replayed this rule for several pp on the saved trajectories. In the table below, entries are stopping rounds; †\dagger misses a later validation gain; ‡\ddagger reaches the LiveMath exploit; "no stop" runs the full budget.

RuleSearchQAGSM8KLiveMathOfficeQASpreadsheetBench
p=1p=12221$^\dagger$1$^\dagger$
p=2p=23332$^\dagger$2$^\dagger$
p=3p=344448$^\dagger$
p=5p=56667no stop
p=10p=101111no stop$^\ddagger$13no stop
Our rule4488no stop

The table shows that small pp stops too early on SpreadsheetBench and OfficeQA, while large pp reaches the LiveMath exploit. Tuning would therefore select p=5p=5, the only value that avoids every failure. This choice, however, can be made only after the full trajectories are observed. The future trajectory is unknown, so there is no online evidence for which pp to use.

A fixed budget N′N' faces the same problem. To keep the gain at round 11 on SpreadsheetBench, N′N' must be at least 11; to avoid the exploit at round 16 on LiveMath, N′N' must be at most 15. Only a budget between 11 and 15 avoids both failures. N′=11N'=11 on SearchQA and GSM8K still runs ten rounds past their last gain at round 1, while our rule stops at round 4. As the same case as patience rule, the choice of N′N' is known only after the full trajectories are observed.

Although this replay is not a matched development and evaluation protocol, it favors the baselines: their pp and N′N' are tuned on the very runs they are evaluated on, while our configuration is fixed in advance. Even with this advantage, neither baseline outperforms our rule. Our rule adapts its stopping round to each run, using one fixed configuration on all five benchmarks without failure, and it does not need the full trajectory to do so.

Does it retain an advantage at comparable test accuracy or computational cost?

At equal test accuracy, yes. Our rule returns the same artifacts as the tuned p=5p=5. Our cost is lower on SearchQA (2.18M vs 3.40M tokens) and GSM8K (1.21M vs 1.84M), and higher on LiveMath (7.38M vs 5.48M). However, p=5p =5 is not known unless we do not stop.

Q2. Component attribution

At the same alarm time, how does change-point-based output selection compare with returning the current incumbent?

In every default setting except GEPA/GSM8K, the two are the same artifact. GEPA/GSM8K is the only exception:

GEPA/GSM8K (alarm round 6)ArtifactUnseen-test accuracy
Incumbent at alarmRound 597.20%
Change-point outputRound 297.20%

The two artifacts differ because an update accepted after the estimated change point raised validation accuracy by 0.0050.005, a net gain of one item out of 200. Their equal unseen-test accuracy may be partly coincidental, but it is what our formulation predicts: a gain below ε=0.01\varepsilon=0.01 is treated as not worthwhile, and a one-item difference is within evaluation noise.

What proportion of the benefit can be attributed specifically to retrospective selection?

Retrospective selection does not aim to raise accuracy or save tokens. The savings come from stopping, and in our runs the selected artifact matches the incumbent at the alarm in unseen-test accuracy. Its purpose is to guard against validation overfitting. Between the change point and the alarm, the loop is already in the non-improvement regime. For updates accepted by the gate in this regime, the validation gain could come from overfitting the validation set. Retrospective selection returns the incumbent before the change point and thus avoids the risk of overfitting.

AuthorsOpenReview ↗

Response to Reviewer wfm2 (2/2)

Q3. Clarification and robustness

How were the default parameters selected, and were they fixed before evaluating the reported tasks?

Each default has a fixed meaning rather than being tuned for performance. ε=0.01\varepsilon=0.01 sets the smallest worthwhile gain to one percentage point of validation accuracy per round. δ=0.05\delta=0.05 is the conventional 5% level and gives an average run length to false alarm of at least 20. The mixture grid 0.1,0.2,0.4\\{0.1,0.2,0.4\\} and the aGRAPA initial fraction λ0=0.1\lambda_0=0.1 are moderate betting fractions within the valid range [0,1/2][0,1/2]. We set these values on SkillOpt/SearchQA, our first task, and kept them unchanged for every later task, model and framework.

Can repeated runs, particularly on OfficeQA and SpreadsheetBench, establish the variability of token savings and test-performance changes?

We repeated SpreadsheetBench with a new seed and the same defaults. The two runs compare as follows.

Original runNew seed
Validation accuracy, round 0 → 1637.5% → 70.0%32.5% → 67.5%
Rounds with accepted updates3, 5, 9, 111, 5, 8
Alarm within 16 roundsNoNo
Token saving0%0%

Both runs keep improving, and neither reaches the alarm threshold. And the returned artifact is the full-run artifact. Two runs show that this outcome is reproducible. We did not repeat OfficeQA, since the full run costs more than 200M tokens and more than 40 hours, which exceeds our rebuttal budget.

W1. The comparisons do not establish an advantage over simpler stopping strategies

Q1 compares our rule with patience stopping and fixed short budgets, and Q2 compares change-point output selection with returning the incumbent at the same alarm time. For gain-threshold rules, we also replayed a patience variant that resets only on a validation gain of at least 0.010.01; it gives the same stopping rounds as the patience rules in Q1.

W2. Evidence for robust performance preservation and transfer remains limited

Accuracy–cost trade-off in Table 7. Changing (ε,δ)(\varepsilon,\delta) changes when the detector stops, but the returned artifact remains unchanged. Across all settings that stop, each benchmark returns the same artifact, so unseen-test accuracy is unchanged. The table gives the tokens saved relative to the full run for each setting in Table 7.

ε\δ\varepsilon \backslash \deltaSearchQA 0.010.050.10GSM8K 0.010.050.10LiveMath 0.010.050.10OfficeQA 0.010.050.10
0.00277%92%92%82%82%87%0%15%38%41%48%56%
0.00589%92%92%82%87%87%0%15%45%41%48%56%
0.0192%92%94%87%90%90%0%38%45%41%48%56%
0.0292%94%94%90%93%93%15%45%54%41%48%56%
0.0594%94%94%93%93%93%38%54%61%48%64%64%

A 0% entry means no alarm within the budget; the full-run artifact is then returned. We will add these values to Table 7.

Default parameters and repeated runs. The defaults were set on our first task and kept unchanged for all later tasks, models and frameworks. A repeated SpreadsheetBench run with a new seed reproduces the original outcome (no alarm, same full-run return). Details are in Q3.

Confidence intervals. The paired intervals compare two fixed artifacts, the returned one and the full-run one, on the same test examples. They therefore measure test-sampling uncertainty, not variation across evolution runs. The OfficeQA interval is wide because its test set has only 172 examples. We agree that an interval containing zero does not establish non-inferiority, and will replace "comparable performance" with "no significant difference on the test set" in the revision.

AuthorsOpenReview ↗

Additional Results on Comparative Evidence and Component Attribution

Thank you again for your review. To further address your concerns, we ran additional experiments related to your Q1 and Q2 and share the results below, together with a small correction to our previous rebuttal.

Comparative evidence

We replayed AutoGluon's adaptive patience rule (AdaptiveES) on all our saved SkillOpt trajectories. Its parameters are fixed by AutoGluon, independently of our runs. It sets patience =round(0.3,τ+10)=\mathrm{round}(0.3\\,\tau+10) (class default), where τ\tau is the round of the last improvement. Our rule uses the paper's defaults ε=0.01\varepsilon=0.01 and δ=0.05\delta=0.05 on every run.

In the table, all paired entries are AdaptiveES / our rule. Stop is the stopping round; "none" means no stop within the budget. Returns is the round of the returned artifact. Saved is the share of full-budget tokens not spent. "Same" means both rules return the same artifact, and ‡\ddagger marks the round-16 reward-hacking artifact.

RunStopReturnsSavedTest accuracy
SearchQA / DeepSeek11 / 41 / 175.1% / 91.6%Same
GSM8K / DeepSeek11 / 41 / 173.2% / 89.8%Same
LiveMath / DeepSeeknone / 816$^\ddagger$ / 10% / 37.8%‡\ddagger
OfficeQA / DeepSeek14 / 83 / 312.6% / 48.4%Same
SpreadsheetBench / DeepSeeknone / none11 / 110% / 0%Same
SearchQA / Qwen3-32Bnone / 1233 / 80% / 75.3%73.79% / 72.43%
GSM8K / Qwen3-32B17 / 85 / 162.9% / 84.5%95.20% / 95.20%
GSM8K (n=100n=100) / DeepSeek11 / 51 / 172.7% / 87.3%Same
SpreadsheetBench, new seednone / none8 / 80% / 0%Same

AdaptiveES never stops earlier than our rule, and our rule saves at least as many tokens on every run. On six runs, both rules return the same artifact. Of the remaining three, AdaptiveES returns the reward-hacking artifact on LiveMath; on GSM8K/Qwen3-32B, both artifacts score 95.20% on unseen test data. On SearchQA/Qwen3-32B, AdaptiveES runs the full budget and its round-33 artifact scores 73.79% on unseen test data, compared with 72.43% for ours, while spending more than four times as many tokens.

Component attribution

The change-point output differs from the alarm-time incumbent in two settings: GEPA/GSM8K with DeepSeek V4 Flash and SkillOpt/GSM8K with Qwen3-32B (see Correction below). In both settings, the change-point output achieves the same unseen-test accuracy as the alarm-time incumbent while using fewer prompt tokens. Selecting this output can therefore reduce inference cost.

The table reports prompt tokens summed over the 500 unseen test queries. Savings are relative to the incumbent at the alarm.

SettingArtifactRoundTest accuracyPrompt tokensSaved
SkillOpt/GSM8K/Qwen3-32B (alarm: 8)Incumbent at alarm595.20%436,828—
SkillOpt/GSM8K/Qwen3-32B (alarm: 8)Change-point output195.20%203,82853.3%
GEPA/GSM8K/DeepSeek V4 Flash (alarm: 6)Incumbent at alarm597.20%478,136—
GEPA/GSM8K/DeepSeek V4 Flash (alarm: 6)Change-point output297.20%357,63625.2%

In these two settings, output selection reduces prompt-token usage without reducing observed unseen-test accuracy.

Correction

We previously stated that the change-point output equals the alarm-time incumbent in every default setting except GEPA/GSM8K with DeepSeek V4 Flash. SkillOpt/GSM8K with Qwen3-32B (Table 5) is a second exception.

In this setting, the change-point estimate returns the round-1 artifact, whereas the incumbent at the alarm (round 8) is the round-5 artifact, after two accepted updates raised validation accuracy from 0.94 to 0.96. On the 500 unseen test examples, both artifacts score 95.20%. Each update gained only two net validation items out of 200, and the combined validation gain did not translate into higher unseen-test accuracy.

Reviewer_25ZTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies two decisions in self-evolving LLM systems: when to stop the evolution loop, and which historical artifact to return once it stops. The authors propose a plug-and-play procedure built on the per-item paired validation outcomes between the incumbent and each candidate. For stopping, they run a betting process restarted at every round, which gives an anytime-valid sequential detector. The detector accumulates evidence that the expected gain of new candidates has dropped below a threshold ε\varepsilon. For output selection, they take the restart with the largest accumulated wealth as an estimate of the change point, and return the artifact just before it. The method is evaluated on two self-evolving frameworks (SkillOpt and GEPA), three LLM families, and five benchmarks. In the three main settings, it uses 48.4%–91.6% fewer tokens than the full-budget run, with comparable unseen-test accuracy.

Critical strengths
  1. Large computational savings at comparable performance. In the main experiments, the method saves 48.4%–91.6% of tokens while keeping unseen-test accuracy comparable to the full-budget run. This suggests that adaptive stopping can remove a substantial amount of unnecessary computation in self-evolving loops.

  2. Lightweight, plug-and-play design. The procedure only uses per-item evaluation outcomes that the evolution loop already produces, and it does not change how candidates are generated or accepted. It is applied to both SkillOpt and GEPA, which supports its applicability across different self-evolving frameworks.

Critical weaknesses
  1. Limited comparison with existing early-stopping strategies. The experiments compare the method mainly against full-budget evolution. Simpler early-stopping baselines, such as patience-based or threshold-based rules, are not evaluated. As a result, it is hard to tell whether the statistical machinery brings a real advantage over these simpler alternatives, either in computation or in final performance.

In addition, the choice of full budgets (40 rounds for SearchQA and GSM8K, 16 rounds for OfficeQA) is not justified. The reported token savings are measured against these budgets, so without that justification the efficiency gains are hard to interpret.

Questions for authors
  1. Figure 1 and the effect of output selection. In Figure 1, the selection-validation accuracy stays the same from round 1 to round 4, but the unseen-test accuracy changes. Under the greedy acceptance rule in Section 3.1, does this mean the retained artifact is identical across these rounds? If so, what causes the variation in unseen-test accuracy? If not, could the authors clarify how artifacts are actually updated and evaluated in this experiment? Could the authors also report the unseen-test performance of the incumbent at the alarm round, next to that of the artifact selected by the change-point estimator? This would show how much the output-selection step contributes on its own.
3 comments

Thank you for the careful review, insightful questions and suggestions. We address your comments and questions below.

Q1. Figure 1 and the effect of output selection

Is the retained artifact identical across rounds 1–4? Yes. No candidate is accepted in rounds 2–4, so the incumbent is the round-1 artifact throughout.

What causes the variation in unseen-test accuracy? The green points are not the performance of incumbent. Each point is the unseen-test accuracy of the candidate proposed in that round. The incumbent's unseen-test accuracy stays at 82.43% over rounds 1–4. We will state this more clearly in the caption.

Incumbent at the alarm versus the change-point output. These two incumbents happen to be the same artifact in all our numerical experiments except GEPA/GSM8K:

GEPA/GSM8K (alarm round 6)ArtifactUnseen-test accuracy
Incumbent at alarmRound 597.20%
Change-point outputRound 297.20%

The two artifacts differ because an update accepted after the estimated change point raised validation accuracy by 0.0050.005, a net gain of one item out of 200. Their equal unseen-test accuracy may be partly coincidental, but it is what our formulation predicts: a gain below ε=0.01\varepsilon=0.01 is treated as not worthwhile, and a one-item difference is within evaluation noise.

Output selection does not aim to raise accuracy or save tokens. The savings come from stopping, and in our runs the selected artifact matches the incumbent at the alarm in unseen-test accuracy. Its purpose is to guard against validation overfitting. Between the change point and the alarm, the loop is already in the non-improvement regime. For updates accepted by the gate in this regime, the validation gain could come from overfitting the validation set. Output selection returns the incumbent before the change point and thus avoids the risk of overfitting.

W1. Simpler stopping rules

We added patience baselines: stop after pp consecutive evaluated rounds without a gain compared with the incumbent's validation accuracy, for p∈1,2,3,5,10p\in\\{1,2,3,5,10\\}. A threshold version of the patience-based method that resets patience only on a gain of at least 0.010.01 gives the same stopping rounds. In the table below, each entry is the stopping round; "no stop" = no stop before the full budget; †\dagger = misses a later worthwhile validation gain; ‡\ddagger = accepts the round-16 reward-hacking artifact on LiveMath.

RuleSearchQAGSM8KLiveMathOfficeQASpreadsheetBench
p=1p=12221$^\dagger$1$^\dagger$
p=2p=23332$^\dagger$2$^\dagger$
p=3p=344448$^\dagger$
p=5p=56667no stop
p=10p=101111no stop$^\ddagger$13no stop
Our rule4488no stop

Small pp stops too early, missing later gains on OfficeQA and SpreadsheetBench; large pp stops too late, reaching the LiveMath exploit. Only p=5p=5 avoids both, but this is unknown unless we run the full trajectories without stopping. Since the problem is online stopping time detection, so there is no evidence for choosing pp. Moreover, no choice of pp comes with a false-alarm guarantee. Our detector uses one fixed configuration (ε=0.01\varepsilon=0.01, δ=0.05\delta=0.05) on all benchmarks and guarantees an average run length to false alarm of at least 1/δ1/\delta.

Choice of full budgets. The budgets are not chosen by us; they follow from the original SkillOpt configurations, which we did not change. SkillOpt runs a fixed number of steps

N=epochs×⌈training-set sizebatch size⌉.N=\text{epochs}\times\Bigl\lceil\frac{\text{training-set size}}{\text{batch size}}\Bigr\rceil.

For example, SearchQA and GSM8K: N=4×⌈400/40⌉=40N=4\times\lceil 400/40\rceil=40. OfficeQA: N=8×⌈50/40⌉=16N=8\times\lceil 50/40\rceil=16. We will list these configurations in the appendix.

Thank you to the authors for the detailed rebuttal and additional experiments.

The clarification of Figure 1 resolves my concern regarding the variation in unseen-test accuracy. The explanation that the full-budget settings follow the original SkillOpt configurations also adequately addresses my question about the choice of iteration budgets.

The newly added patience-based comparisons are helpful and strengthen the empirical evaluation. In particular, the ability to adapt stopping decisions across different benchmarks using a shared configuration is a meaningful advantage. However, the comparisons remain limited to a small set of saved evolution trajectories and do not fully establish the proposed method's advantages over simpler stopping strategies under a matched parameter-selection and evaluation protocol. Therefore, my main concern regarding the comparative evaluation is only partially resolved.

Regarding output selection, I appreciate the clarification that its primary purpose is to mitigate potential validation overfitting rather than directly improve accuracy. Nevertheless, since the selected artifact is identical to the alarm-time incumbent in nearly all reported settings, its independent empirical benefit remains unclear.

Overall, the rebuttal has addressed several of my concerns and strengthened the paper's experimental evidence. However, I do not find the remaining concerns sufficiently resolved to change my overall assessment. I therefore maintain my original rating.

AuthorsOpenReview ↗

More results

Follow-up to Reviewer 25ZT

Thank you for the follow-up. To further address your concerns, we have added a comparison addressing the matched-protocol concern, provided new evidence on the benefits of our output selection, and included a small correction to our previous rebuttal.

Matched protocol

We replayed AutoGluon's adaptive patience rule (AdaptiveES) on all our saved SkillOpt trajectories. Its parameters are fixed by AutoGluon, independently of our runs. It sets patience=round(0.3τ+10)\mathrm{patience} = \mathrm{round}(0.3\tau + 10) (class default), where τ\tau is the round of the last improvement. Our rule uses the paper's defaults ε=0.01\varepsilon = 0.01 and δ=0.05\delta = 0.05 on every run.

In the table, all paired entries are AdaptiveES / our rule. Stop is the stopping round; “none” means no stop within the budget. Returns is the round of the returned artifact. Saved is the share of full-budget tokens not spent. “Same” means both rules return the same artifact, and ‡ marks the round-16 reward-hacking artifact.

RunStopReturnsSavedTest accuracy
SearchQA / DeepSeek11 / 41 / 175.1% / 91.6%Same
GSM8K / DeepSeek11 / 41 / 173.2% / 89.8%Same
LiveMath / DeepSeeknone / 816‡ / 10% / 37.8%‡
OfficeQA / DeepSeek14 / 83 / 312.6% / 48.4%Same
SpreadsheetBench / DeepSeeknone / none11 / 110% / 0%Same
SearchQA / Qwen3-32Bnone / 1233 / 80% / 75.3%73.79% / 72.43%
GSM8K / Qwen3-32B17 / 85 / 162.9% / 84.5%95.20% / 95.20%
GSM8K (n=100n=100) / DeepSeek11 / 51 / 172.7% / 87.3%Same
SpreadsheetBench, new seednone / none8 / 80% / 0%Same

AdaptiveES never stops earlier than our rule, and our rule saves at least as many tokens on every run. On six runs, both rules return the same artifact. Of the remaining three, AdaptiveES returns the reward-hacking artifact on LiveMath; on GSM8K/Qwen3-32B, both artifacts score 95.20% on unseen test data. On SearchQA/Qwen3-32B, AdaptiveES runs the full budget and its round-33 artifact scores 73.79% on unseen test data, compared with 72.43% for ours, while spending more than four times as many tokens.

Output selection

The change-point output differs from the alarm-time incumbent in two settings: GEPA/GSM8K with DeepSeek V4 Flash and SkillOpt/GSM8K with Qwen3-32B (see Correction below). In both settings, the change-point output achieves the same unseen-test accuracy as the alarm-time incumbent while using fewer prompt tokens. Selecting this output can therefore reduce inference cost.

The table reports prompt tokens summed over the 500 unseen test queries. Savings are relative to the incumbent at the alarm.

SettingArtifactRoundTest accuracyPrompt tokensSaved
SkillOpt/GSM8K/Qwen3-32B (alarm: 8)Incumbent at alarm595.20%436,828—
SkillOpt/GSM8K/Qwen3-32B (alarm: 8)Change-point output195.20%203,82853.3%
GEPA/GSM8K/DeepSeek V4 Flash (alarm: 6)Incumbent at alarm597.20%478,136—
GEPA/GSM8K/DeepSeek V4 Flash (alarm: 6)Change-point output297.20%357,63625.2%

In these two settings, output selection reduces prompt-token usage without reducing observed unseen-test accuracy.

Correction

We previously stated that the change-point output equals the alarm-time incumbent in all experiments except GEPA/GSM8K with DeepSeek V4 Flash. SkillOpt/GSM8K with Qwen3-32B (Table 5) is a second exception.

In this setting, the change-point estimate returns the round-1 artifact, whereas the incumbent at the alarm (round 8) is the round-5 artifact, after two accepted updates raised validation accuracy from 0.94 to 0.96. On the 500 unseen test examples, both artifacts score 95.20%. Each update gained only two net validation items out of 200, and the combined validation gain did not translate into higher unseen-test accuracy.

Reviewer_XCKdOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses online stopping and output selection in iterative self-evolving LLM frameworks (SkillOpt, GEPA). Using per-item paired differences X_{t,i} = l(theta'_t, x_i) - l(theta_{t-1}, x_i) in {-1, 0, +1} on a validation set D_val of size n, the authors test the pre-change null hypothesis H_0^(s): E[Z_t | F_{t-1}] >= epsilon (epsilon = 0.01) via a restart betting wealth process W_t^(s) = prod_{u=s..t} prod_{i=1..n} [1 + lambda_{s,u} * (epsilon - X_{u,i})] (using predictable aGRAPA or fixed-mixture betting fractions lambda_{s,u} in [0, 1/2]). An alarm is triggered when M_t = max_{1 <= s <= t} W_t^(s) >= 1 / delta (delta = 0.05), and the system outputs the pre-transition incumbent theta_{nu_hat - 1} where nu_hat = argmax_{1 <= s <= T_alarm} W_{T_alarm}^(s). The authors prove ARL control (E[T_alarm] >= 1 / delta, Proposition 1) and O(1/n) localization error (Proposition 2), saving 48.4%–91.6% tokens across 5 benchmarks while preventing a round-16 reward-hacking skill on LiveMath.

Critical strengths
  • Decoupling Alarm Time from Output Selection: Recognizing that sequential detection incurs a post-change delay (T_alarm > nu) during which greedy gates can accept overfitted candidates, and reusing the restart maximizer nu_hat to roll back to theta_{nu_hat - 1}, is simple, zero-overhead, and effective (e.g., stopping at round 4 on SearchQA and returning round 1, saving 91.6% tokens with +0.43 pp unseen-test accuracy).
  • Strong Positive and Negative Controls: Under a single fixed configuration (epsilon = 0.01, delta = 0.05), the detector remains silent across all 16 rounds on SpreadsheetBench where accuracy steadily rises from 0.375 to 0.70 (Figure 3), yet stops at round 8 on LiveMath (returning round 1) well before the round-16 degenerate "answer A" exploit (Figure 4).
Critical weaknesses
  • Unrealistic Conditional Independence Assumption on a Fixed Validation Set (Proposition 1): Proposition 1 and Eq. (9) in Appendix A.1 require that X_{t,1}, ..., X_{t,n} are conditionally independent given F_{t-1}, where F_{t-1} contains the proposed candidate theta'_t, incumbent theta_{t-1}, and all validation outcomes from rounds 1 ... t-1. However, the exact same validation set D_val = {x_1, ..., x_n} is reused at every round t. Once x_1, ..., x_n are observed in round 1 (and used by the gate to select incumbents), F_{t-1} fixes the validation items; conditioned on F_{t-1} and theta'_t, the only randomness left in X_{t,i} is LLM decoding noise (which is zero under greedy decoding), violating the population sampling independence needed for E[L_t^(s) | F_{t-1}] <= 1.
  • Within-Round Product Over Validation Items Makes the Detector Hypersensitive to a Single Bad Proposal Round: Because Eq. (3) multiplies across all n validation items within round t (L_t^(s) = prod_{i=1..n} [1 + lambda_{s,t} * (epsilon - X_{t,i})]), the effective log-wealth increment in a single round scales linearly with n: approximately n * lambda_{s,t} * (epsilon - Z_t). Consequently, for n = 200 (SearchQA, GSM8K), if the LLM proposer generates just one bad candidate theta'_t at round t with Z_t = -0.08 (which is rejected by the greedy gate and leaves the incumbent theta_t = theta_{t-1} unharmed), a restart process with lambda >= 0.17 jumps by (1 + 0.17 * 0.09)^200 = 20.8 > 1 / delta = 20 in a single round. Thus, rather than testing whether the sequence of proposals across rounds has stalled, multiplying over n items causes a single rejected mutation on a large validation set (n >= 200) to immediately trigger termination.
Questions for authors
  • Clarification Requests:
    1. Initial Betting Fraction in aGRAPA (Appendix B, Line 919): In Appendix B, on the empty window s = t, aGRAPA sets lambda_{s,s} = lambda_0 in (0, 1/2]. What exact numerical value of lambda_0 is used in all experiments? If lambda_0 = 0.2 and n = 200, a single rejected candidate at any round t with Z_t <= -0.066 immediately drives W_t^(t) >= (1 + 0.2 * 0.076)^200 = 20.4 > 1 / delta = 20, stopping evolution on a single bad proposal.
    2. Scaling Behavior as Validation Set Size Grows: Why is the martingale wealth defined as a product over n items prod_{i=1..n} [1 + lambda * (epsilon - X_{u,i})] rather than a product over rounds using the bounded round-average Z_u in [-1, 1], i.e., prod_{u=s..t} [1 + lambda * (epsilon - Z_u)]? Under your current item-product formulation, if a user increases validation size from n = 200 to n = 2000 for higher precision, won't even a tiny single-round dip (Z_t = -0.005) immediately blow up the product over 2000 items past 1 / delta = 20?
  • Suggestions for Future Work:
    1. Round-Level vs. Item-Level Wealth Aggregation: Normalize the within-round log-wealth increment by n (or construct the e-process directly on round-level statistics Z_t) so that stopping sensitivity depends on the number of stalled evolution rounds rather than bursting on a single rejected proposal when n is large.
1 comment
AuthorsOpenReview ↗

Response to Reviewer XCKd

Thank you for the careful review, insightful questions and suggestions. We address your comments and questions below.

Q1. Initial Betting Fraction in aGRAPA (Appendix B, Line 919):

We use λ0=0.1\lambda_0=0.1 in all experiments and will state it in Appendix B.

Your calculation in the example (Zt≤−0.066)(Z_t \leq -0.066) is not how we construct the wealth process. Specifically, the restart wealth is a product over items, not a power of the round average:

Wt(t)=∏i=1n[1+λ0(ε−Xt,i)]=(1+λ0(ε+1))n10(1+λ0(ε−1))n01(1+λ0ε)n00+n11,W_t^{(t)}=\prod_{i=1}^{n}\bigl[1+\lambda_0(\varepsilon-X_{t,i})\bigr]=(1+\lambda_0(\varepsilon+1))^{n_{10}}(1+\lambda_0(\varepsilon-1))^{n_{01}}(1+\lambda_0\varepsilon)^{n_{00}+n_{11}},

where n10n_{10}, n01n_{01} and n00+n11n_{00}+n_{11} count deteriorations, improvements and ties.

For your example (n=200n=200, λ0=0.2\lambda_0=0.2, Zt≤−0.066Z_t\le-0.066), Zt≤−0.066Z_t\le-0.066 requires n10−n01≥14n_{10}-n_{01}\ge14. The wealth is largest at n10=14n_{10}=14, n01=0n_{01}=0:

Wt(t)=1.20214×1.002186=19.1<20.W_t^{(t)}=1.202^{14}\times1.002^{186}=19.1 < 20.

At our λ0=0.1\lambda_0=0.1 it is 1.10114×1.001186=4.6<201.101^{14}\times1.001^{186}=4.6 < 20. So our procedure will not easily stop when facing a single bad proposal.

Q2. Scaling Behavior as Validation Set Size Grows:

Why we use an item-level product rather than a round-level one? Because the item-level product uses the distribution of paired outcomes on the validation set (more information), not only their average. For each item, Xt,iX_{t,i} records whether the candidate improves, deteriorates or ties relative to the incumbent. The round average ZtZ_t keeps only the net difference (n01−n10)/n(n_{01}-n_{10})/n and discards how the candidate performs item by item. The item-level product keeps this information by depending on (n10,n01,n00+n11)(n_{10}, n_{01}, n_{00}+n_{11}) separately. Each round therefore contributes nn observations of the candidate's per-item performance instead of one, so our procedure is more powerful for stopping the self-improvement.

The round-level process ∏u[1+λ(ε−Zu)]\prod_u[1+\lambda(\varepsilon-Z_u)] is also valid, but it uses essentially one observation per round (i.e., net gain). On a plateau (Zu=0Z_u=0), which should stop very quickly, it needs log⁡20/log⁡(1+λε)\log 20/\log(1+\lambda\varepsilon) rounds to reach the alarm: 750 rounds at ε=0.01\varepsilon=0.01 and λ=0.4\lambda=0.4. With n=200n=200 item-level observations per round, this drops to 3.8 rounds, while the total runs generally have 16 or 40 rounds.

For your n=2000n=2000, Zt=−0.005Z_t=-0.005 example, Zt=−0.005Z_t=-0.005 means n10−n01=10n_{10}-n_{01}=10. By the formula in Q1, at λ0=0.1\lambda_0=0.1:

  • 10 deteriorations, 0 improvements, 1990 ties (the largest possible wealth): 1.10110×1.0011990=19.1<201.101^{10}\times1.001^{1990}=19.1 < 20;
  • 100 deteriorations, 90 improvements, 1810 ties: 1.101100×0.90190×1.0011810=7.81.101^{100}\times0.901^{90}\times1.001^{1810}=7.8.

In this example, neither case reaches 20 and the same ZtZ_t could also generate different wealth because the wealth depends on the per-item outcomes, not on ZtZ_t alone. In addition, a larger validation set provides more such outcomes per round, so our detector is more accurate for detecting when to stop.

W1. Unrealistic Conditional Independence Assumption on a Fixed Validation Set

The assumption does not require population sampling, and it holds on a fixed validation set. The validity proof uses independence in only one step:

E[∏i=1n(1+λs,t(ε−Xt,i))∣Ft−1]=∏i=1n(1+λs,t(ε−E[Xt,i∣Ft−1])),E\Bigl[\prod_{i=1}^{n}\bigl(1+\lambda_{s,t}(\varepsilon-X_{t,i})\bigr)\Bigm|\mathcal{F}_{t-1}\Bigr]=\prod_{i=1}^{n}\bigl(1+\lambda_{s,t}(\varepsilon-E[X_{t,i}\mid\mathcal{F}_{t-1}])\bigr),

which requires only that Xt,1,…,Xt,nX_{t,1},\dots,X_{t,n} are independent given Ft−1\mathcal{F}_{t-1}.

Ft−1\mathcal{F}_{t-1} contains everything observed before round tt is evaluated: the item-level outcomes Xu,iX_{u,i} of all earlier rounds u<tu < t, the gate decisions based on them, the incumbent θt−1\theta_{t-1}, and the candidate θt′\theta_t'. Reusing the same items across rounds therefore does not break the assumption, since their earlier outcomes and the gate decisions are part of what we condition on.

Given Ft−1\mathcal{F}_{t-1}, the validation items and both artifacts are fixed, and the only remaining randomness in Xt,iX_{t,i} comes from evaluating item xix_i in round tt. Each item is evaluated in a separate run, and these runs do not share randomness, so the Xt,iX_{t,i} are conditionally independent. Our experiments use the self-evolving systems' default sampled decoding with reasoning enabled, so this randomness is present.

W2. Sensitivity to a single bad proposal

Your calculation is different from how we construct the wealth process. As in Q1, the restart wealth is a product over items, not a power of the round average.

Using the formula in Q1, Zt=−0.08Z_t=-0.08 with n=200n=200 means n10−n01=16n_{10}-n_{01}=16. At λ=0.17\lambda=0.17:

  • 16 deteriorations, 0 improvements (the largest possible wealth): 1.171716×1.0017184=17.3<201.1717^{16}\times1.0017^{184}=17.3 < 20;
  • Another example: 32 deteriorations and 16 improvements give 1.171732×0.831716×1.0017152=10.81.1717^{32}\times0.8317^{16}\times1.0017^{152}=10.8.

No outcome pattern with Zt=−0.08Z_t=-0.08 reaches 20. At our λ0=0.1\lambda_0=0.1, the maximum wealth is 5.65.6.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Summary of Revisions

We thank all reviewers for their careful reading and constructive feedback. We are glad that the reviewers found our problem formulation useful and well motivated, in particular separating when to stop from what to output (XCKd, wfm2), and our procedure lightweight and plug-and-play, requiring no change to how candidates are generated or accepted and applying to both SkillOpt and GEPA (25ZT, wfm2). The reviewers also noted the large token savings with a principled statistical foundation and an explicit false-alarm guarantee (25ZT, wfm2), and the positive and negative controls under a single shared configuration, where the detector stays silent on the improving SpreadsheetBench run and stops on LiveMath before the round-16 reward-hacking artifact (XCKd).

We have uploaded a revised manuscript. The main points raised and how we address them are as follows.

1. Comparison with simpler stopping rules (25ZT, wfm2). Section 4.2 now compares our rule with AutoGluon's adaptive patience rule, whose parameters are fixed independently of our runs, on all nine saved SkillOpt trajectories (Table 2). It never stops earlier than our rule, and our rule saves at least as many tokens on every run. Appendix D adds tuned patience rules, a gain-threshold variant, and fixed short budgets. Only one patience value and a narrow range of budgets avoid every failure, and both can be identified only after the full trajectories are observed.

2. Output selection versus the alarm-time incumbent (25ZT, wfm2). Table 3 compares the two in the two settings where they differ. The change-point output has the same unseen-test accuracy and uses 53.3% and 25.2% fewer prompt tokens at deployment.

3. Validity of the item-level betting construction (XCKd). Section 3.1 now states that the conditioning information includes the item-level outcomes and gate decisions of earlier rounds, so reusing a fixed validation set does not break the conditional independence assumption. Appendix B states the initial betting fraction and gives the exact wealth contributed by a single round, showing that a single rejected candidate does not trigger an alarm.

4. Documentation and robustness (wfm2, 25ZT). Appendix C documents the default detector parameters and the configurations that determine each full-run budget. Table 12 now reports token savings for every detector setting, and within each benchmark every setting that stops returns the same artifact. Appendix E reports a repeated SpreadsheetBench run with a new seed, and Appendix F reports repeated test evaluations that quantify evaluation noise. Throughout the paper, we now describe test results as showing no significant difference rather than comparable performance.

5. Clarification of Figure 1 (25ZT). The caption now states that the green points show the unseen-test accuracy of each round's candidate.

We hope these revisions address the reviewers' concerns. Please let us know if there are any remaining questions or suggestions, and we will be happy to respond during the discussion period.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.