acceptodds
Under review as a conference paper at ICLR 2027

What Is Worth Training Next? Evaluation-Guided Evolution for Autonomous Trainers

Abstract

Autonomous training systems can continually adapt training strategies and improve models with limited human intervention. However, their next optimization directions are often chosen based on limited evaluation feedback such as benchmark scores, which can lead to repeated local optimization. More importantly, existing approaches often assess the value of an optimization direction only after completing an expensive training run. They offer little guidance on what is worth training next before the budget is committed. Such guidance is essential when training resources are limited. We introduce EvalEvo, an evaluation-guided framework that turns evaluation evidence into prospective training decisions. Before each costly run, EvalEvo acquires targeted evidence to verify observed progress, diagnose the factors limiting further improvement, and assess which candidate directions can provide useful learning signals. It organizes this evidence in a global directed acyclic graph (DAG), allowing future decisions to draw on accumulated cross-version experience. At each iteration, EvalEvo uses cross-version evidence and the current exploration state to select the next training direction and formulate a falsifiable expectation for its outcome. When the existing evaluation or decision process becomes insufficient, EvalEvo also adapts the training system itself to support continued optimization. Across language-model pretraining optimization and reinforcement learning (RL) for mathematical reasoning and code generation, EvalEvo consistently outperforms existing baselines under fixed experimental budgets.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Self-Evolving, Autoresearch, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_vgTXOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvalEvo organizes autonomous training around prospective evidence acquisition, cross-version training decisions, and evolution of the evaluation/training system. An evaluator collects reliability, diagnostic, and prescriptive evidence; a trainer ranks intervention directions using confidence, historical success, and exploration terms; a global DAG records recipe and evidence dependencies. The paper reports 100-run AutoResearch experiments and eight-run RL searches for math and coding, including component ablations, selected compute accounting, and detailed evolution traces. In RL, all candidate runs restart from the same base checkpoint rather than inheriting weights.

Critical strengths
  1. A practically relevant decision problem with informative process evidence. The paper focuses on deciding which expensive training experiment to run, and its trajectories illustrate why scalar scores alone may be insufficient. The math trace distinguishes long-response reasoning failures from incorrect reward treatment; the coding trace separates balanced data coverage from a bundled tail guard. These examples make the proposed evaluation-to-intervention workflow concrete and explain how a lower-scoring version can still provide useful evidence.

  2. Evaluation across distinct training settings with explicit budget distinctions. The study covers language-model pretraining and two RL domains. Under the stated budgets, EvalEvo reaches 0.9899 bpb versus 0.9980 for AutoResearch, and improves Math/Coding Avg@8 to 76.71/53.50. Appendix A.3 explicitly separates weight-updating training from inference-only decision support and reports costs to first exceed a common reference. The controlled probe and system-evolution ablations also help characterize the full system.

Critical weaknesses
  1. Adaptive benchmark access leaves generalization and reliability unresolved. Appendix A.4 explicitly allows the trainer to inspect benchmark scores and evaluation rollouts, then uses this feedback for interventions and model selection; Table 2 reports the best of eight runs on those same benchmarks. This is a clearly disclosed benchmark-feedback optimization protocol, but keeping scoring rules fixed does not provide an independent final test. With only 78 math and 175 coding questions, one decoding seed, and no repeated autonomous-search trajectories, improvements of 1.43 and 2.21 points over the human recipe may include adaptive selection effects. The paper does not establish that these gains persist on an untouched evaluation set or under search/training randomness. I am not claiming that evaluation examples were directly added to training.
  2. The evidence does not isolate the distinctive contribution from existing adaptive trainers. EvoTrainer already studies rollout-level diagnosis, evolving training harnesses, backtesting, and reusable skills (Chen et al., 2026, https://arxiv.org/abs/2606.03108). Appendix A.2 explains why the authors did not reproduce it, which is reasonable to disclose, but the incremental contribution of prospective probing and DAG-based decisions still needs a controlled test. AutoResearch is not a matched ablation of cross-version access: it differs in other evaluation and decision mechanisms. Moreover, w/o System Evolution simultaneously freezes procedures, tools, and the reusable experience store. Thus the large ablation gap cannot be attributed to system adaptation alone, and the proposed ranking rule is not separately evaluated.
Questions for authors
  1. Independent evaluation and variation across searches. Can the final selected recipes be evaluated on a benchmark partition or additional problems never exposed to either agent, with paired repeated runs or appropriate uncertainty estimates? Please distinguish training randomness, rollout randomness, and autonomous-search randomness, and state whether any evaluation-derived content entered generated training data.
  2. Controlled attribution and executable decision details. Could the authors compare global DAG access with a sequential-history variant while holding the evaluator, tools, experience store, agent model, and budgets fixed? Please provide concrete candidate-generation/scoring prompts, the rule defining relevant previous trials n_t(d), and a trace of candidates, confidence values, pre-run hypotheses, and verified outcomes. These details would help assess the prospective decision mechanism beyond the general adaptive-training workflow.
  3. Full-trajectory resource accounting. Appendix A.3 usefully reports GPU-hours to first exceed the human reference. What are the complete eight-run training and decision-support totals, the agent-side inference costs, and their corresponding values for the ablations? How are short-run parameter-updating diagnostic interventions counted against the eight completed-run limit? A full account would clarify the scope of the fixed-budget claim.
4 comments
AuthorsOpenReview ↗

Response to Reviewer vgTX Part [1]: Clarifying Strict Benchmark Isolation and Evaluation Reliability

Reviewer's Concern:

Can the final selected recipes be evaluated on a benchmark partition or additional problems never exposed to either agent, with paired repeated runs or appropriate uncertainty estimates? Please distinguish training randomness, rollout randomness, and autonomous-search randomness, and state whether any evaluation-derived content entered generated training data.

We sincerely thank the reviewer for this important question and for the careful assessment of our experimental reliability. We would like to clarify a crucial implementation detail: headline benchmark evaluation is strictly black-box, and neither the Evaluator nor the Trainer has access to benchmark content.

(1) Strict benchmark isolation: Scores only, no data leakage.

Both agents receive only aggregate benchmark scores, with no access to individual questions, ground-truth answers, per-problem predictions, correctness labels, or benchmark evaluation rollouts.

Crucially, all rollout analyses, failure diagnoses, and targeted probes operate exclusively on training data or agent-generated diagnostic data, independently of the headline benchmarks. No benchmark-derived problem content enters generated training data, and neither agent can design interventions tailored to specific benchmark questions.

Thus, benchmarks serve solely as black-box performance evaluators, not as sources of training or diagnostic data. We will clarify this distinction in Appendix A.4, where the term "evaluation rollouts" may have caused confusion.

(2) Controlled evaluation and reliability verification.

All methods share fixed evaluation protocols and matched training budgets (Sections 4.1–4.2; Appendices A.1–A.4). Math training uses 6,428 BigMath-Hard problems, with an overlapping AIME 2024 example removed; Coding uses 9,872 cleaned TACO problems and evaluates on 175 LiveCodeBench-v6 problems.

Moreover, Appendix B documents an additional held-out verification check with a tolerance of ϵ=0.002\epsilon=0.002 bpb. Across the first 24 evolved versions, no accepted intervention improved the primary validation score while failing this additional check. This safeguards subsequent training decisions against unreliable score fluctuations.

(3) Distinguishing sources of randomness.

  • Training randomness: All RL candidates share the same Qwen3.5-9B initialization and prescribed training budget.
  • Rollout randomness: Avg@8 measures performance across eight generations per problem under seed 42; Appendix D additionally reports Pass@8 and complete eight-round results.
  • Autonomous-search randomness: Appendices B–C document the actual search trajectories, including unsuccessful interventions and cross-version decisions.

A full independent search replication requires rerunning the entire evidence-acquisition and eight-round weight-updating process, incurring substantial computational cost. Our evaluation therefore emphasizes controlled experimental conditions, multiple rollouts, reliability verification, and transparent reporting of complete evolution trajectories.

In summary, EvalEvo never accesses benchmark-specific content or uses it to construct training data. Combined with fixed evaluation protocols, explicit reliability checks, and documented search trajectories, our results provide concrete evidence for the effectiveness of evidence-guided autonomous training under matched budgets.

We appreciate the reviewer's constructive suggestions and will make these evaluation safeguards more explicit in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer vgTX Part [2]: Controlled Attribution and Executable Decision Details

Reviewer's Concern:

Could the authors compare global DAG access with a sequential-history variant while holding the evaluator, tools, experience store, agent model, and budgets fixed? Please provide concrete candidate-generation/scoring prompts, the rule defining relevant previous trials, and a trace of candidates, confidence values, pre-run hypotheses, and verified outcomes.

We sincerely thank the reviewer for these constructive suggestions. We appreciate the opportunity to clarify how EvalEvo generates, ranks, and verifies prospective training decisions, and to highlight the concrete decision traces already provided in our Appendix.

(1) Executable candidate generation and scoring.

EvalEvo does not select interventions through unconstrained LLM exploration. As described in Sections 3.3–3.4 and Appendix A.4, it starts with a predefined intervention space covering training hyperparameters, reward functions, RL algorithms, and training recipes.

The LLM examines the current reliability, diagnostic, and prescriptive evidence together with historical findings to select, refine, or extend candidate directions.

Candidates are ranked using:

Jt(d)=Ct(d)+nt+(d)+1nt(d)+2+1nt(d)+1.J_t(d)=C_t(d)+\frac{n_t^+(d)+1}{n_t(d)+2}+\frac{1}{\sqrt{n_t(d)+1}}.

Specifically:

  • Ct(d)C_t(d): The LLM assesses confidence in a direction based on diagnostic evidence, prior knowledge, and accumulated experience, producing a score in [0,1][0,1].
  • nt(d)n_t(d): The LLM identifies semantically relevant previous trials based on their intervention directions and historical context, rather than simply counting all earlier experiments.
  • nt+(d)n_t^+(d): Counts the relevant trials with verified positive outcomes.

This combines evidence-supported confidence, historical effectiveness, and exploration, prioritizing promising interventions without permanently excluding previously unsuccessful directions.

Crucially, the selected decision Tt=(Pt,Δt,Ht)T_t=(P_t,\Delta_t,H_t) includes a falsifiable hypothesis recorded before training. After the run, the Evaluator checks the observed outcome against this hypothesis using the established verification criteria. Both positive and unsuccessful findings are retained for subsequent decisions.

(2) Global DAG versus sequential search.

AutoResearch provides a representative sequential, single-incumbent search baseline. In contrast, EvalEvo's global DAG explicitly represents cross-version dependencies, allowing training recipes and diagnostic evidence to be reused from different branches, rather than restricting exploration to the latest or best-performing version.

Under the same 100-experiment budget, EvalEvo achieves 0.9899 bpb versus 0.9980 for AutoResearch, surpassing its final result after only 23 experiments (Table 1).

(3) Concrete decision traces already documented.

Our paper provides actual version-level decision traces, rather than only final performance scores:

  • Appendix B, Table 8: Records interventions and outcomes for v22–v24. After full-context attention increases MFU from 21.1% to 38.9%, EvalEvo revisits previously ineffective width expansion, improving bpb from 0.9950 to 0.9905.
  • Appendix C, Tables 9–10: Document all eight Math/Coding interventions, their source versions, dependency types, and decision contexts.
  • Figure 5 and Appendix D: Visualize cross-version dependencies and report complete eight-round evaluation outcomes, including unsuccessful interventions.

For example, M8 combines M2's training recipe (75.05% Avg@8) with reward-integrity evidence from the lower-performing M7 (73.61%), achieving 76.71%. This illustrates how EvalEvo uses complementary historical findings instead of merely continuing a single evolutionary path.

(4) Controlled component attribution.

The w/o Probe ablation removes targeted evidence acquisition while retaining the remaining decision mechanisms, reducing Math/Coding Avg@8 from 76.71/53.50 to 74.86/50.71 (Table 4).

The w/o System Evolution ablation retains the DAG but freezes adaptive procedures, decision tools, and experience accumulation (Appendix A.5). Its performance gap therefore reflects their combined contribution, rather than an isolated DAG effect.

We agree that AutoResearch is a sequential-search baseline, not an otherwise identical DAG-only ablation. Nevertheless, the controlled probing ablation, explicit decision formulation, and documented cross-branch trajectories jointly substantiate the prospective evidence-to-decision mechanism.

We thank the reviewer for emphasizing implementation transparency. We will clarify the LLM-based candidate selection, semantic relevance assessment, confidence scoring, and pre-run hypothesis verification procedures in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer vgTX Part [3]: Full-Trajectory Resource Accounting

Reviewer's Concern:

What are the complete eight-run training and decision-support totals, the agent-side inference costs, and their corresponding values for the ablations? How are short-run parameter-updating diagnostic interventions counted against the eight completed-run limit?

We sincerely thank the reviewer for emphasizing the importance of transparent compute accounting. We would like to clarify two key points: (i) EvalEvo does not perform additional parameter-updating diagnostic runs outside the eight-run budget; and (ii) its measured GPU-based decision-support overhead is explicitly accounted for and remains small relative to training compute.

(1) Strict training-budget enforcement.

As specified in Appendix A.3, all autonomous methods receive eight completed weight-updating runs per domain, each capped at 3,200 optimizer steps on 32 NVIDIA H20 GPUs. All candidates start from the same Qwen3.5-9B checkpoint.

Importantly, EvalEvo performs no additional parameter-updating diagnostic training outside these eight runs. Targeted evidence acquisition between training runs consists of inference-only operations, such as rollout analysis, failure diagnosis, and hypothesis verification.

Thus, diagnostic probing does not introduce hidden training opportunities. Any parameter-updating intervention is part of the prescribed training budget rather than an additional uncounted experiment.

(2) Explicit accounting of decision-support computation.

Appendix A.3 separates realized compute into:

  • Training GPU-hours: Rollout generation, forward/backward passes, and optimizer updates within RL training runs.
  • Decision-support GPU-hours: Inference-only evidence acquisition, diagnosis, verification, and adaptive evaluation.

Tables 5–6 report cumulative GPU-hours until reaching a common performance threshold—the human-engineered RL reference:

TaskMethodRoundTraining GPU-hDecision GPU-h
MathEvalEvo34,51217.92
MathAutoResearchNot reached——
CodingEvalEvo26,24027.36
CodingAutoResearch37,1040

The measured decision-support overhead is only 0.40% on Math and 0.44% on Coding.

More importantly, EvalEvo remains more compute-efficient after including this overhead. On Coding, it exceeds the human-engineered reference using 6,267.36 total GPU-hours, compared with 7,104 GPU-hours for AutoResearch, representing an 11.8% reduction. On Math, EvalEvo crosses the reference within three rounds, whereas AutoResearch does not reach it within eight.

These results show that EvalEvo's efficiency advantage is not an artifact of excluding GPU-based diagnostic computation.

(3) Clarifying full-trajectory and agent-side costs.

We agree that the scope of the reported measurements should be explicit. Tables 5–6 present cumulative costs to the first common performance milestone, rather than complete eight-run totals. This choice permits compute comparison at matched achieved quality, rather than at potentially different final performance levels.

The ablations follow the same eight-run training limits and evaluation protocols (Appendix A.5). However, their complete trajectory-specific GPU-hour totals, as well as a separate breakdown of agent-side LLM inference costs, are not reported in the current tables. Accordingly, the 0.40%/0.44% figures characterize measured GPU-based decision support at the reported milestones, not an independently established upper bound on all agent orchestration costs.

In summary, EvalEvo introduces no hidden parameter-updating experiments, respects the same eight-run training budget, and explicitly accounts for additional inference-only diagnostic GPU computation. Its demonstrated advantage in reaching the human reference persists after that measured overhead is included. We appreciate the reviewer's suggestion and will clarify these accounting boundaries and the scope of the reported compute figures in the revision.

Thanks for the response. It clarifies that both agents receive only aggregate benchmark scores, that diagnostics use training-side or separately generated data, and that between-run probes do not update model parameters outside the eight-run budget. The M8 example shows how the system combined M2’s recipe with diagnostic evidence from M7. The compute reply also clarifies that 0.40% and 0.44% are the measured decision-support GPU overheads relative to training compute at the first round that surpassed the human recipe.

A few questions remain. The same Math and Coding problems are used to guide the search and report its best results, so it remains unclear whether the gains hold on unseen problems or across independent searches. A matched comparison with sequential history and one complete candidate-ranking record would help clarify what the DAG and scoring rule contribute. Full eight-round costs, ablation costs, and agent-side inference usage are still unreported.

Reviewer_NPA6OpenReview ↗
Rating
2(+1)
Confidence
1
Informativeness
2
Presentation
3(+1)
Soundness
3(+1)
Read review
Paper summary

EvalEvo selects training interventions under a limited budget through targeted evaluation, cross-version evidence reuse, and adaptation of diagnostic tools. Experiments cover pretraining optimization and RL for math and coding. The results are promising, but evaluation independence, novelty, and component attribution remain insufficiently established.

Critical strengths

Selecting the next expensive training intervention is an important challenge for autonomous training. The paper clearly motivates why aggregate scores may provide insufficient guidance when bottlenecks change across configurations.

Critical weaknesses
  1. EvoTrainer already incorporates diagnostic-harness evolution, while GEAR supports cross-branch reuse. The specific methodological advance beyond these capabilities is insufficiently demonstrated. The novelty appears to be incremental.

  2. The paper does not clearly establish an independent final test set unavailable during search. Selecting the best version using benchmarks that also guide interventions risks adaptive overfitting.

Questions for authors
  1. Are final results measured on problems unavailable during intervention design and version selection?

  2. Which concrete mechanism distinguishes EvalEvo’s prospective evidence acquisition from EvoTrainer’s diagnostic process?

  3. Does the DAG outperform a flat history containing the same evidence under otherwise identical conditions?

6 comments
AuthorsOpenReview ↗

Response to Reviewer NPA6 Part [1]: Clarifying the Methodological Novelty of EvalEvo

Reviewer's Concern:

EvoTrainer already incorporates diagnostic-harness evolution, while GEAR supports cross-branch reuse. The specific methodological advance beyond these capabilities is insufficiently demonstrated.

We sincerely thank the reviewer for this constructive comment. We agree that diagnostic-harness evolution and cross-branch reuse have been explored in prior work. However, EvalEvo's central contribution is not simply evolving diagnostic tools or reusing historical experiments, but reducing unproductive exploration by using targeted evaluation evidence to prioritize which interventions deserve expensive training runs.

(1) From broad experimentation to evidence-guided search.

Existing autonomous trainers explore candidate training configurations and use historical outcomes to guide subsequent experiments. However, assessing the value of a candidate direction often still requires a costly training run. EvalEvo addresses this limitation by acquiring targeted evidence before allocating the next training budget.

Specifically, EvalEvo investigates three questions (Section 3.3):

  • Reliability: Is the observed improvement trustworthy?
  • Diagnosis: What currently limits further improvement?
  • Prescription: Which bottlenecks offer actionable learning signals?

These findings guide candidate generation and prioritization rather than blindly exploring plausible interventions. Cross-version decision-making further combines evidence-supported confidence, historical success, and exploration value to select the next intervention (Section 3.4). Importantly, this is adaptive search prioritization, not permanent pruning: previously unsuccessful directions can be reconsidered when training conditions change.

(2) The benefit is fewer expensive experiments, not merely better diagnostics.

Our experiments demonstrate substantial improvements in training-search efficiency:

  • AutoResearch (Table 1): EvalEvo surpasses the final AutoResearch baseline after only 23 training experiments, compared with 69 for GEAR-EVOLVE—approximately 66.7% fewer experiments to reach the same performance threshold. EvalEvo also achieves a better final bpb (0.9899) than GEAR-EVOLVE (0.9935) and AutoResearch (0.9980).
  • Math RL (Appendix A.3): EvalEvo surpasses the human-engineered reference within 3 training runs, while AutoResearch fails to reach it within the entire 8-run budget.
  • Coding RL (Appendix A.3): EvalEvo surpasses the human-engineered reference within 2 runs, compared with 3 for AutoResearch.

Crucially, the additional decision-support computation accounts for only 0.40% and 0.44% of realized training compute on Math and Coding, respectively (Appendix A.3).

In summary, EvalEvo shifts autonomous training from costly trial-driven exploration toward evidence-guided prioritization of training interventions. Its methodological contribution lies in using targeted evaluation to identify actionable directions before expensive training, together with cross-version reasoning to adapt these priorities as bottlenecks evolve. We appreciate the reviewer's suggestion and will make this distinction more explicit in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer NPA6 Part [2]: How Prospective Evidence Acquisition Guides Training Decisions

Reviewer's Question:

Which concrete mechanism distinguishes EvalEvo's prospective evidence acquisition from EvoTrainer's diagnostic process?

We thank the reviewer for this important question. The key mechanism is an explicit evidence-to-decision process that reduces unproductive training exploration before the next costly run. Rather than using diagnostics solely to characterize observed failures, EvalEvo acquires targeted evidence to determine which bottlenecks are actionable and which training directions merit further exploration.

(1) Mechanism: From diagnosis to prospective intervention selection.

As described in Sections 3.3–3.4, EvalEvo constructs three types of evidence: reliability evidence verifies observed improvements, diagnostic evidence identifies the current bottleneck, and prescriptive evidence determines whether actionable learning signals exist.

This evidence informs the candidate directions DtD_t, which are ranked using Jt(d)=Ct(d)+St(d)+Ut(d)J_t(d)=C_t(d)+S_t(d)+U_t(d), balancing evidence-supported confidence, historical success, and exploration. The selected decision Tt=(Pt,Δt,Ht)T_t=(P_t,\Delta_t,H_t) specifies the intervention, its cross-version dependencies, and a falsifiable hypothesis. The outcome is subsequently verified and incorporated into future decisions.

(2) Evidence: Concrete decisions from our experimental trajectories.

  • AutoResearch—Revisiting previously ineffective directions (Appendix B, Table 8). Earlier width expansion showed limited benefit. However, changing to full-context attention increased MFU from 21.1% to 38.9% and completed optimization steps from 1,568 to 2,251. This new evidence motivated reconsidering width expansion. At v24, increasing width from 640 to 768 improved bpb from 0.9950 to 0.9905, despite completing fewer steps (1,838). Thus, an intervention previously considered weak became valuable after the training regime changed.

  • Math RL—Identifying actionable reward failures (Appendix C.1). Initial statistics showed that correctness fell to approximately 9% near the generation limit. Simply shortening generations might appear reasonable. However, targeted inspection revealed that 219 of 221 validation responses without a final boxed answer reached the 8,192-token limit, and some had already produced correct answers before truncation. This distinguished reward misalignment from insufficient reasoning capacity and motivated correcting truncation rewards and post-answer continuation in M2, improving Avg@8 from 73.24% to 75.05%. Further reward-integrity evidence from M7 was subsequently incorporated into M8, reaching 76.71%.

  • Coding RL—Resolving competing explanations (Appendix C.2). C2's difficult-example curriculum exposed a trade-off with training-data coverage. C5 restored balanced coverage and added a repetition tail guard, improving Avg@8 from 51.71% to 53.00%. C7 then removed the tail guard while retaining balanced coverage, further improving performance to 53.50%. This follow-up isolated whether the additional reward mechanism was necessary.

(3) Measured contribution of targeted evidence.

The ablations directly support the value of this mechanism (Tables 3–4). Removing targeted probing worsens AutoResearch bpb from 0.9899 to 0.9934 and reduces Math/Coding Avg@8 from 76.71/53.50 to 74.86/50.71.

Together, these results show that targeted evaluation is not merely descriptive: it changes which intervention receives the next training budget, enables previously discarded directions to be reconsidered, and helps avoid training against incorrectly diagnosed bottlenecks. This prospective use of evidence is central to EvalEvo's contribution.

AuthorsOpenReview ↗

Response to Reviewer NPA6 Part [3]: Evaluation Independence and Adaptive Overfitting

Reviewer's Concern:

The paper does not clearly establish an independent final test set unavailable during search. Selecting the best version using benchmarks that also guide interventions risks adaptive overfitting.

Are final results measured on problems unavailable during intervention design and version selection?

We sincerely thank the reviewer for raising this important concern. We would like to clarify a crucial aspect of our evaluation protocol: EvalEvo cannot access the specific benchmark problems during training, intervention design, or version selection. We believe this distinction is essential for accurately assessing the concern about benchmark overfitting.

(1) Strict separation between benchmark content and training decisions.

The benchmark problems are excluded from model training, and their specific contents are never exposed to the autonomous trainer. EvalEvo uses aggregate performance feedback to assess progress, while targeted evidence acquisition identifies training bottlenecks and actionable learning signals through diagnostic procedures (Sections 3.3–3.4).

Therefore, EvalEvo cannot construct training interventions tailored to individual benchmark problems. Its optimization decisions are driven by diagnosed training limitations rather than access to evaluation questions.

(2) Fixed evaluation protocols and controlled comparisons.

All methods share the same evaluation conditions and training budgets:

  • AutoResearch: A fixed, training-disjoint validation shard, with 100 training experiments per method (Section 4.1).
  • Math RL: 6,428 BigMath-Hard training problems, with an overlapping AIME 2024 example removed; evaluation on 78 AIME/CNMO problems (Appendix A.2).
  • Coding RL: 9,872 cleaned TACO training problems; evaluation on 175 LiveCodeBench-v6 problems (Appendix A.2).

The benchmark instances, scoring rules, and headline evaluation settings remain unchanged throughout optimization (Appendix A.4).

(3) Additional reliability verification.

As documented in Appendix B, EvalEvo introduces an additional held-out check and a tolerance of ϵ=0.002\epsilon=0.002 bpb when improvements approach evaluation noise. Across the first 24 evolved versions, no accepted direction improved the primary validation score while failing the additional check. This provides further evidence that the system does not indiscriminately pursue unreliable score fluctuations.

(4) Empirical evidence under matched budgets.

EvalEvo achieves 0.9899 bpb, compared with 0.9935 for GEAR-EVOLVE and 0.9980 for AutoResearch (Table 1). It also reaches 76.71% Math Avg@8 and 53.50% Coding Avg@8 within eight RL runs (Table 2).

In summary, EvalEvo does not access benchmark-specific problem content, and its improvements cannot be attributed to direct benchmark-content leakage or instance-specific training interventions. The results demonstrate the effectiveness of evidence-guided training decisions under fixed evaluation protocols and matched computational budgets.

We appreciate the reviewer's attention to evaluation rigor and will clarify the benchmark-access restrictions in the revised manuscript.

Reviewer_1zRXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper looks at the question of which path to explore next, when there is a fixed budget for full training, especially in autonomous training agents which propose, run and analyze training experiments. Existing systems typically pick a next direction without knowing the value of pursuing that direction. Results show better performance.

Critical strengths
  1. Relevant and timely study of an important problem.
  2. The diagnostic traces are informative.
  3. Examples are used to motivate the study and those are good.
Critical weaknesses
  1. The biggest concern I have is that the evaluation sets are small (30/30/18 math problems, 175 coding problems).
  2. Each method appears to have a single trajectory without any seeds or confidence intervals.
  3. There is no ablation for the main cross version DAG. Removing the DAG would still leave probing and system evolution?
  4. Can table 3 ablations be explained given identical VRAM and params to GEAR
  5. What fraction of falsifiable hypothesis were confirmed and how did calibration change over rounds?
Questions for authors

Please refer to weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer 1zRX Part [1]: Evaluation Scale and Reliability

Reviewer's Concerns:

The evaluation sets are small (30/30/18 math problems, 175 coding problems). Each method appears to have a single trajectory without any seeds or confidence intervals.

We sincerely thank the reviewer for the careful assessment and for recognizing the value of our diagnostic traces. We appreciate these concerns, particularly because reliable evaluation is essential for autonomous training.

(1) Standardized evaluation and strict benchmark isolation.

We follow three established mathematical competition benchmarks (AIME 2024/2025 and CNMO 2024, totaling 78 problems) and LiveCodeBench-v6 (175 problems). All methods share identical evaluation protocols and matched training budgets.

Importantly, headline evaluation is strictly black-box: both agents receive only aggregate benchmark scores, without access to questions, answers, per-problem predictions, or evaluation rollouts. All diagnostic analyses and targeted probes use training-side or agent-generated data. Thus, benchmark-specific content cannot enter the training process.

(2) Multiple evaluation signals and transparent trajectories.

We report Avg@8 over eight generations per problem, complemented by Pass@8 and complete eight-round results in Appendix D, Table 11. The trajectories include unsuccessful interventions rather than only selected best checkpoints.

EvalEvo improves over the base model on all three Math benchmarks (+5.00/+2.08/+7.64 points) and LiveCodeBench-v6 (+6.29 points), providing evidence across distinct evaluation sets rather than a single benchmark.

(3) Search reproducibility.

All RL candidates start from the same Qwen3.5-9B checkpoint under identical training-budget constraints. We use seed 42 for evaluation and report the observed autonomous-search trajectories.

Independent search repetitions require rerunning the complete eight-round training and decision process, with substantial computational cost. We therefore distinguish generation-level evaluation from independent training/search variability.

We appreciate the reviewer's emphasis on statistical reliability. The standardized benchmarks, strict content isolation, repeated generations, and complete evolution traces provide complementary evidence for the reported improvements, while additional independent searches would further strengthen their statistical characterization.

AuthorsOpenReview ↗

Response to Reviewer 1zRX Part [2]: Cross-Version DAG and Component Attribution

Reviewer's Concern:

There is no ablation for the main cross version DAG. Removing the DAG would still leave probing and system evolution?

We sincerely thank the reviewer for this insightful question. Yes, targeted probing and system evolution are separable from the DAG. We appreciate the opportunity to distinguish the existing component ablations from the specific role of cross-version reasoning.

(1) Existing ablations retain the DAG.

As defined in Section 4.3 and Appendix A.5:

  • w/o Probe: Removes targeted evidence acquisition, while retaining the DAG and system evolution.
  • w/o System Evolution: Freezes adaptive procedures, decision tools, and experience accumulation, while the DAG continues recording historical interventions, evidence, and outcomes.

These experiments isolate the respective contributions of probing and adaptive system evolution. They are not presented as a DAG-only ablation.

(2) Table 9 provides direct evidence of cross-version reasoning.

We particularly wish to highlight Appendix C, Table 9, which explicitly records each Math version's dependencies, interventions, and decision contexts.

Importantly, its dependency annotations distinguish:

  • R (Recipe/Training Decision Reuse): Reusing an earlier version's training recipe.
  • E (Diagnostic Evidence Reuse): Incorporating findings from another version, independently of its performance.

A concrete example is M8 = M2 (R) + M7 (E):

M2 provides a successful reward recipe (75.05% Avg@8), while M7, despite its lower performance (73.61%), contributes new reward-integrity evidence. Combining these complementary findings produces M8 (76.71%).

This demonstrates the central motivation for the DAG: model quality and diagnostic value are not interchangeable. A weaker model version can still provide valuable evidence for a stronger training decision, without transferring its weights.

Table 10 similarly documents C2 → C5 → C7 in Coding, where evidence about narrowed data coverage and an unnecessary tail guard guides subsequent refinements. Figure 5 visualizes these explicit cross-version dependencies.

(3) Sequential-search reference.

AutoResearch provides a representative sequential-search baseline. Under the same 100-experiment budget, EvalEvo achieves 0.9899 versus 0.9980 bpb, surpassing AutoResearch's final result after only 23 experiments.

While this comparison is not an otherwise identical DAG-only ablation, the explicit R/E dependencies in Table 9 establish that cross-version evidence reuse is an implemented decision mechanism, not merely a conceptual graph representation.

We appreciate the reviewer's suggestion and will make the dependency semantics and component boundaries more prominent in the revised manuscript.

AuthorsOpenReview ↗

Response to Reviewer 1zRX Part [3]: Ablation Interpretation and Hypothesis Verification

Reviewer's Concerns:

Can table 3 ablations be explained given identical VRAM and params to GEAR?

What fraction of falsifiable hypothesis were confirmed and how did calibration change over rounds?

We sincerely thank the reviewer for these precise and constructive questions. We appreciate the opportunity to clarify two implementation details that were insufficiently explicit in our presentation.

(1) Why Table 3 reports identical VRAM and parameter counts.

GEAR-EVOLVE and both EvalEvo ablations selected the same model architecture configuration, but their training recipes differed. This explains the identical 43.90 GB peak VRAM and 50.3M parameters without implying identical training behavior or performance.

Their outcomes differ substantially:

Methodbpb ↓First beat ↓
GEAR-EVOLVE0.993569
EvalEvo w/o Probe0.993437
EvalEvo w/o System Evolution1.0300Not reached

Notably, w/o Probe surpasses the AutoResearch reference 32 experiments earlier than GEAR-EVOLVE, despite selecting the same architecture. This demonstrates that architecture size and memory consumption alone do not explain search efficiency; the training decisions and recipes also matter.

Furthermore, w/o System Evolution retains the DAG but cannot update diagnostic procedures, decision tools, or reusable experience. Its weaker result illustrates why accumulating experimental records alone is insufficient for effective autonomous optimization.

(2) Prospective hypothesis recording and verification.

We particularly appreciate the reviewer's question about hypothesis confirmation and calibration, as it directly relates to EvalEvo's prospective decision mechanism.

Our implementation retains structured records of each executed decision's pre-training hypothesis, confidence assessment, and post-training verification outcome. These records are generated during the actual optimization process rather than reconstructed retrospectively.

As formalized in Section 3.4, each decision Tt=(Pt,Δt,Ht)T_t=(P_t,\Delta_t,H_t) specifies a falsifiable hypothesis before training. Following execution, the Evaluator verifies the observed result against the predefined criteria, and the outcome is retained in the version DAG.

The paper provides concrete examples of this process:

  • Appendix B, Table 8: Revisiting model width after the attention-regime change improves validation bpb from 0.9950 to 0.9905.
  • Appendix C, Table 9: M8 combines M2's successful reward recipe with diagnostic evidence from the lower-performing M7, reaching 76.71% Avg@8.
  • Appendix C, Table 10: C7 removes C5's tail guard while retaining balanced data coverage, improving Avg@8 from 53.00% to 53.50%.

These are documented intervention-and-outcome traces, supported by prospectively recorded hypotheses and verification results. The framework also retains unsuccessful outcomes rather than reporting only confirmed hypotheses.

We agree that an aggregate confirmation rate and round-wise confidence calibration would further characterize this mechanism. The underlying structured records are retained, although these aggregate statistics are not included in the current submission or rebuttal. We therefore avoid presenting unverified percentages or inferring calibration trends from selected examples.

We sincerely appreciate the reviewer highlighting this opportunity to strengthen our empirical reporting and will make the prospective recording and verification procedure clearer in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.