acceptodds
Under review as a conference paper at ICLR 2027

Learning What to Practice: Diagnosis-Guided Self-Evolution for Language Models

Abstract

Self-play supports the self-evolution of language models, but solver performance can plateau or decline across rounds without guidance. Existing unguided methods typically use difficulty, learnability, or diversity signals to keep questions challenging and varied, without identifying which unresolved reasoning weaknesses to target. Existing guided methods rely on external task resources such as human examples, document corpora, or specified difficulty targets. We introduce DiagEvo, which guides question generation using the solver's failure history from self-play, without external task resources. Its diagnostician extracts recurring error causes and stores them in an error-cause memory. The memory groups related causes under skill nodes and tracks each as Active or Mastered according to self-consistency on targeted questions. The challenger uses these states and recurrence counts to balance cause-targeted generation with free exploration. Double-confidence filtering retains intermediate-difficulty questions only when the most common solver answer has a clear vote lead. With the default 4B diagnostician, DiagEvo outperforms all baselines in mean accuracy across nine benchmarks for each solver: Qwen3-4B, Qwen3-8B, and OctoThinker-8B. On Qwen3-8B, DiagEvo reaches 72.3% mean accuracy across five mathematical reasoning benchmarks, 4.5 percentage points above R-Zero. Its overall mean accuracy across nine benchmarks is 57.4%, 3.5 percentage points above SPICE. Ablations show that mixed generation, memory-state updates with cross-state stitching, and double-confidence filtering contribute to these gains.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Large Language Model Reasoning, Self-Play, Self-Evolution, Curriculum Learning, Reinforcement Learning

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_b19VOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

DiagEvo extends R-Zero-style challenger–solver self-play with an error-cause memory. A frozen external LLM (the diagnostician) compares failed and pseudo-label-agreeing trajectories, extracts transferable error causes, and organizes them under skill nodes with Active/Mastered states. These states and recurrence counts set the mix between free exploration and cause-targeted question generation. A double-confidence filter keeps only questions whose leading answer clearly beats the runner-up. On Qwen3-4B/8B-Base and OctoThinker-8B, the method reports gains of about 1.1–1.3 points in overall average over DARC.

Critical strengths

The method is clearly described and the experiments are carefully executed. The paper uses three solvers and three runs each, ablates every component, and validates pseudo-label accuracy with human verification. This makes the reported results credible, but it does not compensate for the limited contribution.

Critical weaknesses
  1. Limited novelty and incomplete related work. The method is an incremental extension of R-Zero, and each of its components has close precedents: contrasting successful and failed trajectories to extract reusable insights comes from ExpeL, AutoGuide, and ReasoningBank; targeting capability gaps identified from failures is the idea of TRACE, which receives only one sentence of discussion; double-confidence filtering is a minor variant of existing consistency- or entropy-based filtering (SRT, EMPO). Highly relevant prior work is also missing. EvoCurr (Cheng et al., 2025) uses a curriculum LLM that adaptively generates intermediate tasks based on solver performance and failures, which directly overlaps with the paper’s central claim. Learned curriculum methods for RL reasoning, such as Self-Evolving Curriculum (Chen et al.), are likewise not discussed. The paper should clarify what is novel beyond applying feedback-driven curriculum generation to label-free self-play.

  2. Insufficient and largely saturated benchmarks. Several benchmarks offer little headroom: GSM8K is above 91% for the Qwen3-8B base model before training. Others are too small or noisy to separate methods: AMC has 40 problems, GPQA-Diamond shows standard deviations up to 1.3, and all methods score about 11–12 on BBEH. As a result, the 1.1–1.3 point margin over DARC, whose scores are copied single-run results, is hard to interpret. Training and curriculum construction are limited to math. There are no harder or more discriminative benchmarks (AIME 24/25/26, HMMT, LiveCodeBench), and no agentic or multi-turn settings, where an error-cause memory should matter most.

Questions for authors
  1. Please discuss EvoCurr and other feedback-driven curriculum methods, and clarify the novelty of DiagEvo relative to them and to TRACE.

  2. Can you report results on harder, less saturated benchmarks (e.g., AIME 24/25/26, HMMT, LiveCodeBench) and on agentic or multi-turn tasks?

  3. The diagnostician is an external post-trained model, and gains grow with its scale. Can you add controls for: R-Zero with the same diagnostician but without the memory; the solver’s own base model as the diagnostician; direct distillation from the diagnostician?

  4. This would show whether the memory mechanism, rather than access to a stronger model, drives the gains.

5 comments
AuthorsOpenReview ↗

Response to Reviewer b19V (Part 1/3)

Thank you for your comments.

Q1.

Please discuss EvoCurr and other feedback-driven curriculum methods, and clarify the novelty of DiagEvo relative to them and to TRACE.

A1. DiagEvo uses recurring error causes to keep self-play directed at unresolved weaknesses while both the challenger and solver learn across rounds. We share with TRACE and related methods the broad idea of using feedback to guide learning. DiagEvo's contribution is how it maintains and uses this feedback during self-play: the same error cause connects diagnosis, new question generation, and progress tracking as weaknesses persist, improve, or return.

TRACE, EvoCurr, and Self-Evolving Curriculum (SEC) use feedback for capability training, intermediate task design, or category selection, respectively. None uses alternating RL to train both a challenger LLM that generates new questions and a solver LLM. DiagEvo builds on R-Zero's alternating training and introduces error-cause memory to maintain practice priorities throughout this process.

Our starting point is the problem in §1: “Different questions can expose the same error cause, so difficulty alone does not identify what the solver should practice next.” In self-play, the question distribution and the solver change together. A harder or longer question can leave the same reasoning weakness unresolved, and a weakness that no longer receives direct practice can recur. Difficulty and learnability indicate whether a question is useful for training; they do not track which recurring weaknesses remain.

We therefore retain transferable error causes from failed trajectories across rounds to guide future question generation.

This need to maintain practice priorities explains the memory design. Diagnosis uses current failures at the end of each round. Active and Mastered states determine when to target a cause directly, keep it for cross-state stitching, or reactivate it after a matching failure. Recurrence counts prioritize Active causes and set the mix of cause-targeted generation and free exploration. The challenger samples both its update questions and the solver curriculum under this memory schedule.

The works you cited address related aspects of experience reuse, targeted capability training, adaptive curricula, and label-free RL, and comparing what their feedback or memory controls helps locate our contribution:

WorkCentral ideaRole of feedback or memory
ExpeLLearn reusable insights from success and failurePast insights and examples guide inference.
AutoGuideLearn when a guideline appliesGuidelines from offline trajectories are selected for the current context.
ReasoningBankRetain reasoning experience across tasksMemory grows during test interactions and guides later actions.
TRACETrain the capabilities behind recurring failuresDiagnosed gaps define verifiable environments for capability adapters and their composition.
EvoCurrReach a difficult final task through verified intermediate tasksExecution feedback changes the curriculum while preserving accepted progress.
Self-Evolving CurriculumSelect problem categories by expected learning gainA learned curriculum policy changes which categories are sampled for training.
SRTTrain with rewards from the model's own majority voteAgreement supplies rewards without a ground truth verifier.
EMPOTrain on unlabeled questions by reducing semantic entropyEntropy supplies the training signal and filters questions.
DiagEvoUse recurring error causes to guide what to practice as self-play evolvesThe same cause connects diagnosis, generation, and progress tracking across questions and rounds.
AuthorsOpenReview ↗

Response to Reviewer b19V (Part 2/3)

A1 (continued).

TRACE also trains on diagnosed weaknesses. It identifies capability gaps from collected trajectories, builds verifiable training environments, and trains capability adapters and a router to combine them. It does not train the environment generator with RL. DiagEvo addresses how practice priorities should keep changing when both the challenger and solver are updated each round, including when to reduce direct targeting and when to revisit a recurring weakness.

EvoCurr evolves its curriculum and solver policy together. Its designer uses solver performance and failures to propose intermediate tasks toward a supplied final task. The designer LLM remains fixed. The solver generates policy code or trains an RL policy, as in Overcooked. Executability and score thresholds determine which tasks are accepted, and earlier progress is retained when a new task fails. DiagEvo has no supplied final task or externally verified path of intermediate tasks. It follows recurring error causes from failures on self-generated questions.

Self-Evolving Curriculum (SEC) updates its curriculum policy alongside RL training of the solver LLM. It treats problem categories as bandit arms and updates their values by expected learning gain. The policy selects existing training problems; it is not a challenger LLM that learns to generate new questions. DiagEvo extracts specific error causes from current failures to guide new question generation.

ExpeL, AutoGuide, and ReasoningBank use insights, guidelines, or reasoning memories to help solve incoming tasks. ReasoningBank also grows its memory without ground-truth labels. DiagEvo uses failure history to schedule training questions while both the challenger and solver evolve.

A frozen external diagnostician extracts the causes, while the training questions and failure trajectories come from self-play.

The experiments test this motivation directly. Under identical double-confidence filtering, pure free exploration peaks at 69.5 Math Avg in round 3, whereas DiagEvo continues improving to 72.3 in round 5 before declining to 72.0 and 71.4 in rounds 6 and 7 (Figure 3a). Tracking error causes thus extends the effective evolution phase and delays plateau and decline, rather than eliminating late-round degradation entirely. Across three runs (Table 2), removing state updates gives 70.0 ± 0.3 Math Avg and 37.1 ± 0.2 General Avg; keeping state updates without cross-state stitching gives 70.7 ± 0.3 and 37.5 ± 0.4; and full DiagEvo gives 72.3 ± 0.3 and 38.8 ± 0.4. In addition, the control in A3 using the same diagnostician without cross-round memory scores 70.0 and 37.2, testing whether current-round diagnoses alone suffice without maintaining failure history across rounds.

Double-confidence filtering supports this loop by combining an absolute consistency band with a 1.6× vote-margin constraint. Using the absolute constraint alone drops performance to 70.7 ± 0.3 Math Avg and 37.5 ± 0.2 General Avg (Table 2), supporting the relative constraint alongside memory guidance.

In the revised manuscript: Section 1 states the motivation; §2.3 explains the memory design. Section 4 and Appendices A.2 and A.3 compare the cited memory, curriculum, and label-free RL methods.

AuthorsOpenReview ↗

Response to Reviewer b19V (Part 3/3)

Q2.

Can you report results on harder, less saturated benchmarks (e.g., AIME 24/25/26, HMMT, LiveCodeBench) and on agentic or multi-turn tasks?

A2. The requested AIME and HMMT evaluations show that DiagEvo's gains extend to difficult mathematical benchmarks with low Base accuracy. DiagEvo has the highest mean across three runs on each added benchmark.

All methods use the same 30 problems per benchmark, 32 independent responses per problem, T=0.6, top_p=1, no top_k truncation, a 4,096-token limit, and identical prompts and grading. Scores are mean@32 (%), with mean ± sample SD computed across three runs before rounding. Base and released DARC each have one evaluation. All runs are in Appendix F, Table 13.

Qwen3-8B methodAIME 2024AIME 2025HMMT 2025-Feb
Base10.410.41.6
R-Zero, official training configuration13.9 ± 0.314.0 ± 0.92.2 ± 0.1
R-Zero, training settings matched to DiagEvo14.3 ± 0.314.7 ± 0.32.3 ± 0.1
DARC, rerun with the same time budget13.6 ± 0.513.6 ± 0.22.1 ± 0.3
DARC, released checkpoint13.513.91.9
DiagEvo16.4 ± 0.815.5 ± 0.33.6 ± 0.6

Base starts at 10.4% on AIME and 1.6% on HMMT, compared with 91.1% on GSM8K. HMMT accuracy remains low, and 32 responses do not increase the 30 distinct problems. AMC's 40 problems, GPQA-Diamond's variation across runs, and low BBEH scores also limit single-benchmark conclusions.

We supplemented DARC's single reported run with three reruns: 60 hours on 8 H200 GPUs, about 53% of full DARC runtime. Each runs 1,808 challenger updates and one solver pass of 109 updates. The shared validation rule selects step 109, evaluated with the same harness as R-Zero and DiagEvo. On the original nine benchmarks, DARC reaches 69.0 ± 0.3 Math Avg, 36.7 ± 0.6 General Avg, and 54.6 ± 0.4 Overall; DiagEvo reaches 72.3 ± 0.3, 38.8 ± 0.4, and 57.4 ± 0.3. DARC's released checkpoint scores 71.5, 37.5, and 56.4 under the same evaluation settings.

We have not evaluated AIME 2026, LiveCodeBench, or agentic and multi-turn tasks. Appendix C states: “DiagEvo builds its curriculum from mathematical questions.” Our study trains on math and evaluates zero-shot transfer to single-turn general reasoning. Code and interactive tasks need execution environments and multi-turn feedback beyond this setup.

In the revised manuscript: Section 3.2 summarizes the added results. Appendix F gives evaluation settings (Table 9), DARC comparisons (Table 10), and complete per-run AIME/HMMT results (Table 13), separate from the original averages. Section 3.1 and Appendix C, “Evaluation scope,” state the task coverage and its limits.

Q3.

The diagnostician is an external post-trained model, and gains grow with its scale. Can you add controls for: R-Zero with the same diagnostician but without the memory; the solver’s own base model as the diagnostician; direct distillation from the diagnostician?

This would show whether the memory mechanism, rather than access to a stronger model, drives the gains.

A3. We ran the three requested controls. Using the same diagnostician without memory or directly distilling its solutions gave lower scores than full DiagEvo. A frozen Base diagnostician retained most of the gain. All controls use the same Qwen3-8B initialization and evaluation harness. Each is one five-round run; full DiagEvo uses three.

Qwen3-8B settingMath AvgGeneral Avg
R-Zero + the same diagnostician, without memory70.037.2
Solver's own base model as diagnostician71.337.9
Direct distillation from the diagnostician67.133.4
DiagEvo, default diagnostician72.3 ± 0.338.8 ± 0.4

Same diagnostician, without memory. Current-round causes guide the next round's generation and are then discarded. There is no accumulated cause history, Active/Mastered states, skill-node layer, or cross-state stitching. Per-round free and cause-targeted question counts, filtering, and update budgets match full DiagEvo.

Solver's own base model as diagnostician. A separate frozen copy of the initial Qwen3-8B-Base checkpoint performs diagnosis and memory maintenance. It does not update with the solver; the rest of DiagEvo stays unchanged.

Direct distillation. The default diagnostician generates full solutions and answers for the questions used to train DiagEvo. The solver learns these through SFT from the same Base, without error-cause memory. Learning rate and checkpoint are selected on the shared validation set under a comparable training budget. This tests teacher supervision on shared questions; the first control tests generation without memory.

In the revised manuscript: Section 3.3 summarizes all three controls. Appendix G, “Additional Controls,” gives each setup, run count, and result (Table 19).

Reviewer_cg2ZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DiagEvo, a self-play framework that uses the solver's past failures to guide future question generation. It uses a separate diagnostician to compare reasoning trajectories and identify recurring error causes, which are organized in a hierarchical memory with Active and Mastered states. The challenger then uses this memory to generate questions targeting the solver's weaknesses while still allowing free exploration. The method also includes double-confidence filtering to reduce ambiguity in majority-vote pseudo-labels. Experiments on three solver models across nine reasoning benchmarks show improvements in average performance over existing self-play baselines. The authors also conduct ablation studies to examine the effects of the main components.

Critical strengths
  1. Rather than relying only on question difficulty, DiagEvo uses the solver's past failures to decide what to practice next. I find the idea of using error-cause memory to guide question generation interesting, as it addresses a limitation of existing self-play methods.
  2. DiagEvo improves the average performance across nine reasoning benchmarks for all three solver models. The ablation studies also provide useful evidence for the effectiveness of the main components. These results suggest that the approach works across different solvers rather than being limited to a particular model.
Critical weaknesses
  1. The quality of the error diagnosis is not directly validated. The diagnostician identifies error causes by comparing trajectories that agree or disagree with majority-vote pseudo-labels, without checking which reasoning is actually correct. A difference between two trajectories does not necessarily indicate the source of an error. While the ablations show that memory-guided generation helps, it remains unclear whether the gains come from accurately identifying and targeting specific weaknesses. I would like to see a direct evaluation of diagnosis quality or a baseline using unrelated error causes for targeted generation.
  2. The definition of skill mastery relies too heavily on self-consistency. A cause is marked as Mastered based on answer agreement rather than correctness, and the promotion score can even include questions removed by the filtering step. In principle, a single highly consistent question could trigger promotion without showing that the underlying weakness has been fixed. The paper also does not directly evaluate the correctness of the pseudo-labels retained for training. This makes it difficult to tell whether the memory is tracking genuine skill improvement or just changes in the solver's answer consistency.
Questions for authors
  1. How accurate are the error causes identified by the diagnostician? Have the authors manually checked whether they reflect actual reasoning errors?
  2. How much of the gain comes from targeting the correct weaknesses? Would using random or unrelated error causes for targeted generation lead to similar improvements?
4 comments
AuthorsOpenReview ↗

Response to Reviewer cg2Z (Part 1/2)

Thank you for your comments.

Q1.

How accurate are the error causes identified by the diagnostician? Have the authors manually checked whether they reflect actual reasoning errors?

A1. Our manual audit found that most sampled error causes describe actual reasoning errors, including some drawn from wrong reference labels. We found 167/200 valid diagnoses (83.5%), including 34/59 with wrong references. Causes were sampled randomly from memory over the first five rounds of the Figure 3 run. We traced each cause's first entry and checked its question, reference solution, failed solver trajectory, and diagnosis.

Reference answers were obtained by sampling 16 responses per question from dpsk-v4.1-flash-10.17, taking the mode, and manually checking the solution steps rather than treating the model majority vote as ground truth. Two master's students with backgrounds in mathematics and statistics independently checked each sample before seeing each other's judgments to determine whether the diagnosed error actually occurred in the source reasoning. They discussed disagreements using the original reasoning and the reference solution, and a third, more experienced researcher reviewed unresolved disagreements; all three were members of our team.

Reference pseudo-labelValid causes / inspected causesValidity
Correct133/14194.3%
Incorrect34/5957.6%
Overall167/20083.5%

Diagnosis is less reliable when the reference pseudo-label is wrong (59/200 cases). Even so, 34 of those 59 causes still identify a real mistake in the solver's reasoning steps. This audit directly evaluates the content of each diagnosis against the original trajectory rather than treating every difference between two trajectories as an error.

In the revised manuscript: Section 3.4 summarizes diagnosis reliability. Appendix H.1 gives the sampling and manual verification procedure, with results split by reference-label correctness in Table 21.

Q2.

How much of the gain comes from targeting the correct weaknesses? Would using random or unrelated error causes for targeted generation lead to similar improvements?

A2. The requested control shows that matched unrelated error causes do not give the same gains as the solver's diagnosed weaknesses. Replacing the target causes lowers Math Avg by 2.5 points and General Avg by 2.0 points. The control reaches 69.8 / 36.8 in one five-round Qwen3-8B run, versus 72.3 ± 0.3 / 38.8 ± 0.4 for full DiagEvo across three runs.

We replace target causes with causes from an independent pool, matched in mathematical domain, length, and phrasing format but unrelated to the current solver's diagnosed weaknesses. Generation proportions, double-confidence filtering, and all training settings remain fixed.

The manual diagnosis audit in A1 provides complementary evidence that the targeted causes reflect real reasoning errors.

In the revised manuscript: Section 3.3 summarizes the unrelated-cause control. Appendix G, “Additional Controls,” gives how causes are replaced, the run count, and the result (Table 19).

AuthorsOpenReview ↗

Response to Reviewer cg2Z (Part 2/2)

Additional concern from Critical Weaknesses (quoted in full).

The definition of skill mastery relies too heavily on self-consistency. A cause is marked as Mastered based on answer agreement rather than correctness, and the promotion score can even include questions removed by the filtering step. In principle, a single highly consistent question could trigger promotion without showing that the underlying weakness has been fixed. The paper also does not directly evaluate the correctness of the pseudo-labels retained for training. This makes it difficult to tell whether the memory is tracking genuine skill improvement or just changes in the solver's answer consistency.

Response. Our audit found that most promotions met the 70% accuracy threshold on their supporting questions. Mastered remains a reversible curriculum state (§2.3), not proof that the underlying weakness is resolved. Nine of 161 promotions fell below 70% verified accuracy, including 1 of the 12 based on a single question.

Promotion stops direct single-cause targeting, keeps the cause available for cross-state stitching, and allows reactivation to Active if a matching failure recurs.

Generated questions carry no supplied ground truth. Checking every candidate with a stronger model during training would add substantial inference cost, and repeated responses can still agree on an error. We therefore use the solver's existing responses to decide when to reduce direct practice during training, and we audit how often this decision lacks true correctness support offline using the protocol in A1: two independent checks, followed by a third researcher resolving remaining disagreements.

Equation 6 averages self-consistency over all candidates targeting a cause in that round, including filtered candidates. Filtering selects questions in the target consistency range with a clear lead for the majority answer. Promotion decides whether to continue direct targeting. A highly consistent question excluded from GRPO training can still inform this scheduling decision. Using all targeted candidates avoids assessing a cause only on questions that pass the difficulty filter.

A single targeted question can indeed trigger promotion when only one candidate targets that cause in a round. Using the verification protocol in A1, we audited all 161 promotions in the first five rounds of the Figure 3 run. For every promotion, we scored the solver's original responses against verified reference answers on each supporting candidate question (including filtered ones), averaged the per-question accuracy, and compared it with the 70% threshold. The overall rate of promotions below this threshold was 5.6%.

We also tracked a separate random sample of 100 early first promotions across three runs through round 5. Three cases could not be verified initially and were kept as uncertain. Nine of the remaining 97 verifiable promotions were below 70% true accuracy, and seven of those nine were reactivated at least once, including within the same round. Their outcomes at round 5 are:

Outcome among the nine early incorrect promotionsCount
Original promotion questions reach at least 70% true accuracy4
Active, but original questions remain below 70%3
Mastered, with original questions still below 70%1
Outcome at round 5 cannot be determined1

Four cases reached 70% on their original promotion questions, and three were Active but below 70%. These outcomes differ from the seven cases that were ever reactivated. The trace shows that incorrectly promoted causes can be reactivated. Some cases also improve on their original questions. It does not show that reactivation caused every improvement or establish gains on unseen questions.

The three-run ablations in Table 2 test whether these state updates help overall training despite imperfect promotion signals: no state updates gives 70.0 ± 0.3 Math Avg and 37.1 ± 0.2 General Avg; state updates without cross-state stitching gives 70.7 ± 0.3 and 37.5 ± 0.4; and full DiagEvo gives 72.3 ± 0.3 and 38.8 ± 0.4.

We also directly audited the pseudo-labels retained for training across the same five rounds by sampling 200 questions before filtering and 200 questions from the retained pool. Verified label accuracy increases from 115/200 (57.5%) before filtering to 149/200 (74.5%) in the retained pool (+17.0 percentage points), while 25.5% of retained labels remain wrong.

In the revised manuscript: Section 2.3 clarifies Mastered and the use of filtered candidates. Section 3.3 (Table 2) gives the state-update ablations. Section 3.4 summarizes reliability; Appendix H.1 gives the label audit (Table 21), promotion and single-question checks (Table 22), and H.2 gives the follow-up outcomes (Table 23).

I have read the response and additional experiments. I will maintain my original rating.

AuthorsOpenReview ↗

Thank you for your feedback

Dear Reviewer cg2Z,

Thank you for taking the time to read our responses and additional experiments, and for getting back to us.

Your questions about whether diagnosed error causes reflect real reasoning errors and how the Mastered state is supported were particularly helpful. They prompted us to check diagnosis quality directly and examine memory-state transitions more closely, which strengthened the evidence in the revised manuscript.

We appreciate the care you put into the review and the concrete suggestions that helped us improve the paper.

Reviewer_kqtQOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DiagEvo, a challenger–solver self-play framework for mathematical reasoning that adds an error-cause memory on top of the R-Zero loop. After each round, a frozen external LLM diagnostician (Qwen3-4B-Instruct-2507 by default) compares failed solver trajectories against a stored reference trajectory that agrees with the filtered pseudo-label, extracts a 10–20-word description of the reasoning failure, and routes it into a two-level memory of skill nodes (category) and error causes. Causes are tracked as Active or Mastered based on solver self-consistency on questions that target them. The challenger mixes free exploration with cause-targeted generation, with the mixing ratio set by the volume of current active failures, and can stitch an Active cause with a Mastered sibling into one composite question. A double-confidence filter retains questions only when majority self-consistency lies in [0.25, 0.75] and the top answer leads the runner-up by a factor of 1.6. On Qwen3-4B-Base, Qwen3-8B-Base and OctoThinker-8B-Hybrid-Base, the method reports the best overall mean across nine benchmarks against Base, R-Zero, Absolute Zero, SPICE, R-Few and DARC; on Qwen3-8B it reaches 72.3% mathematical average (4.5 points above the authors' R-Zero reproduction, 1.2 above DARC). Ablations on Qwen3-8B remove each component, and a 7-round analysis shows the memory-guided variant peaks later and higher than unguided self-play, with stable question length and higher pseudo-label accuracy on a shared question set.

Critical strengths
  1. The motivation holds. Difficulty alone does not say what the solver gets wrong; conditioning the challenger on diagnosed failures addresses that within self-play that does not require external task data.
  2. The diagnosis task is well posed: compare a failed trajectory with an agreeing one from the same question and name the earliest divergence. That is easier for a small model than grading a solution.
  3. The empirical scope is broad enough to establish a claim: three solver base models from two model families (Qwen3-4B-Base and Qwen3-8B-Base; OctoThinker-8B-Hybrid-Base), three different LLM diagnosticians, nine benchmarks, eight component ablations, fixed-ratio and hyperparameter sweeps (Tables 8–10), 7-round co-evolution dynamics, and a pseudo-label accuracy study against human-verified oracle answers. Table 1 reports 3-seed means with per-benchmark standard deviations for the proposed method.
  4. The paper is candid about limitations that matter: Mastered is defined by self-consistency rather than correctness , pseudo-label accuracy still falls to 75% by round 5, and performance declines after round 5 in the 7-round run.
  5. Appendix I gives the full prompt templates for the challenger, solver and all four diagnostician calls, plus the retrieval thresholds and routing between them.
  6. The added machinery is cheap in time: diagnosis and memory maintenance are 5.7% of wall-clock (Table 6), with the rest being sampling and GRPO that R-Zero also pays.
Critical weaknesses
  1. Compute matching in the main run is not quite clear:) The solver trains for "up to 12" steps per round, but the number of candidates generated, the number retained after filtering, and the steps actually run are never reported. So training samples per round depend on the filter, and the filtering ablations, the τ sweep, and the in-house R-Zero reproduction are not known to be matched. The R-Zero row's own configuration (pool, filter, steps, iterations) is not stated.
  2. The R-Zero comparison. That row is the authors' own reproduction, but its pool size, filter, steps per iteration and number of iterations are not stated, so it is unclear whether it saw the same number of training questions and updates as DiagEvo. The reproduction is checked only against DARC's reproduction (67.6 vs 67.8), not against the R-Zero paper, which reports 69.5 on the same five benchmarks and the same checkpoint, a +6.7 gain over base versus +4.4 here.
  3. Table 2 is single-seed and several effects are within noise. Table 1's per-benchmark stds bound the 3-seed Math Avg std at 0.2–0.4 and General Avg at 0.4–0.8, so two single runs differ by ~0.6–1.2 (math) and ~1.2–2.2 (general) at 2σ. Random-pair stitching (−1.0), the 0.7/0.4 gain attributed to state updates alone, the 1.1/1.2 margin over fixed ratios (Table 8), and nearly every General Avg delta in Table 2 fall inside that. The large effects (frozen challenger −3.8, single-mode generation −2.2/−2.8, no filtering −2.9) do not.
  4. Pseudo-label quality can be further examined. About a quarter of retained training questions carry a wrong pseudo-label by round 5 (Figure 3c), so the diagnostician is often given a wrong trajectory as the reference and a possibly correct one as the failure, with Prompt 1 telling it not to question the reference; whether the causes extracted in these cases are valid is never checked. And the 75% vs 72% vs 65% comparison itself is on 200 "challenger-generated" questions without saying which run's challenger generated them; if it was the DiagEvo run, its solver is in-distribution and the gap partly measures that rather than label quality.
  5. Reactivation (Mastered → Active) is defined but never shown to happen. Figure 3b gives only net Active/Mastered counts, which cannot separate new causes, promotions and reactivations.
  6. The skill-node hierarchy is not ablated. It is used only to pick the stitching partner, to merge nodes at round end, and as the "Knowledge area" line in the cause-targeted prompts (Appendix I, Modes 2 and 3). Random-pair stitching (−1.0, within noise) ablates only the first. A variant with no skill-node layer at all (one list of causes, no node assignment or merging, no "Knowledge area" line, stitching partner drawn from all Mastered causes) is what would show whether the hierarchy matters, and no such variant is reported.
Questions for authors
  1. For each in-house run (Base, R-Zero, every Table 2 variant, the τ and k sweeps): how many candidate questions were generated per round, how many survived filtering, and how many solver and challenger steps and rollouts were actually run? Was the 12-step cap ever hit short, and were candidates resampled to fill it?
  2. What training configuration did the in-house R-Zero reproduction use? If it differs from DiagEvo's (rounds, batch, steps, checkpoint selection), can you rerun it matched?
  3. The same Qwen3-8B-Base checkpoint scores 4.6 points differently on OlympiadBench here than in the R-Zero paper, and your R-Zero reproduction gains 2.3 points less over base than R-Zero reports. Have you evaluated under the R-Zero authors' harness, and what decoding parameters, max generation length, prompt format and judge settings did you use?
  4. Can you add seeds to Tables 2 and 8, and standard deviations to the aggregate columns of Table 1?
  5. Which run's challenger generated the 200 shared questions in Figure 3c? And for a sample of extracted error causes, oracle-labelled as in Appendix H: what fraction come from wrong pseudo-labels, and are those causes still valid skill descriptions?
  6. Per round, how many causes were created, promoted and reactivated, and how many stitched questions were generated?
  7. What is the result with no skill-node layer: one flat list of causes, no node assignment or merging, no "Knowledge area" line in the prompts, and the stitching partner drawn from all Mastered causes?
10 comments
AuthorsOpenReview ↗

Response to Reviewer kqtQ (Part 1/5)

Thank you for your comments.

Q1.

For each in-house run (Base, R-Zero, every Table 2 variant, the τ and k sweeps): how many candidate questions were generated per round, how many survived filtering, and how many solver and challenger steps and rollouts were actually run? Was the 12-step cap ever hit short, and were candidates resampled to fill it?

A1. The recorded training quantities confirm that filtering did not shorten solver training in the ablations or parameter sweeps. Full DiagEvo, every Table 2 variant, and the τ and k sweeps complete all 12 solver updates per round without resampling candidates. Each generates 8,000 curriculum candidates; full DiagEvo also completes all 6 challenger updates in Appendix E.

Curriculum generation follows challenger training. Retained questions are used without repetition, and questions left after the 12 solver updates are not used for solver training. The five-round run takes 60 hours on 8 H200 GPUs.

Each variant and sweep setting (original Tables 9 and 10, now Tables 17 and 18) retains at least 3,072 questions, enough for all 12 solver updates. Challenger training executes 6 updates per round whenever enabled, while the frozen challenger control executes 0 challenger updates. Base is evaluated directly without training.

The table gives exact retention counts for the full DiagEvo run. Per-round counts for the other variants and either R-Zero setting are not reported here. The shared update settings establish equal solver training quantities per round for the DiagEvo ablations and sweeps; they do not establish equal retention rates or total compute.

RoundCandidatesAfter absolute filterAfter both filtersUsed for solver updatesRetained but unused
18,0003,9823,3043,072232
28,0003,9843,2233,072151
38,0004,2813,7993,072727
48,0004,5023,9363,072864
58,0004,4543,9583,072886

Per round, solver optimization uses 12 × 256 = 3,072 distinct questions with 8 rollouts each, totaling 24,576 responses. Curriculum construction separately samples 96,000 solver responses (12 for each of the 8,000 candidates). Challenger optimization generates 12,288 outputs (6 × 256 × 8), excluding the solver responses used to score them.

In the revised manuscript: Appendix E gives update settings (Tables 3 and 4), full-method retention counts (Table 5), and response counts in the surrounding text. Section 3.3 and Appendix E state the shared update counts for the ablations and sweeps. R-Zero settings are in Table 6 and A2 here.

AuthorsOpenReview ↗

Response to Reviewer kqtQ (Part 2/5)

Q2.

What training configuration did the in-house R-Zero reproduction use? If it differs from DiagEvo's (rounds, batch, steps, checkpoint selection), can you rerun it matched?

A2. We completed the requested matched R-Zero reruns, and DiagEvo's advantage remains after aligning training quantities and checkpoint selection. Across three runs, DiagEvo leads by 3.0 Overall points. We also reproduced R-Zero three times with its paper training configuration.

The paper configuration uses the hyperparameters in R-Zero, Appendix B; the matched configuration uses DiagEvo's quantities in Appendix E. Both use GRPO with BF16 and FlashAttention 2.

Training parameterR-Zero, paper configurationR-Zero, matched to DiagEvoDiagEvo
InitializationQwen3-8B-BaseQwen3-8B-BaseQwen3-8B-Base
Independent training runs333
Training rounds555
Curriculum candidates per round8,0008,0008,000
Challenger global batch size128256256
Challenger updates per round566
Challenger optimization group size488
Solver global batch size128256256
Solver updates per round151212
Solver optimization group size588
Construction responses per candidate101212
Learning rate, both models1e-61e-61e-6
Weight decay, both models0.010.010.01
KL penalty coefficient, both models0.010.010.01
Training rollout temperature1.01.01.0
Training rollout top_p0.990.990.99

The first reproduction evaluates R-Zero under its published training hyperparameters. The second matches DiagEvo in training rounds, candidate pool size, batch sizes, update steps, and rollout group sizes, while retaining R-Zero's challenger and solver loop without diagnosis or error-cause memory.

Both R-Zero settings retain the original majority-vote share interval of 25% to 75%, inclusive. Candidates are kept when 3 to 7 of the 10 construction responses match the majority answer in the paper setting, or 3 to 9 of the 12 responses in the matched setting.

Optimization group size counts the rollouts per question during GRPO policy updates, whereas construction responses are sampled separately to determine majority-vote pseudo-labels. Both R-Zero settings complete all listed updates in every round. These are training parameters; evaluation decoding settings are specified in A3.

Both R-Zero settings and DiagEvo run for five rounds and use the same checkpoint selection rule (Appendix F, Table 9). The fixed validation set contains 300 MATH training questions at Levels 3 and 4 and 300 MMLU validation questions, disjoint from all nine test benchmarks. Selecting the checkpoint with the highest validation accuracy chooses round 3 in both R-Zero settings and round 5 in DiagEvo, and that single selected checkpoint is evaluated on all test benchmarks. Matching the training quantities in the table directly addresses the configuration comparison requested here, while our separate DARC rerun in Appendix F (Table 10) matches elapsed runtime on the same hardware.

Qwen3-8B settingMath AvgGeneral AvgOverall
R-Zero, official training configuration68.4 ± 0.535.9 ± 0.354.0 ± 0.1
R-Zero, training settings matched to DiagEvo69.0 ± 0.136.2 ± 0.354.4 ± 0.1
DiagEvo72.3 ± 0.338.8 ± 0.457.4 ± 0.3

Each row reports the mean ± sample SD over three runs, with R-Zero summaries recomputed at the available benchmark precision under the shared evaluation settings in A3. The paper-configuration results replace the original single-run R-Zero scores of 67.8 / 36.2 / 53.8. Matching DiagEvo's training quantities raises R-Zero's mean scores; DiagEvo retains higher means on all three aggregates.

In the revised manuscript: Table 1 (§3.2) updates the Qwen3-8B R-Zero row. Appendix E, Table 6, lists both training configurations. Appendix F, “Validation and Checkpoint Selection,” gives the selection rule; Tables 10, 11, and 12 give comparisons and per-run scores.

AuthorsOpenReview ↗

Response to Reviewer kqtQ (Part 3/5)

Q3.

The same Qwen3-8B-Base checkpoint scores 4.6 points differently on OlympiadBench here than in the R-Zero paper, and your R-Zero reproduction gains 2.3 points less over base than R-Zero reports. Have you evaluated under the R-Zero authors' harness, and what decoding parameters, max generation length, prompt format and judge settings did you use?

A3. Our evaluation uses R-Zero's released code and applies identical settings to all models on each benchmark. We directly evaluated Base and all reproduced models using the same test questions, prompts, decoding parameters, and grading rules. Our OlympiadBench Base score is 40.1, compared with 44.7 reported in R-Zero. Here, gains over Base are computed using our own Base evaluation.

Base, both R-Zero settings, DARC, its released checkpoint, and DiagEvo use the same test questions, prompts, answer extraction, grading, and decoding parameters:

BenchmarksTemperaturetop_ptop_k truncationResponses per questionMaximum output tokensMetric
AMC, AIME 2024, AIME 2025, HMMT 20250.61Disabled324,096mean@32
Minerva Math, MATH-500, GSM8K, OlympiadBench01Disabled14,096Accuracy
MMLU-Pro, SuperGPQA, BBEH01Disabled18,192Accuracy
GPQA-Diamond01Disabled120,000Accuracy

These settings are fixed across all compared models within every benchmark. For example, every model uses a 20,000-token limit on GPQA-Diamond, and every model uses a 4,096-token limit with 32 sampled responses on AMC. Different benchmarks follow their respective evaluation scripts, and no model receives a different decoding or length allowance within the same comparison.

The four single-response mathematical benchmarks follow R-Zero's generation script. MMLU-Pro, SuperGPQA, and BBEH follow R-Zero's dedicated scripts with an 8,192-token output limit. GPQA-Diamond follows DARC's evaluator, including its 20,000-token limit, for every model.

The R-Zero paper reports mean@32 for AMC and AIME without stating the sampling temperature, while its released generation script sets T=0. We use T=0.6 for AMC and the added AIME/HMMT evaluations so that mean@32 averages 32 independently sampled responses per question, since repeating greedy decoding does not produce 32 random samples. Each response is graded individually and averaged across questions and samples; this is mean accuracy, not pass@32 or majority-vote accuracy. Every compared model uses this same sampling protocol.

We preserve the harness prompt format. Mathematical questions use the system instruction “Please reason step by step, and put your final answer within \boxed{}.” followed by the problem in the user message. General reasoning prompts include the question and applicable answer options and request a boxed final answer. We apply the tokenizer chat template when present and the harness plain-text format otherwise.

Mathematical grading runs the harness answer extraction and equivalence check first, followed by GPT-4o rechecking at judge temperature 0.1 for responses not accepted by the symbolic check. General reasoning uses each benchmark's answer extraction and exact matching, including BBEH normalization rules.

For the trained models, we compare our three-run means against Tables 1 and 2 of the R-Zero paper on the shared benchmarks:

MeasureR-Zero paperOur paper configurationOur matched configuration
Math Avg, common 5 benchmarks69.568.469.0
Math Avg gain over each study's Base6.75.05.5
General Avg, common 3 benchmarks34.534.434.7
OlympiadBench48.946.146.4

The common General Avg excludes GPQA-Diamond, which the R-Zero paper does not report, and the Math Avg gains are computed relative to each study's own Base scores.

These results document the remaining differences on OlympiadBench and in relative Math Avg gain without identifying their exact cause, and the AMC sampling temperature does not explain the greedy OlympiadBench difference.

In the revised manuscript: Appendix F, “Benchmark Evaluation,” states the shared evaluation conditions and gives decoding and scoring settings (Table 9 and its accompanying text). Tables 14 and 15 compare the original reports and our results. Table 1 (§3.2) distinguishes evaluated and quoted scores.

Reviewer_sheLOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces DiagEvo, a self-evolution framework that uses the solver’s own failures to decide what to practice next. The core mechanism is an error-cause memory: failed trajectories are diagnosed into reusable causes, grouped under skill nodes, and tracked with Active/Mastered states. The challenger then mixes free exploration with cause-targeted generation based on recurrence frequencies. On the solver side, training data is selected by a double-confidence filter that combines absolute self-consistency with a relative vote-margin constraint. Empirically, the paper evaluates on 3 solvers and 9 benchmarks, and reports improvements over reproduced R-Zero and competitive reported baselines. The ablation section argues that mixed generation, memory-state updates/cross-state stitching, and relative-confidence filtering each contribute.

Critical strengths

The strongest aspect is coherence: diagnosis, memory maintenance, generation scheduling, and filtering are not presented as independent tricks but as one closed loop. The component ablations are also informative; they support that failure-aware targeting adds value beyond generic difficulty control. I also appreciate the empirical scope. The paper includes multiple solver families, nine benchmarks, three-run reporting for DiagEvo, and a useful runtime breakdown. This is a stronger package than average for this line of work.

Critical weaknesses

My main reservation is with the strongest comparative claim. Several key baseline rows are inherited from prior reports rather than rerun end-to-end in the same stack. Since margins to the top baseline are not large in some settings, this weakens confidence in hard “best overall” language. A second issue is epistemic: supervision is still agreement-based rather than correctness-based. The paper acknowledges this, but does not yet quantify high-consensus wrong labels deeply enough to show the memory is robust against this failure mode. Finally, the practical story around the external diagnostician is incomplete. The paper gives overhead percentages and larger-diagnostician trends, but I still miss a cleaner budget-performance frontier showing when the extra machinery is or is not worth it.

Questions for authors

Could you provide a fully matched rerun of at least DARC on Qwen3-8B (same training budget, checkpoint selection, and evaluation harness)? This would materially strengthen the main comparison. What fraction of retained pseudo-labels are later judged incorrect, and how often do these errors affect memory-state transitions (e.g., false Mastered or unnecessary reactivation)? Please add a diagnostician-budget study (smaller diagnostician, fewer diagnosis calls, or less frequent memory updates) to clarify practical deployment trade-offs.

4 comments
AuthorsOpenReview ↗

Response to Reviewer sheL (Part 1/2)

Thank you for your comments.

Q1.

Could you provide a fully matched rerun of at least DARC on Qwen3-8B (same training budget, checkpoint selection, and evaluation harness)? This would materially strengthen the main comparison.

A1. We completed the requested DARC reruns and found that DiagEvo retains its advantage under matched conditions. Across three Qwen3-8B runs with the same training budget, checkpoint selection rule, and evaluation settings, DiagEvo leads by 2.8 Overall points.

The budget is 60 hours on 8 H200 GPUs (Appendix E), about 53% of the runtime of the original full DARC configuration.

In each 60-hour run on the same eight H200 GPUs, DARC completes 1,808 challenger updates. The recorded curriculum starts with 60,000 candidates. The format check removes 3,498 questions. The length check then removes 504 more, leaving 55,998 for solver training. The solver completes one pass with 109 updates. We save solver checkpoints every 15 updates and evaluate the final checkpoint as well. Because DARC runs one challenger training stage followed by one solver training stage, we report its executed update steps rather than imposing DiagEvo's alternating rounds.

Checkpoint selection uses the same fixed validation set across all methods: 300 MATH training questions at Levels 3 and 4 plus 300 MMLU validation questions, disjoint from all test sets. We select the single checkpoint with the highest validation accuracy and evaluate it on all test benchmarks. This rule selects step 109 for DARC, round 3 for both R-Zero settings, and round 5 for DiagEvo.

Qwen3-8B methodMath AvgGeneral AvgOverall
DARC, rerun with the same time budget69.0 ± 0.336.7 ± 0.654.6 ± 0.4
DARC, released checkpoint evaluated with our harness71.537.556.4
DiagEvo72.3 ± 0.338.8 ± 0.457.4 ± 0.3

The DARC rerun and DiagEvo rows report means ± sample SD over three runs, with DARC aggregates recomputed from the per-benchmark scores. The second row evaluates the single released DARC checkpoint in the same harness. DARC's reported scores in the original Table 1 are 71.1 / 37.8 / 56.3; these are separate from both direct evaluations above. Math Avg, General Avg, and Overall cover the original 5, 4, and 9 benchmarks.

For every benchmark, Base, both R-Zero settings, the DARC rerun, the released DARC checkpoint, and DiagEvo use the exact same test questions, prompts, answer extraction, grading, and decoding parameters:

BenchmarksTemperaturetop_ptop_k truncationResponses per questionMaximum output tokensMetric
AMC, AIME 2024, AIME 2025, HMMT 20250.61Disabled324,096mean@32
Minerva Math, MATH-500, GSM8K, OlympiadBench01Disabled14,096Accuracy
MMLU-Pro, SuperGPQA, BBEH01Disabled18,192Accuracy
GPQA-Diamond01Disabled120,000Accuracy

Token limits follow the corresponding R-Zero or DARC evaluation scripts across benchmarks and remain identical across all models within each benchmark. AMC and AIME/HMMT use random sampling to report mean accuracy over 32 responses per question, while the other benchmarks use one greedy response. Mathematical scoring applies answer equivalence checking followed by GPT-4o rechecking at judge temperature 0.1. General reasoning benchmarks apply identical answer extraction and matching rules to every model.

In the revised manuscript: Appendix E, “DARC Reproduction,” gives the budget, filtering counts, and update steps. Appendix F gives checkpoint selection, evaluation settings (Table 9), and direct comparisons (Table 10). Section 3.2 summarizes these results; Table 1 identifies quoted scores and updates the Qwen3-8B R-Zero row.

AuthorsOpenReview ↗

Response to Reviewer sheL (Part 2/2)

Q2.

What fraction of retained pseudo-labels are later judged incorrect, and how often do these errors affect memory-state transitions (e.g., false Mastered or unnecessary reactivation)?

A2. Our manual audit found that most promotions met the correctness threshold despite noisy labels. Follow-up showed that incorrect promotions can be reactivated. We audited all 161 promotions and 34 reactivations across the first five rounds of the Figure 3 run:

State transition checkCountRate
Promotions below 70% when scored by verified correctness9/1615.6%
Unnecessary reactivations attributed to incorrect pseudo-labels3/348.8%

For promotion, we rescored the original solver responses on all supporting candidates against verified answers, averaged per-question accuracy, and compared it with 70%. The nine failures lack correctness support; we do not attribute all nine to wrong pseudo-labels. For reactivation, checking the triggering reasoning and reference label identified three unnecessary events caused by wrong pseudo-labels.

We also tracked a separate random sample of 100 early first promotions across three runs to round 5. Three were initially unverifiable and remain uncertain. Of the other 97, nine were below 70% correctness; seven were reactivated, including same-round reversals. At round 5, four of the nine reached 70% on their original promotion questions; three were Active below 70%; one remained Mastered below 70%; and one had an uncertain outcome. Reactivation and improved correctness are distinct observations. Some errors persist, and this trace alone does not establish that reactivation caused improvement.

To measure label quality, we separately sampled 200 questions from each of the pre-filter and retained pools across the same five rounds:

Question poolCorrect pseudo-labelsAccuracy
Before filtering115/20057.5%
Retained after filtering149/20074.5%

Label accuracy is 17.0 points higher after filtering in these samples; 51/200 retained labels (25.5%) remain wrong. Reference answers use 16 dpsk-v4.1-flash-10.17 responses per question, followed by manual checking of the modal answer and solution steps. Majority vote alone is not treated as ground truth. Two master's students in mathematics and statistics checked each sample independently before comparing judgments. They discussed disagreements against the source reasoning and reference solution; a more experienced researcher reviewed unresolved cases. All three were team members. Cases still unresolved remained uncertain. This verification procedure also supports the state checks above.

In §2.3, “Mastered is a curriculum state based on high self-consistency, not verified correctness.” It stops direct targeting while keeping the cause available for cross-state stitching and reactivation. Self-generated questions carry no supplied ground truth. Checking every candidate with a stronger model during training would add substantial inference cost, and repeated samples can still agree on a wrong answer. We use the solver's existing self-consistency to decide when to reduce direct practice, and check true correctness separately here.

The three-run ablation tests whether state updates remain useful under these noisy training signals. State updates without stitching give 70.7 / 37.5 Math/General Avg, versus 70.0 / 37.1 without state updates (Table 2). This supports their training value alongside the direct checks of state reliability above.

In the revised manuscript: Section 2.3 defines Mastered. Section 3.4 summarizes the audits; Appendix H.1 gives the protocol and label and state checks (Tables 21 and 22), and H.2 tracks early promotions (Table 23).

Q3.

Please add a diagnostician-budget study (smaller diagnostician, fewer diagnosis calls, or less frequent memory updates) to clarify practical deployment trade-offs.

A3. The requested budget study shows that diagnosis calls can be reduced while retaining most of DiagEvo's performance. Across three runs, diagnosing half the eligible failed trajectories reduces Math Avg by 0.5 points and General Avg by 0.4 points.

We uniformly sampled 50% or 25% of eligible failed trajectories per round. Memory still updates every round; other training settings stay fixed.

Eligible trajectories diagnosedMath AvgGeneral Avg
100%72.3 ± 0.338.8 ± 0.4
50%71.8 ± 0.438.4 ± 0.3
25%71.0 ± 0.637.8 ± 0.4

Results are means ± sample SD over three runs. The percentages refer to the fraction of eligible failed trajectories diagnosed per round, not total system calls or runtime. Full diagnosis and memory maintenance take 5.7% of total runtime (Appendix E, Table 8).

In the revised manuscript: Section 3.2 and Appendix G, “Diagnosis Budget” (Table 20), report the sampling procedure and results.

AuthorsOpenReview ↗

Follow-up to Reviewer sheL

Dear Reviewer sheL,

Thank you for the time and care you put into reviewing our work. Your suggestions helped us strengthen the comparisons and examine the method's reliability and cost. We summarize the additions below and would be grateful for your thoughts.

  1. Matched DARC rerun on Qwen3-8B (Part 1/2, A1): We completed three DARC reruns with the same training budget (60 hours on 8 H200 GPUs), validation checkpoint selection rule, and evaluation settings. DiagEvo retains a 2.8-point Overall advantage (57.4 ± 0.3 vs. 54.6 ± 0.4; Appendices E and F, Tables 9 and 10). The added training details and matched R-Zero comparison also helped clarify baseline fairness in our discussion with Reviewer kqtQ.

  2. Pseudo-label errors and memory-state transitions (Part 2/2, A2): Our audit of the first five rounds found that most promotions met the 70% verified correctness threshold despite label noise. Only 9/161 fell below it, not all due to wrong pseudo-labels; 3/34 reactivations were unnecessary due to wrong labels. In a separate sample across three runs, 7 of 9 early promotions below the threshold were reactivated (Appendices H.1 and H.2, Tables 21 to 23).

  3. Diagnostician budget trade-off (Part 2/2, A3): Diagnosing half of the eligible failed trajectories per round retains most of DiagEvo's gains. Across three runs, Math and General Avg decrease by only 0.5 and 0.4 points (Appendix G, Table 20).

Thank you again for your time and consideration.

AuthorsOpenReview ↗

Follow-up to Reviewer sheL

Dear Reviewer sheL,

Thank you again for your helpful review. Your questions about baseline fairness, memory states, and diagnosis cost helped us strengthen the experimental evidence and explain our choices more clearly. We would appreciate your feedback on whether the matched DARC reruns, memory-state checks, and diagnosis budget results address your concerns (Parts 1/2 and 2/2). If any concerns remain, we would be happy to discuss them further.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.