- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
DiagEvo extends R-Zero-style challenger–solver self-play with an error-cause memory. A frozen external LLM (the diagnostician) compares failed and pseudo-label-agreeing trajectories, extracts transferable error causes, and organizes them under skill nodes with Active/Mastered states. These states and recurrence counts set the mix between free exploration and cause-targeted question generation. A double-confidence filter keeps only questions whose leading answer clearly beats the runner-up. On Qwen3-4B/8B-Base and OctoThinker-8B, the method reports gains of about 1.1–1.3 points in overall average over DARC.
Critical strengths
The method is clearly described and the experiments are carefully executed. The paper uses three solvers and three runs each, ablates every component, and validates pseudo-label accuracy with human verification. This makes the reported results credible, but it does not compensate for the limited contribution.
Critical weaknesses
-
Limited novelty and incomplete related work. The method is an incremental extension of R-Zero, and each of its components has close precedents: contrasting successful and failed trajectories to extract reusable insights comes from ExpeL, AutoGuide, and ReasoningBank; targeting capability gaps identified from failures is the idea of TRACE, which receives only one sentence of discussion; double-confidence filtering is a minor variant of existing consistency- or entropy-based filtering (SRT, EMPO). Highly relevant prior work is also missing. EvoCurr (Cheng et al., 2025) uses a curriculum LLM that adaptively generates intermediate tasks based on solver performance and failures, which directly overlaps with the paper’s central claim. Learned curriculum methods for RL reasoning, such as Self-Evolving Curriculum (Chen et al.), are likewise not discussed. The paper should clarify what is novel beyond applying feedback-driven curriculum generation to label-free self-play.
-
Insufficient and largely saturated benchmarks. Several benchmarks offer little headroom: GSM8K is above 91% for the Qwen3-8B base model before training. Others are too small or noisy to separate methods: AMC has 40 problems, GPQA-Diamond shows standard deviations up to 1.3, and all methods score about 11–12 on BBEH. As a result, the 1.1–1.3 point margin over DARC, whose scores are copied single-run results, is hard to interpret. Training and curriculum construction are limited to math. There are no harder or more discriminative benchmarks (AIME 24/25/26, HMMT, LiveCodeBench), and no agentic or multi-turn settings, where an error-cause memory should matter most.
Questions for authors
-
Please discuss EvoCurr and other feedback-driven curriculum methods, and clarify the novelty of DiagEvo relative to them and to TRACE.
-
Can you report results on harder, less saturated benchmarks (e.g., AIME 24/25/26, HMMT, LiveCodeBench) and on agentic or multi-turn tasks?
-
The diagnostician is an external post-trained model, and gains grow with its scale. Can you add controls for: R-Zero with the same diagnostician but without the memory; the solver’s own base model as the diagnostician; direct distillation from the diagnostician?
-
This would show whether the memory mechanism, rather than access to a stronger model, drives the gains.
5 comments
Response to Reviewer b19V (Part 2/3)
A1 (continued).
TRACE also trains on diagnosed weaknesses. It identifies capability gaps from collected trajectories, builds verifiable training environments, and trains capability adapters and a router to combine them. It does not train the environment generator with RL. DiagEvo addresses how practice priorities should keep changing when both the challenger and solver are updated each round, including when to reduce direct targeting and when to revisit a recurring weakness.
EvoCurr evolves its curriculum and solver policy together. Its designer uses solver performance and failures to propose intermediate tasks toward a supplied final task. The designer LLM remains fixed. The solver generates policy code or trains an RL policy, as in Overcooked. Executability and score thresholds determine which tasks are accepted, and earlier progress is retained when a new task fails. DiagEvo has no supplied final task or externally verified path of intermediate tasks. It follows recurring error causes from failures on self-generated questions.
Self-Evolving Curriculum (SEC) updates its curriculum policy alongside RL training of the solver LLM. It treats problem categories as bandit arms and updates their values by expected learning gain. The policy selects existing training problems; it is not a challenger LLM that learns to generate new questions. DiagEvo extracts specific error causes from current failures to guide new question generation.
ExpeL, AutoGuide, and ReasoningBank use insights, guidelines, or reasoning memories to help solve incoming tasks. ReasoningBank also grows its memory without ground-truth labels. DiagEvo uses failure history to schedule training questions while both the challenger and solver evolve.
A frozen external diagnostician extracts the causes, while the training questions and failure trajectories come from self-play.
The experiments test this motivation directly. Under identical double-confidence filtering, pure free exploration peaks at 69.5 Math Avg in round 3, whereas DiagEvo continues improving to 72.3 in round 5 before declining to 72.0 and 71.4 in rounds 6 and 7 (Figure 3a). Tracking error causes thus extends the effective evolution phase and delays plateau and decline, rather than eliminating late-round degradation entirely. Across three runs (Table 2), removing state updates gives 70.0 ± 0.3 Math Avg and 37.1 ± 0.2 General Avg; keeping state updates without cross-state stitching gives 70.7 ± 0.3 and 37.5 ± 0.4; and full DiagEvo gives 72.3 ± 0.3 and 38.8 ± 0.4. In addition, the control in A3 using the same diagnostician without cross-round memory scores 70.0 and 37.2, testing whether current-round diagnoses alone suffice without maintaining failure history across rounds.
Double-confidence filtering supports this loop by combining an absolute consistency band with a 1.6× vote-margin constraint. Using the absolute constraint alone drops performance to 70.7 ± 0.3 Math Avg and 37.5 ± 0.2 General Avg (Table 2), supporting the relative constraint alongside memory guidance.
In the revised manuscript: Section 1 states the motivation; §2.3 explains the memory design. Section 4 and Appendices A.2 and A.3 compare the cited memory, curriculum, and label-free RL methods.
Response to Reviewer b19V (Part 3/3)
Q2.
Can you report results on harder, less saturated benchmarks (e.g., AIME 24/25/26, HMMT, LiveCodeBench) and on agentic or multi-turn tasks?
A2. The requested AIME and HMMT evaluations show that DiagEvo's gains extend to difficult mathematical benchmarks with low Base accuracy. DiagEvo has the highest mean across three runs on each added benchmark.
All methods use the same 30 problems per benchmark, 32 independent responses per problem, T=0.6, top_p=1, no top_k truncation, a 4,096-token limit, and identical prompts and grading. Scores are mean@32 (%), with mean ± sample SD computed across three runs before rounding. Base and released DARC each have one evaluation. All runs are in Appendix F, Table 13.
| Qwen3-8B method | AIME 2024 | AIME 2025 | HMMT 2025-Feb |
|---|---|---|---|
| Base | 10.4 | 10.4 | 1.6 |
| R-Zero, official training configuration | 13.9 ± 0.3 | 14.0 ± 0.9 | 2.2 ± 0.1 |
| R-Zero, training settings matched to DiagEvo | 14.3 ± 0.3 | 14.7 ± 0.3 | 2.3 ± 0.1 |
| DARC, rerun with the same time budget | 13.6 ± 0.5 | 13.6 ± 0.2 | 2.1 ± 0.3 |
| DARC, released checkpoint | 13.5 | 13.9 | 1.9 |
| DiagEvo | 16.4 ± 0.8 | 15.5 ± 0.3 | 3.6 ± 0.6 |
Base starts at 10.4% on AIME and 1.6% on HMMT, compared with 91.1% on GSM8K. HMMT accuracy remains low, and 32 responses do not increase the 30 distinct problems. AMC's 40 problems, GPQA-Diamond's variation across runs, and low BBEH scores also limit single-benchmark conclusions.
We supplemented DARC's single reported run with three reruns: 60 hours on 8 H200 GPUs, about 53% of full DARC runtime. Each runs 1,808 challenger updates and one solver pass of 109 updates. The shared validation rule selects step 109, evaluated with the same harness as R-Zero and DiagEvo. On the original nine benchmarks, DARC reaches 69.0 ± 0.3 Math Avg, 36.7 ± 0.6 General Avg, and 54.6 ± 0.4 Overall; DiagEvo reaches 72.3 ± 0.3, 38.8 ± 0.4, and 57.4 ± 0.3. DARC's released checkpoint scores 71.5, 37.5, and 56.4 under the same evaluation settings.
We have not evaluated AIME 2026, LiveCodeBench, or agentic and multi-turn tasks. Appendix C states: “DiagEvo builds its curriculum from mathematical questions.” Our study trains on math and evaluates zero-shot transfer to single-turn general reasoning. Code and interactive tasks need execution environments and multi-turn feedback beyond this setup.
In the revised manuscript: Section 3.2 summarizes the added results. Appendix F gives evaluation settings (Table 9), DARC comparisons (Table 10), and complete per-run AIME/HMMT results (Table 13), separate from the original averages. Section 3.1 and Appendix C, “Evaluation scope,” state the task coverage and its limits.
Q3.
The diagnostician is an external post-trained model, and gains grow with its scale. Can you add controls for: R-Zero with the same diagnostician but without the memory; the solver’s own base model as the diagnostician; direct distillation from the diagnostician?
This would show whether the memory mechanism, rather than access to a stronger model, drives the gains.
A3. We ran the three requested controls. Using the same diagnostician without memory or directly distilling its solutions gave lower scores than full DiagEvo. A frozen Base diagnostician retained most of the gain. All controls use the same Qwen3-8B initialization and evaluation harness. Each is one five-round run; full DiagEvo uses three.
| Qwen3-8B setting | Math Avg | General Avg |
|---|---|---|
| R-Zero + the same diagnostician, without memory | 70.0 | 37.2 |
| Solver's own base model as diagnostician | 71.3 | 37.9 |
| Direct distillation from the diagnostician | 67.1 | 33.4 |
| DiagEvo, default diagnostician | 72.3 ± 0.3 | 38.8 ± 0.4 |
Same diagnostician, without memory. Current-round causes guide the next round's generation and are then discarded. There is no accumulated cause history, Active/Mastered states, skill-node layer, or cross-state stitching. Per-round free and cause-targeted question counts, filtering, and update budgets match full DiagEvo.
Solver's own base model as diagnostician. A separate frozen copy of the initial Qwen3-8B-Base checkpoint performs diagnosis and memory maintenance. It does not update with the solver; the rest of DiagEvo stays unchanged.
Direct distillation. The default diagnostician generates full solutions and answers for the questions used to train DiagEvo. The solver learns these through SFT from the same Base, without error-cause memory. Learning rate and checkpoint are selected on the shared validation set under a comparable training budget. This tests teacher supervision on shared questions; the first control tests generation without memory.
In the revised manuscript: Section 3.3 summarizes all three controls. Appendix G, “Additional Controls,” gives each setup, run count, and result (Table 19).
Response to Reviewer b19V (Part 1/3)
Thank you for your comments.
Q1.
A1. DiagEvo uses recurring error causes to keep self-play directed at unresolved weaknesses while both the challenger and solver learn across rounds. We share with TRACE and related methods the broad idea of using feedback to guide learning. DiagEvo's contribution is how it maintains and uses this feedback during self-play: the same error cause connects diagnosis, new question generation, and progress tracking as weaknesses persist, improve, or return.
TRACE, EvoCurr, and Self-Evolving Curriculum (SEC) use feedback for capability training, intermediate task design, or category selection, respectively. None uses alternating RL to train both a challenger LLM that generates new questions and a solver LLM. DiagEvo builds on R-Zero's alternating training and introduces error-cause memory to maintain practice priorities throughout this process.
Our starting point is the problem in §1: “Different questions can expose the same error cause, so difficulty alone does not identify what the solver should practice next.” In self-play, the question distribution and the solver change together. A harder or longer question can leave the same reasoning weakness unresolved, and a weakness that no longer receives direct practice can recur. Difficulty and learnability indicate whether a question is useful for training; they do not track which recurring weaknesses remain.
We therefore retain transferable error causes from failed trajectories across rounds to guide future question generation.
This need to maintain practice priorities explains the memory design. Diagnosis uses current failures at the end of each round. Active and Mastered states determine when to target a cause directly, keep it for cross-state stitching, or reactivate it after a matching failure. Recurrence counts prioritize Active causes and set the mix of cause-targeted generation and free exploration. The challenger samples both its update questions and the solver curriculum under this memory schedule.
The works you cited address related aspects of experience reuse, targeted capability training, adaptive curricula, and label-free RL, and comparing what their feedback or memory controls helps locate our contribution: