acceptodds
Desk rejected at ICLR 2027

What Actually Evolves in Self-Evolving Reinforcement Learning?

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Self-evolving reinforcement learning (RL) aims to train a model that learns from self-generated tasks, without additional human annotations. It typically alternately trains two roles: a proposer that generates training tasks and a solver that learns from them, spontaneously improving the performance on downstream tasks. However, it remains unclear what drives these gains, and whether the proposer can keep generating tasks that sustain the solver’s improvement. In this study, we systematically examine current self-evolving RL across six methods and six models on seven reasoning benchmarks, with RL on human-annotated answers (human-annotated RL) as a reference. We track how pass@1 and pass@128 change, identify the source of the gain, and test whether later proposers write more useful tasks than earlier ones. Our analysis reveals three findings. (1) Most solver gains come from questions the base model already answers correctly. (2) Self-evolving RL corrects far fewer errors than human-annotated RL. (3) Proposers do not teach more after further training: the same solver does no better, and sometimes worse, when trained on tasks from a later proposer than on tasks from an earlier one. Our theoretical analysis of a single solver update shows that it can only strengthen answers the solver already gives. Although recursive self-improvement aims to go beyond human supervision, current self-evolving RL rarely outperforms human-annotated RL. Taken together, our findings suggest that current gains primarily reflect sharpening, and that self-evolving RL has so far only partially achieved sustained recursive self-improvement.

Primary area:
reinforcement learning
Keywords:
self-evolving reinforcement learning, self-improvement, large language models, reasoning, evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_AszSOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates what improves during self-evolving reinforcement learning, separating solver improvement from proposer utility. It evaluates six methods across six base models on seven mathematical reasoning benchmarks. The authors find that most pass@1 gains occur on questions whose initial plurality answer is already correct, while later proposers do not consistently provide more useful training tasks. Generated-task audits identify deteriorating question validity and label quality in some methods. A simplified single-update analysis explains how label sources determine answer reinforcement and how proposers reweight training questions.

Critical strengths

An interesting evaluation perspective. The paper distinguishes improved answer consistency from error correction and separately examines proposer utility. These diagnostics help assess claims of recursive self-improvement beyond aggregate accuracy.

Substantial empirical coverage. The study covers multiple methods and model families, tracks training checkpoints, and supplements rule-based grading with semantic regrading and a smaller reasoning-validity analysis.

Critical weaknesses

Limited novelty and insight. The main observations overlap substantially with prior analyses of sharpening and collapse in unsupervised RLVR [1, 2] and degradation in R-Zero. The broader evaluation is valuable, but the paper does not clearly establish a new mechanism or explain when proposer evolution improves learning.

Insufficient support for the broader conclusions. Plurality-based grouping and net probability gains do not directly establish limited capability acquisition or fewer corrected errors. The proposer results also require reconciliation with R-Zero’s trained-versus-untrained comparison. The simplified theory does not resolve these gaps or establish an early–later proposer ordering.

References

[1] Shumailov et al. AI models collapse when trained on recursively generated data. Nature, 2024.

[2] He et al. How Far Can Unsupervised RLVR Scale LLM Training? ICLR 2026.

Questions for authors
  1. For Finding 1 (Most solver gains come from questions the base model already answers correctly) and Finding 2 (Self-evolving RL corrects far fewer errors than human-annotated RL.). Since an initially incorrect plurality answer does not imply that the model cannot solve the question, could you report gains by initial correct-answer probability and separate question-level improvements from regressions? Could you clarify how the additional analyses support these interpretations, and what they establish beyond the recursive-data degradation studied in AI Models Collapse [1] and the sharpening mechanism analyzed in Unsupervised RLVR [2] ?
  2. For Finding 3 (Proposers do not teach more after further training.). Section 4.1 finds no benefit from updating the proposer, whereas R-Zero reports that a trained Challenger outperforms an untrained one [3, Table 1]. How do you reconcile these results? Does this discrepancy persist under matched training and evaluation settings, and how does it affect the scope of Finding 3?
  3. Theoretical explanation of the findings. What theoretical insight does Section 5 add beyond the sharpening analysis in [2]? Please clarify which findings follow from the analysis, particularly since Proposition 2 does not establish whether later proposers teach more.

References

[1] Shumailov et al. AI models collapse when trained on recursively generated data. Nature, 2024.

[2] He et al. How Far Can Unsupervised RLVR Scale LLM Training? ICLR 2026.

5 comments

We sincerely thank the reviewer for the careful reading and the valuable questions. To address these concerns, we ran the additional analyses that were requested. All numbers use the paper's protocol (2,866 questions, 128 samples each).

What the paper adds. Here we summarize three of our main contributions that are not in previous papers.

  1. Other label sources do not correct more errors. Labels from a proposer or an environment prevent the loss of correct answers. But on questions that the base model gets wrong, every self-evolving label source improves by a median of only 3–4 pp, about half of the 7.0 pp of human-annotated RL.
  2. A per-question comparison of self-evolving and human-annotated RL on the same six models. Both gain mostly on questions that the base model often solves (as Yue et al., 2025, report for human labels), and there they gain similar amounts. On questions that the base model rarely solves, human labels gain 2.0–2.7 times as much.
  3. A later proposer does not teach more. In five methods, with three paired seeds each, tasks from a later proposer teach a fixed solver no more than tasks from an early one.

Reply to Q1

We agree: a wrong plurality answer does not show that the model cannot solve the question. To better understand what actually happens, we regrouped all 42 settings by p0, the fraction of the 128 base-model responses that are correct. The table gives medians over the 36 self-evolving (SE) and the 6 human-annotated RL (HA) settings. "Up / down" counts the questions whose correct-response rate changes by at least 10 pp.

p0QuestionsChange, SE (pp)Change, HA (pp)Gain share, SEGain share, HAUp / down, SEUp / down, HA
0374.5+0.43+0.870.3%0.4%1.5 / 08.5 / 0
(0, 0.05]299.5+2.80+7.601.6%2.9%30 / 087.5 / 0
(0.05, 0.2]268.5+14.89+30.906.5%9.6%141 / 1194.5 / 4
(0.2, 0.5]1035.5+35.22+48.1160.8%52.8%754 / 5987 / 7
(0.5, 0.8]932+30.60+35.7029.6%35.0%732 / 1918 / 2.5
(0.8, 1]29.5+10.95+15.540.4%0.6%11.5 / 028 / 0
  1. Finding 1 stays the same without the plurality split. Questions with p0 ≤ 0.05 are about 24% of the questions and give a median of 1.8% of the self-evolving gain. Questions with p0 > 0.2 are about 67% of the questions and give 91.2%. Per setting, the share from p0 ≤ 0.2 is 4.6–29.5% (63.6% for SPICE on OctoThinker).
  2. The net gains do not hide regressions. In the median self-evolving setting, 8.5 questions drop by 10 pp or more and 1,962.5 rise by 10 pp or more (4 of the 36 settings have more than 100 drops; the most is 413, for SPICE on OctoThinker). At the question level, a self-evolving setting newly solves a median of 73 questions (0 of 128 correct before, ≥1 after) and loses 61.5. The median net is +3.5 questions (pass@128 +0.12 pp), next to a pass@1 gain of +23.4 pp. Human-annotated RL newly solves 97 and loses 56.5 (net +23). This shows that training makes correct answers more frequent but hardly enlarges the set of solvable questions.
  3. Finding 2 holds with the new analysis. In the three bins with p0 ≤ 0.2, human-annotated RL raises the correct-response rate 2.0–2.7 times as much as self-evolving RL. Above 0.2, the ratio is 1.2–1.4.

We hope that these results address the concern about the plurality-based grouping and the net gains.

Our comparison with [1], [2] and other related work:

WorkGenerated tasksLabel sourcesPer-question analysisHuman-label RL referenceProposer studied
Shumailov et al. [1]✗None (no RL)✗✗✗
He et al. [2]✗Intrinsic; self-verification (preliminary)25 problems, 1 modelCurves on 1 model; AIME24 gain on 7✗
Yue et al. (2025)✗Human labels onlyAccuracy histograms; solved-set comparison✓ (its only setting)✗
Ours✓ 6 methods5: self-voting, proposer-stated, documents, environment, human✓ 2,866 questions, 42 settings✓ 6 models, same analysis✓ 5 methods on a fixed solver

[1] studies likelihood retraining without RL. Our result is of a different kind: pass@1 rises in all 42 settings, and we ask what the rise consists of.

[2] is the closest work. Its main experiments use intrinsic rewards on fixed question sets without a proposer, so it does not test results (2) and (3). For external rewards, [2] gives preliminary in-domain evidence (self-verification on Countdown) that they "may escape the confidence-correctness ceiling". Result (1) is complementary: external verification on code or games (AZR, SPIRAL) and document answers (SPICE) give no extra error correction on math.

We will add the p0 table, its figure and this comparison with prior work to the revised manuscript.

Reply to Q2

We thank the reviewer for this question. It led us to a direct, matched test.

R-Zero's Table 1 (arXiv v1) gives "Iter 1 minus Base Challenger" = +1.52 (Qwen3-8B) and +3.70 (Qwen3-4B) on a seven-benchmark average. Each row is one training run; the paper reports no seeds or intervals, and it does not give the training budget of the Base Challenger row. We tested whether the gap persists under matched settings. Our full-loop and frozen-proposer runs use the same code, solver budget and data seed; only proposer training differs. We evaluated them with R-Zero's released evaluation code and with our protocol.

EvaluationModelRunsRound 1, trained minus untrained proposer (pp)
R-Zero Table 1 (v1)8B / 4B1 per row+1.52 / +3.70
R-Zero's released evaluation code, our runs8B3 paired seeds+0.91, −0.60, +0.41 (mean +0.24)
same4B1 pair−1.50
Our protocol, 128 samples, 2,866 questions8B3 paired seeds+0.06, −0.45, −0.28 (mean −0.22; t interval −0.87 to +0.43)
Our protocol, 128 samples, 1,847 questions4B1 pair−0.08
  • Under matched training and evaluation, the gap does not persist. With R-Zero's evaluation code, every paired 95% interval includes zero, and round 2 gives the same picture (8B mean +0.07; 4B −0.80). With 128 samples, the 8B interval excludes +1.52.
  • The published 8B gap is within the noise of that metric. In the seven-benchmark average, AMC and the two AIME sets hold 100 questions but 3/7 of the weight. The question-sampling standard error of a difference of two such averages is about 2.5 points, so +1.52 is 0.6 standard errors (+3.70 for 4B is 1.6). Between arXiv v1 and v4, a new measurement of the AIME columns alone moved R-Zero's final 8B average by −0.97.
  • R-Zero's own table agrees in part. The untrained Challenger already gives 64% (8B) and 32% (4B) of the first-iteration gain over the base model.
  • Limits. Our proposer gets the update budget stated in R-Zero's paper (20 optimizer updates per round); the released script uses 192, and we did not test it. Our 4B pair has one seed. Our run of R-Zero's evaluation code omits the GPT-4o re-judging step, so we compare only within one harness.

Scope of Finding 3. Finding 3 rests mainly on the fixed-solver comparison of early and later proposers in five methods (Section 4.2). R-Zero's Table 1 does not test this. We also repeated that comparison at the last stage of the R-Zero run: the round-9 solver, trained on tasks from the round-1 proposer or from the round-10 proposer (which was trained against that solver), with three paired seeds. Later minus early is −0.66, −1.15 and −0.13 pp (mean −0.64), against a mean of −0.67 at round 5 in the paper.

Dear Authors,

Thank you for the detailed rebuttal and the additional experiments. Most of my concerns have been addressed, and I will raise my rating to 2.

Best regards,

Reviewer AszS

Reviewer_z4YfOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

The authors investigate whether self-evolving reinforcement learning helps language models develop new reasoning abilities or mainly strengthens existing ones. Across numerous methods and math reasoning benchmarks, the authors find that most performance gains come from making the model more consistent on questions it can already answer. They find that later proposers do not consistently generate more useful training tasks than earlier ones.

Critical strengths

This is a well motivated paper!

First, the authors avoid only looking at the accuracy and study why does the model improve by examining whether training helps models solve previously unsolved questions or just answer already familiar questions more often. They find that a median of 95.9% of the pass@1 gain comes from questions the base model already can answer.

Second, the author evaluate 6 self-evolving RL methods on 6 models and 7 math benchmarks. This shows their finding is convincing and the observed patterns hold across different settings.

Critical weaknesses

Overall, this is an interesting paper which investigates an important problem on what model really learns in the self-evolving process. However, the experiment focus mainly on math reasoning, so it is unclear whether the findings generalize to other tasks like coding or long horizon tool use. In addition, the theoretical analysis considers only a single solver update, which limits its ability to explain the behavior of the full iterative self-evolving process.

Questions for authors

Overall, this is a well written paper. However, I still have 2 minor concerns.

First, the authors claim later proposers do not always generate useful training tasks than earlier ones. I am curious is it due to lack of task diversity or the quality of generated tasks? Also, maybe the later proposers become more beneficial when paired with solvers at different training stages. So, I would like to see how does the author interpret the proposer analysis.

Second, I agree with the finding that most performance gains come from questions the model could already answer correcrtly. However, why does self-evolving RL struggle to correct initially incorrect answers, even when the model has some probability of generating the correct answer? Is the main limitation come from the quality of self-generated labels or the difficulty of reinforcing low probability correct answers?

1 comment

We thank the reviewer for the positive feedback and the questions. We will add the answers below to the revised paper.

Q1: How do we interpret the proposer analysis?

Diversity or quality? In R-Zero both fall. Invalid questions rise from 36% in round 1 to 61% in round 10, correct labels fall from 76% to 23%, and the ten most common labels cover 58% of round-10 questions, up from 36%. But neither one explains why later tasks teach no more. Vocabulary Dropout keeps its tasks diverse, and AZR's tasks get better over training (all but three of 1,500 stored outputs are correct, and trivial tasks fall from 18% to none). In both methods, later tasks still teach a fixed solver no more than early ones. In round one, even a question set with 56% invalid questions gives the same gain as the original set (+22.1 against +22.4 pp).

Solvers at other training stages? Our fixed solver is not the base model. For R-Zero it is the solver after round 5 of 10 (64.3% pass@1, against 42.2% for the base model), and the later proposer is the one trained against exactly this solver. To test the reviewer's suggestion directly, we repeated the comparison with the round-9 solver and the round-10 proposer, with the same budgets, seeds and evaluation. Later tasks again teach no more: later minus early pass@1 is −0.66, −1.15 and −0.13 pp over three seeds (mean −0.64), against a mean of −0.67 at round 5.

Our interpretation. The limit is in what the solver can learn from a task, not in the tasks alone. The gain on each question is set by the solver's own answer distribution and by the label. A proposer can only change which questions are used, so a later proposer does not teach more, whether its tasks get worse, as in R-Zero, or better, as in AZR (Section 5).

Q2: Why are initially wrong answers hard to correct?

Both causes that the reviewer lists occur.

  • Low probability. The base models answer these questions correctly in only 2–4% of samples. The reward arrives only when the solver samples the correct answer, so the gain is proportional to p(1−p)p(1-p) and is small here. In a new analysis by initial correct rate, questions with a rate of at most 5% rise by 2.8 pp under self-evolving RL, and questions with a rate of 20–50% rise by 35.2 pp (7.6 and 48.1 pp for human-annotated RL). The correct rate does rise on initially wrong questions in all 42 settings, but usually not enough to change the most frequent answer.
  • Label quality. With correct labels the same questions gain more than twice as much: 7.0 pp for human-annotated RL against 3.2 pp for self-evolving RL. A majority-vote label is the solver's own most frequent answer, so on these questions it is wrong, and training reinforces it: R-Zero raises the wrong leading answer by 16 pp, about twice as much as any other method (appendix table on initial-answer reinforcement).
  • External checks. AZR and SPIRAL have correct labels, but for code and game tasks. They never give a correct label on a mathematics question that the solver gets wrong, and they gain 2–4 pp on these questions, like the other self-evolving methods.

W1: Scope.

We agree that results beyond mathematics would make the conclusions more general. The limit is compute: some training runs take more than a week on eight GPUs. For the rebuttal we evaluated our AZR checkpoints in AZR's own domain, with the AZR paper's code protocol and official scripts. AZR does not improve HumanEval+ or MBPP+ on either model (−3.0 to −0.3 pp). Its in-domain gain is on LiveCodeBench: +4.4 to +4.7 pp on Qwen2.5-7B, and +2.4 pp on Qwen3-8B after we correct an answer-format failure (−18.9 pp with the official extractor). For comparison, the same checkpoints gain +13.9 and +8.5 pp on mathematics. We will add these results to the paper, and we are extending the evaluation to search tasks.

W2: Single-update theory.

This is a very valuable point. The model describes one solver update. It explains the direction and size of the change in each round, and how the label source sets them. It does not describe the process across rounds. We will state this limit more clearly in the revised version, and we list a multi-round analysis as future work.

Reviewer_J3VbOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This is an empirical analysis paper. It covers six self-evolving RL methods (R-Zero, SPICE, SPIRAL, AZR, Agent0, Vocabulary Dropout) on six 4–8B base models, with GRPO on human-labelled DAPO-Math-17k as a reference. That is more than 350 checkpoints, each sampled 128 times on 2,866 math questions.

The main findings:

  1. Gains come from what the model already knew: questions where the base model's plurality answer is already correct account for a median of 95.9% of the pass@1 gain, and about 31% of the gain is answer formatting. Pass@128 barely moves or falls.

  2. Self-evolving RL fixes fewer errors: it corrects about half as many errors as human-label RL (3.2 vs 7.0 percent on initially wrong questions). External labels (code, games) prevent erosion but don't close this gap.

  3. Later proposers don't help: they teach a fixed solver no more than early ones. In R-Zero, the share of invalid questions rises and label accuracy falls from 76% to 23% over 10 rounds. A frozen proposer, reused questions, or even mostly invalid perturbed questions give the same round-1 gain.

A one-step KL-regularized update model explains the pattern: the gain is proportional to p(1−p)·D, and a proposer can only re-weight which questions are trained on.

Critical strengths

This paper does thorough experiments and reaches interesting conclusion with solid theoretic support.

Critical weaknesses
  1. Narrow scope: The paper focuses on math only at 4–8B. Several methods (AZR, SPIRAL, Agent0) target code, games or tools, and judging them on math measures transfer, not their intended domain.

  2. No human validation of LLM judgements: The work's task validity, label correctness and CoT validity all rest on Claude judges. The paper reports agreement between two judge passes, but there is no human spot-check in the main text.

  3. The 4,096-token output limit is in deep question: this may truncate long reasoning on AIME and Olympiad problems and depress pass@k, especially at the hardest end where "new capability" would show up.

  4. Incremental relative to prior work: The "sharpening, not expansion" conclusion largely extends Yue et al. (2025) from RLVR to self-evolving RL. The contribution is mainly the systematic measurement, plus the proposer-side findings.

Questions for authors

Can you add a human audit of the LLM-judged task validity?

2 comments

We thank the reviewer for the careful reading and the constructive feedback.

W1: Scope. We agree that for AZR, SPIRAL and Agent0, mathematics does not fully reflect what these methods learn, and we will add this point to the limitations discussion in the revised version. We selected mathematics because it permits one controlled comparison across all six methods, and each of these papers also reports results on mathematical reasoning. For the rebuttal we also evaluated our AZR checkpoints in AZR's own domain, with the AZR paper's code protocol and official scripts. AZR does not improve HumanEval+ or MBPP+ on either model (−3.0 to −0.3 pp). Its in-domain gain is on LiveCodeBench: +4.4 to +4.7 pp on Qwen2.5-7B, and +2.4 pp on Qwen3-8B after we correct an answer-format failure. The same checkpoints gain +13.9 and +8.5 pp on mathematics, so the evaluation on mathematics does not understate AZR. We will add these results to the paper.

We do not claim the result for models above 8B. This limit comes from the GPU compute available to us: a single run can take more than a week on eight GPUs, as we discuss in the limitations section.

W2 / Q1: Human audit. To check the quality of our LLM judge, we conducted a human audit, as the reviewer requested. Two annotators independently labelled 232 generated questions and their answers without access to the judge output. Annotator–judge agreement is 92.7–94.8% (κ = 0.85–0.90) for task validity and 93.3–94.0% (κ = 0.87–0.88) for label correctness. Agreement between the two annotators is 94.8% (κ = 0.90) for task validity and 100% (κ = 1.00) for label correctness on the 123 questions that both solved. We will add this audit to the revised version.

W3: The 4,096-token limit. We strongly agree with the reviewer that a 4,096-token output limit can make the results on the hardest questions less reliable. We therefore tested this in two ways: on the broad question set with a 16,384-token limit, and on AIME, the hardest questions, with a 32,768-token limit.

Continuation to 16,384 tokens (1,847 questions). We continued every sample that reached 4,096 tokens, with identical sampling, up to 16,384 tokens (21 model–method pairs, 1,847 questions, 128 samples).

  • Across all 42 peak and final checkpoints, pass@1 changes by at most 0.9 points and pass@128 by at most 0.8.
  • Four of the six base models change by at most 0.1 points in pass@1 and 0.6 in pass@128.
  • Granite-4.1-8B and OLMo-3-7B write long outputs at initialization. With the longer limit their initializations gain 4.6–4.9 points in pass@128, and their trained checkpoints do not. Relative to initialization, pass@128 at the OLMo gold-label peak goes from +0.4 to −3.5.

Fresh sampling at 32,768 tokens (AIME only). To further address the reviewer's concern that the token limit can cause problems on complex questions such as AIME, we added an evaluation in which we sampled all 60 AIME 2024 and 2025 questions again (128 samples each) with a 32,768-token limit. The table below gives AIME pass@1 and pass@128 for the base model and the peak checkpoint of each method, for two base models. The experiment is still running, and we will add the other models to the revised version.

AIME 2024+2025, 32,768 tokensQwen3-4B pass@1Qwen3-4B pass@128Qwen2.5-7B pass@1Qwen2.5-7B pass@128
Base model4.550.02.840.0
R-Zero6.050.05.340.0
SPICE5.356.74.845.0
SPIRAL10.048.34.348.3
Agent06.251.74.448.3
Vocabulary Dropout6.650.05.448.3
Human-label RL15.358.36.048.3
  • On AIME, the results at 32,768 tokens lead to the same conclusions as the results at 4,096 tokens: every method raises pass@1, R-Zero does not raise pass@128 on either model, and no self-evolving method exceeds human-label RL on either metric.
  • The numbers also change little between the two limits. On the same 60 AIME questions, pass@1 differs by at most 0.6 points for every checkpoint in the table. Pass@128, averaged over the self-evolving methods, differs by about 2 points: 49.0 at 4,096 tokens and 51.3 at 32,768 tokens on Qwen3-4B, and 44.3 and 46.0 on Qwen2.5-7B.
  • The checkpoint that gains most from the longer limit is the human-label RL peak on Qwen3-4B. Its AIME pass@128 rises from 51.7 to 58.3, because 36% of its AIME samples reached the limit at 4,096 tokens.

In summary, the 4,096-token limit understated two base models in the first test and the human-label RL peak on Qwen3-4B in the second test. It did not hide new capability of the self-evolving methods, including on AIME, so our conclusions are unchanged. We will add all of these results to the revised version.

W4: Relation to prior work.

WorkGenerated tasksLabel sources comparedShare of gain attributed per questionHuman-label RL on the same modelsProposer studied
Yue et al. (2025)✗1: human labels✗Only setting✗
He et al. (2026)✗1: intrinsic (certainty, majority vote)✗1 model✗
Ours✓ 6 methods4: majority vote, proposer-stated, external checks, human labels✓✓ 6 models✓

On the solver side, our aggregate result agrees with Yue et al.: pass@1 rises and pass@128 does not. We do not claim this pattern as new. But Yue et al. train on fixed, human-labelled data, and self-evolving methods were proposed to go beyond that limit by generating new tasks. Their result does not tell us if this works. We show that it does not, and we add two results. First, the label source decides what happens after the first gain: methods that label by majority vote, such as R-Zero, erode as their labels get worse, and methods with external checks do not. Second, external checks are not sufficient: these methods still correct half as many errors as human-label RL on the same models and questions (3.2% against 7.0%).

The proposer side has no counterpart in Yue et al., because fixed data has no proposer. The proposer is the central component of a self-evolving method: it replaces the human dataset, and open-ended improvement requires that it gives better tasks over time. We test this with the solver held fixed, and later proposers teach no more than early ones.

We will add this table and discussion to the introduction and the related-work section.

We will add this discussion to the revised paper. We thank the reviewer again for these comments.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.