acceptodds
Under review as a conference paper at ICLR 2027

AutoCode-RL: Climbing the Difficulty Pyramid with Verifiable Synthetic Problems

Abstract

Reinforcement learning with verifiable rewards (RLVR) trains a coding agent by rewarding programs that pass tests, so the model learns only from problems it solves some of the time. Competitive-programming problems, however, form a pyramid: easy problems are plentiful and hard ones are rare, so the problems that can still teach a model become scarce as it improves.We propose AutoCode-RL, in which a frozen setter model adjusts the difficulty of existing contest problems for the solver being trained: plentiful lower-rated problems become harder, so that the solver sometimes fails, and problems beyond its reach become easier stepping stones toward them. An automated pipeline builds and validates tests for each rewrite, and the solver learns from whether its programs pass them.Training GPT-OSS-20B on rewrites of Codeforces problems released after the benchmark and base model raises its single-attempt solve rate (pass@1) on LiveCodeBench-Pro by a relative on easy and on medium problems. Our trained model also solves one problem from the hard split, where the base model fails all attempts. AutoCode-RL outperforms training on the original problems with the same budget and supervised fine-tuning on a stronger model's correct solutions. To our knowledge, the resulting scores are the highest reported for GPT-OSS-20B on LiveCodeBench-Pro.The simplified problems work as stepping stones: the trained model solves seven of their harder originals that the base model never solves, none of which appeared in training. The same method also improves Qwen3.5-9B.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Reinforcement Learning, Synthetic Data, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_6TW1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The research background of this paper addresses the issue of extreme difficulty polarization in competitive programming datasets used for model training: when problems are too simple, agents solve them almost universally, whereas when they are too difficult, agents rarely succeed. Neither scenario provides a positive learning signal for reinforcement learning (RL). To resolve this, the authors propose a paradigm called "AutoCode-RL." The core approach involves rewriting existing problems—enhancing the complexity of overly simple problems and simplifying overly complex ones—to generate medium-difficulty problems that are more conducive to training. This method achieves superior results compared to baselines while requiring a smaller training dataset. Using original problems from recent Codeforces contests, the authors employed a fixed generator and two sets of prompts to produce rewritten problems that pass test-case validation, forming the synthetic component of the "FRESH-CF" dataset. During the training phase, the GPT-OSS-20B model was trained using the GRPO algorithm and subsequently evaluated on LCB Pro and LiveCodeBench v6 Hard; the results demonstrate that training on these synthetic problems outperforms training on the original, unmodified problems. To verify the generalizability of this paradigm, the authors replicated the experiments by substituting the GPT-OSS-20B model with Qwen3.5-9B, confirming that the approach yields performance gains across different base models. Furthermore, the authors extended their evaluation to include mathematical reasoning datasets, demonstrating that models trained via AutoCode-RL also exhibit improved performance in the domain of mathematics.

Critical strengths

This paper proposes a new paradigm for training reinforcement learning models specifically for competitive programming problems. The authors identify a key challenge: the difficulty of such problems tends to be highly polarized. To address this, they employ a problem-generation model to perform bidirectional problem rewriting, resulting in the creation of the FRESH-CF dataset. Compared to the baseline "BenchEvolver" mentioned in the original work, the proposed approach—AutoCode-RL—enhances model capabilities while requiring a smaller training set and fewer inference steps. Given its low computational overhead, AutoCode-RL offers a valuable reference for training intelligent agents to solve competitive programming problems.

Critical weaknesses
  1. This paper has certain limitations regarding methodological innovation. Its contributions are primarily focused on the combination of algorithms and the construction of a dataset. Specifically, the authors employ a fixed generator—guided by prompts provided in the appendix—to modify original problem statements, while utilizing the existing "AutoCode" framework for the construction and verification of these adapted problems. For model training, the paper relies on the established GRPO algorithm. The primary incremental contribution lies in the creation of the FRESH-CF dataset and the subsequent evaluation of the solver’s performance; however, as a new research paradigm, the work lacks a significant degree of novelty.
  2. The validation of the proposed innovations is insufficient. While the authors claim that adapting competitive programming problems facilitates the solver's learning process, they fail to specify the individual contributions of the bidirectional difficulty adjustments.
  3. The training process utilizes only a single random seed, failing to account for the variance that might arise from different random seeds. Consequently, the paper does not adequately demonstrate the reproducibility of the reported performance gains. Furthermore, the trained solver exhibits a relatively low success rate on difficult problems.
  4. The presentation and content of the tables and figures are flawed. The visualizations are overly simplistic and lack clear labeling of key information, which leads to ambiguity for the reader.
Questions for authors
  1. During the training process, the paper utilizes both "hard" (more challenging) and "simplified" problems. However, the respective contributions of these two types of training sets to the solver's learning process are not explicitly quantified. Could the authors provide supplementary experiments—or other results—that help distinguish the impact of each?
  2. Regarding the solver's out-of-distribution (OOD) generalization: while the paper demonstrates that the FRESH-CF paradigm improves performance on mathematical reasoning tasks and on "hard" problems derived from "simplified" ones, the evaluation for programming tasks itself remains incomplete. Specifically, the performance on algorithm categories not covered in the training set has not been explicitly assessed; there remains a possibility that the test problems overlap with the algorithms present in the training data. Further clarification on this point is requested.
  3. Why was Qwen3.5-9B selected as the second solver for the supplementary experiments, rather than other open-source models? Was this choice primarily driven by model capabilities, parameter scale, computational resources, or other factors? Furthermore, given that only one additional Qwen-based model was used for cross-model validation, could the authors elaborate on whether the AutoCode-RL paradigm possesses broader generalizability?
2 comments

Thank you for the detailed summary and the questions. We reply to each weakness and question below.

(W1) Methodological novelty. Our main contribution is to show that, for code RL, rewriting real problems in both directions of difficulty gives better training data than the real problems themselves. A frozen setter, decoupled from the solver, makes each problem harder or easier, toward the range the solver can learn from. At a matched rollout budget, the rewrites train a stronger solver than the original problems they came from (Easy391 pass@1 77.2 against 72.0; base 68.0), and the new ablation (reply to W2) shows that neither operator alone matches using both. Simplified problems also act as stepping stones: they improve the solver on their harder originals, which it never trains on, while training on those originals directly does not help them. Finally, we release a contamination-clean corpus with verified tests, and RL on it beats supervised fine-tuning on a stronger model's accepted solutions.

Compared with BenchEvolver, which also generates problems for RL, AutoCode-RL uses about a tenth of the training problems and nominal rollouts. It scores higher on the 96 easy problems BenchEvolver evaluates (74.2 against a reported 72.9). Under our protocol, BenchEvolver's released checkpoint also trails ours on all 391 easy problems (73.5 against 77.2) and on Medium56 (12.2 against 14.3).

(W2, Q1) Contributions of the two adjustment directions. Reviewer ZmpC raised the same point, and Table A in the general response has the full results. We trained enhancement-only (84 problems) and simplification-only (63 problems) arms with the same 10,240-rollout budget as the main runs.

The two directions help on different problems. Among the arms trained on a single kind of problem, simplification-only has the highest Easy391 pass@1 (72.6, against 72.0 for originals and 70.3 for enhancement), and enhancement-only has the highest Medium56 pass@1 (12.8, against 12.7 for originals and 12.2 for simplification; base 9.8). Using both gives the best result on both sets (77.2 and 14.3). Simplified problems also act as stepping stones to harder ones. On the 63 harder originals of the simplified problems, which the synthetic arms never train on, the simplification-only model solves 32 within 16 samples, against 25 for the base model and 23 for the enhancement-only model.

These results are in Appendices K and I of the revision, and our reply to Reviewer ZmpC (W1/Q1 and W2/Q2) discusses them in more detail.

(W3) Single training seed; low success on hard problems. A full-parameter run takes 16 H200 GPUs for about a day, and our compute budget does not cover another training seed. For post-training at this cost, we do not think repeated seeds are the only way to show that a result is reliable, and the paper offers two other kinds of evidence.

First, the main comparison uses paired statistics over problems, and it holds on the full 391-problem easy split, four times the size of the subset in the main comparison. Synthetic minus originals is +5.2 points (95% CI [+3.1, +7.4]).

Second, the revision contains six full-parameter runs from the same base model, each on a different task pool: originals, synthetic, their union, synthetic plus 79 originals, enhancement-only and simplification-only. All six beat the base model on LCB-Pro easy (Easy391 70.3 to 77.2 against 68.0 for the five runs evaluated there; Easy96 72.3 against 65.0 for the sixth), and the synthetic run is the best. These runs differ in data as well as in randomness, so they do not replace repeated seeds of one configuration. They do show that the improvement over the base model does not depend on a single run.

On hard problems, low absolute success is expected. The base model solves none of them (0 of 1,024 samples, 16 per problem), and neither does the model trained on the originals. Ours solves two distinct hard problems, each with a program that passes all 60 official tests. For this reason we report the hard split as the number of problems solved, from zero to two, instead of a pass@1 percentage.

(Continued in the next comment.)

(Continued from the previous comment.)

(W4) Figures and tables. Thank you. We have redrawn the main result figures. Figure 3 now shows pass@1 on Easy391 and Medium251 for all five training configurations, with every value labeled and the base model marked, next to Medium56 pass@k with labeled endpoints. Figure 4 now combines the analysis of generated tasks with the evaluation on the 63 harder originals (formerly Figure 5) in four panels: how the algorithm tags change from seed to rewrite for each operator, the distribution of rating changes with its mean, pass@k on the harder originals, and the result on each of them. Each new caption states the metric, the problem set and, where relevant, the number of samples per problem, and the tables added in the revision follow the same rule. If a particular figure or table is still unclear, we would be glad to revise it further.

(Q2) Algorithm categories outside training; overlap. We checked this in three ways.

First, no problems are shared. Training seeds come from Codeforces contests held after the LCB-Pro collection window (October 2025 to June 2026, against September 2025 or earlier), so no problem or contest appears in both.

Second, we looked at performance by algorithm category. The annotator used in §6 (a frozen GPT-5.5) tags every training and evaluation problem with algorithm categories. We split the 630 tagged LCB-Pro problems (Easy391 and Medium251) by whether they carry a tag that appears on fewer than 5% of the 147 training tasks. These rare tags are Geometry (0%), Two Pointers (2.1%), Brute Force (2.8%) and Constructive (3.5%).

LCB-Pro problemsNbase pass@1 (%)AutoCode-RL pass@1 (%)difference [95% CI]
with a tag rare in training13044.653.5+8.8 [+5.0, +12.9]
all tags common in training50049.656.4+6.8 [+4.8, +8.8]

The gain on problems with a rare tag is at least as large as on the rest, and this also holds with a 10% threshold (+8.3 [+5.3, +11.3] against +6.4 [+4.2, +8.7]). Per tag, pass@1 rises by 12.5 points on Two Pointers (20 problems), 9.3 on Constructive (89) and 9.3 on Graphs (54), all of which appear in fewer than 9% of training tasks. On Geometry, which no training task has, accepted samples go from 14 to 20 out of 36 (9 problems).

Third, we compared problem statements. In statement-embedding space, the rewrites are not closer to the evaluation problems than their seeds are. Taken alone, the rewrite pool is further from the evaluation set than the seed pool: the mean cosine similarity between each evaluation problem and its nearest training problem is 0.037 lower on easy and 0.020 lower on medium. Yet the rewrites train the stronger model.

These checks indicate that the gains do not come from overlap between training and test categories. We will add the per-category analysis to the appendix.

(Q3) Why Qwen3.5-9B; generality across models. We chose Qwen3.5-9B to test AutoCode-RL on a different model family, architecture and scale. GPT-OSS-20B is a mixture-of-experts model, while Qwen3.5-9B is a dense model from another family with a native thinking mode. At 9B parameters, full-parameter RL also fits our hardware. We trained Qwen3.5-9B mainly on AMD GPUs, whereas GPT-OSS-20B was trained on NVIDIA H200s, so the second backbone also checks that the gains are not tied to one GPU platform.

Trained on the same 147 synthetic problems, Qwen3.5-9B improves Easy96 pass@1 from 62.6% to 69.4% and Medium56 pass@16 from 21.4% to 33.9%, solving 19 problems instead of 12 (Appendix J).

The task construction and the execution reward make no assumption about the solver's architecture, and the method improves both model families, on two different GPU platforms.

We hope this clarifies our contribution and the new evidence, and we are happy to answer further questions. If our replies address your concerns, we would be grateful if you would consider raising your score.

Reviewer_22qtOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • Proposes AutoCode-RL framework, which uses a frozen LLM as the setter to rewrite problems based on its difficulties to synthesize additional dataset which provides richer reward signal for GRPO style RL.
  • Built Fresh-CF, a new dataset from Codeforces contests held between October 2025 and June 2026.
  • After training, saw a lot of increases in LCB-Pro, and authors report this is the highest for gpt-oss-20b model.
Critical strengths
  • AutoCode-RL's design is logical and addresses an issue that is well known. Moreover, the results so far on LCB is quite impressive and showed its effectiveness and promise.
  • Publically released Fresh-CF, which is a useful dataset for people to use and carry out coding RL tasks on.
Critical weaknesses
  • The paper mostly relies on LCB results. The authors should add at least a few more coding benchmarks such as CodeContests, TACO, BigCodeBench, etc. This can establish that the model is not only learning some features that are specific to LCB (I am aware of the Math benchmarks in the appendices, this comments is written with that in mind). This is my main concern.
  • Would be nice to do an ablation study which uses the frozen GPT-OSS-20b as the setter, and see the performance difference compare to the current setup which uses GPT-5.5. This provides more information on the design effectiveness of AutoCode-RL vs BenchEvolver.
Questions for authors
  • Have you guys done any human study on the quality and correctness of Fresh-CF? If not, would be nice to do a small scale study and report the findings.
  • I can't seem to access the anonymous dataset on huggingface, would you mind ensuring that is accessible?
2 comments

Thank you for the positive comments on the design and the corpus, and for the concrete suggestions. We start with your main concern.

(W1) Benchmarks beyond LiveCodeBench. We added CodeContests and 1,000 TACO problems, comparing the base model with the final AutoCode-RL checkpoint at 4 samples per problem (Appendix C; §5 summarizes the result).

BenchmarkModelpass@1 (%)pass@4 (%)
CodeContestsBase53.070.3
CodeContestsAutoCode-RL60.073.3
TACOBase30.038.9
TACOAutoCode-RL32.440.0

AutoCode-RL improves both metrics on both benchmarks, so the gains are not tied to LiveCodeBench problems.

We should also clarify that the paper already uses two different benchmarks, LiveCodeBench-Pro and LiveCodeBench. Despite the similar names, they are separate datasets. The main results are on LiveCodeBench-Pro (easy, medium and hard splits). LiveCodeBench v6 Hard is a separate set of 80 AtCoder and LeetCode problems, on which AutoCode-RL improves pass@1 by 10 points (Appendix C). With CodeContests and TACO, the coding evaluation now covers four benchmarks. The paper also reports zero-shot transfer to competition math: MATH-500 goes from 95.2% to 96.5%, and AIME 2024+2025 from 73.5% to 77.3%.

(W2) GPT-OSS-20B as the setter. Thank you for the suggestion. Two parts of the paper bear on whether the gains come from the setter's strength.

First, the main comparison holds the setter fixed. The original problems in our corpus get their tests from the same GPT-5.5 construction pipeline as the rewrites (§3.2), so the comparison between training on rewrites and training on originals differs only in whether the problems were rewritten. At the same rollout budget, training on the rewrites reaches 77.2% Easy391 pass@1, against 72.0% for training on the originals (base 68.0%).

Second, the setter's knowledge does not reach the solver directly. The setter supplies only problem statements and tests. The solver never sees its solutions or reasoning and learns from its own attempts through execution rewards. If the gain came from transferring a stronger model's knowledge, imitating a stronger model's correct solutions should work at least as well. Table 4 tests this with full-parameter SFT of GPT-OSS-20B on Kimi K2.7's correct solutions and reasoning for 295 fresh-CF problems. The best SFT checkpoint reaches 68.4% Easy96 pass@1 and 39.3% Medium56 pass@16, against 74.2% and 50.0% for RL (base 64.7% and 32.1%).

Running the pipeline with GPT-OSS-20B as the setter would show how far the method goes without a frontier model, and we will add it to the future work.

(Q1) Human study of Fresh-CF quality. Yes. For the revision, an ICPC World Finalist manually reviewed five simplified and five enhanced problems from Fresh-CF and found no correctness or test-data issues. The audit is described in Appendix L.

It adds to two automatic checks. The pipeline's validators, checkers, generators and reference execution filter every task package (details in our reply to Reviewer ZmpC, Q3). And during training, the solver wrote an accepted program for 140 of the 147 synthetic tasks, independently of the reference solution.

(Q2) Anonymous dataset access. Thank you for your interest in our dataset. We checked again, and access works on our side: the dataset is public and does not require a Hugging Face login.

https://huggingface.co/datasets/anonymous1926/autocode-fresh-cf

It contains the 373 test packages, the GRPO prompts and the SFT data. The anonymous code repository is also available:

https://anonymous.4open.science/r/AutoCodeRL-6A6A/

If the page still does not load for you, please let us know the error message and we will be happy to help.

Thank you again. We would like to know whether the added benchmarks address your main concern. If they do, we would be grateful if you would consider raising your score.

Thanks for the reply. This addresses my main concern, and I have raised my score accordingly.

Reviewer_ZmpCOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether difficulty-adjusted synthetic programming tasks can improve code reinforcement learning. A frozen setter makes easier contest problems harder problems easier. The resulting task pool trains a separate solver with GRPO and execution rewards. Experiments report gains over the base model and training on the original tasks.

Critical strengths

• Training on the generated problems improves held-out coding accuracy over training on the original problems. • The paper presents a clear, reusable pipeline and demonstrates improvements on two model families.

Critical weaknesses

• It has not been determined what each operator contributes, nor is it clear whether selecting a primitive task that is easier to learn would yield comparable benefits. • Neither operator is evaluated separately under matched training conditions. The improvement on the 63 harder originals follows joint training, leaving the specific benefit of their paired simplifications unclear. • The paper does not report an independent check of whether the generated reference solutions correctly implement the problem statements. • The widening performance gap at larger sampling budgets does not establish capability expansion.

Questions for authors

• How does enhancement-only, simplification-only, and combined training perform under the same training budget? • Does adding simplification improve performance on the 63 harder originals beyond enhancement-only training? • Does the pipeline check generated reference programs against independent solutions or brute-force results? • What evidence distinguishes capability expansion from an increased probability of generating existing correct solutions?

5 comments

Thank you for the review. We answer the four questions in order; two of them led to new experiments.

(W1, Q1) Operator contributions and easier original tasks. The full results are in Table A of the general response; we summarize them here. We added enhancement-only (84 tasks) and simplification-only (63 tasks) arms next to AutoCode-RL (147 tasks, both operators). All arms use the same 10,240-rollout budget and training settings, and we evaluate the final checkpoints with 4 samples per problem on Easy391 and 16 on Medium56.

Each operator helps most in the direction it moves the difficulty. Among the arms trained on a single kind of problem, simplification-only has the highest Easy391 pass@1 (72.6%, against 72.0% for originals and 70.3% for enhancement), and enhancement-only has the highest Medium56 pass@1 (12.8%, against 12.7% and 12.2%; base 9.8%). AutoCode-RL, which uses both, has the highest pass@1 on both sets: 77.2% on Easy391, 6.9 points above enhancement-only and 4.7 above simplification-only (both significant), and 14.3% on Medium56. This supports using both directions.

On easier original tasks: the originals-only pool already includes the 1200–1600-rated seeds that we enhance. At the same rollout budget, training on the rewrites gives 77.2% Easy391 pass@1 and training on this pool gives 72.0%. The difficulty rating alone does not explain the difference. Enhanced problems and unrewritten originals have the same median annotated difficulty (1900), but the solver passes 61–63% of the enhanced problems on first exposure and only 27–29% of the unrewritten originals. Our answer to Q2 below shows how simplified problems help on the harder originals.

Appendix K reports these results, and Appendix I the harder-originals comparison.

(W2, Q2) Do simplifications help their harder originals beyond enhancement?. Yes. Table A in the general response includes the 63 harder originals (Harder63, 16 samples per problem). None of the three synthetic pools (enhancement-only, simplification-only and AutoCode-RL) contains these problems.

Compared with enhancement-only, AutoCode-RL is 4.9 points higher at pass@1 (16.4% against 11.5%) and 14.3 points higher at pass@16 (50.8% against 36.5%), and it solves nine more problems (32 against 23). Most of the coverage gain comes from simplification. The simplification-only model also solves 32 problems, and it is 6.3 points above enhancement-only at pass@4 (30.0% against 23.7%). Adding enhancement on top raises pass@1 from 13.3% to 16.4% and pass@4 from 30.0% to 34.5%.

The originals-only model trains on these 63 problems directly. Relative to the base it gains four solved problems and loses four, so coverage stays at 25/63. Training on their simplified versions reaches 32/63 without ever seeing the originals. Simplified problems therefore act as stepping stones to their harder originals, and using both operators gives the best single-attempt accuracy.

The comparison is now in Appendix I, and we updated the discussion of transfer to harder originals accordingly.

(Continued in the next comment.)

(Continued from the previous comment.)

(W3, Q3) Independent check of reference solutions. Each generated task goes through automated cross-checks between validators, checkers, test generators and the reference solution. Three candidate input validators and three output checkers are compiled and checked against unit tests. Six generators produce inputs, which the selected validator filters. The reference then produces the expected outputs and must pass the finished suite. These checks rejected 10 of 157 rewrite attempts, leaving 147 executable tasks.

This test construction comes from AutoCode, whose study reported 98.7% agreement with official verdicts on LLM submissions to 720 Codeforces problems (§3.2). On top of that, the solver wrote accepted programs for 140 of the 147 synthetic tasks during training. These programs are written independently of the reference and pass the same test suites.

Our central claim is that training on these tasks improves the solver, and we test it directly: on held-out LiveCodeBench-Pro, LiveCodeBench, CodeContests and TACO, and against training on the originals at a matched budget. A separate brute-force solver and differential-testing suite for every task is not needed to establish this.

For the revision, an ICPC World Finalist also manually reviewed five simplified and five enhanced problems and found no correctness or test-data issues (Appendix L).

(W4, Q4) Capability expansion versus higher probability. Three kinds of evidence bear on this: how the gain changes with the sampling budget, hard problems the base model never solves, and transfer to harder originals that were never trained on.

On Medium56, our advantage over the base grows from 4.5 points at pass@1 to 16.1 points at pass@16. With 16 samples per problem, coverage goes from 19/56 to 28/56 (ten problems newly solved, one lost). The gain is in both single-attempt accuracy and the set of problems solved with multiple attempts.

On Hard64, the base model produces no accepted program in 1,024 samples (16 for each of the 64 problems). AutoCode-RL solves two different problems, 2046F1 and 2006E, in two separate evaluations with 16 samples per problem, and each accepted program passes all 60 official tests. On 2046F1 we drew 128 more base samples and none was accepted, while a further run of AutoCode-RL produced another program that passes all 60 tests.

Qwen3.5-9B shows the same pattern: Medium56 pass@16 rises from 21.4% to 33.9% (Appendix J). On the 63 harder originals, simplification-only training reaches 50.8% pass@16, against 39.7% for training directly on those originals (Appendix I). In each of these settings, training on difficulty-adjusted tasks increases the number of problems the model solves.

We hope this answers your questions, and we are happy to discuss any of them further. Since our replies and the new experiments address all four of your points, we would be grateful if you would reconsider your rating.

Thank you for the additional experiments and content. I think they have addressed some of my concerns.

Q1: I am still not convinced why problem rewriting is necessary. Existing methods such as DEPO [1] and DAPO [2] already improve RL training through data selection and filtering. The authors have not explained why these approaches are insufficient or compared against them.

Q4: I think there is still a logical issue in Section 5. The authors assume that the pass@k gap should shrink if RL only increases the probability of generating existing correct solutions. This assumption does not generally hold. Therefore, a larger gap at pass@16 does not establish capability expansion. Yue et al. [3] investigate this issue using larger sampling budgets and reasoning-path coverage analysis. I think the authors should provide stronger evidence or revise their claim.

References

[1] Tang et al. Towards High Data Efficiency in Reinforcement Learning with Verifiable Reward. ICLR, 2026.

[2] Yu et al. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. NeurIPS, 2025.

[3] Yue et al. Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model? NeurIPS, 2025.

Comments on the paper

1 comment

We thank all three reviewers. Three concerns come up in more than one review, and we answer them here with new experiments and revisions. Replies to each review follow below it.

Contribution of each rewrite operator. Reviewers ZmpC and 6TW1 asked what each rewrite operator contributes. We trained two new arms, enhancement-only (the 84 enhanced problems) and simplification-only (the 63 simplified problems), with the same rollout budget, hyperparameters and seed as the main runs. Table A collects the results for all five arms, and our replies to both reviewers refer to it.

Table A. Pass rates (%) of the final checkpoints. Easy391 uses 4 samples per problem; Medium56 and Harder63 use 16. Harder63 = the 63 harder originals whose simplified versions are in the training set; the originals arm trains on them, the three synthetic arms never see them. Solved = problems with at least one accepted sample.

Training problemsNEasy391 pass@1Medium56 pass@1Harder63 pass@1Harder63 pass@16 (solved / 63)
None (base)–68.09.812.039.7 (25)
Originals only22672.012.712.239.7 (25)
Enhancement only8470.312.811.536.5 (23)
Simplification only6372.612.213.350.8 (32)
AutoCode-RL (both operators)14777.214.316.450.8 (32)

The two operators help on different problems. Among the three arms trained on a single kind of problem, simplification-only has the highest Easy391 pass@1 (72.6, against 72.0 for originals-only and 70.3 for enhancement-only; +4.5 over the base, significant). Simplified problems improve the model's command of easy problems. Enhancement-only has the highest Medium56 pass@1 among the same three arms (12.8, against 12.7 and 12.2; +3.0 over the base, significant), so the harder rewrites show their benefit on medium-difficulty problems in the same way. AutoCode-RL uses both and is best on both sets (77.2 and 14.3). On Easy391 it is 6.9 points above enhancement-only and 4.7 above simplification-only, both significant.

On Harder63, simplified problems act as stepping stones to their harder originals. Simplification-only training never sees these 63 problems, yet it raises pass@16 from 39.7 to 50.8 and solves 32 of them within 16 samples, against 25 for the base. Training on the originals themselves stays at 25, and enhancement-only training reaches 23. AutoCode-RL also solves 32 and has the highest pass@1 (16.4).

Table A combines Appendix K (Easy391, Medium56) and Appendix I (Harder63) of the revision.

Evaluation beyond LiveCodeBench and seed variance. Reviewer 22qt asked for benchmarks other than LiveCodeBench, and Reviewer 6TW1 asked about the single training seed.

We added CodeContests and 1,000 TACO problems (Appendix C, summarized in §5). Pass@1 goes from 53.0 to 60.0 on CodeContests and from 30.0 to 32.4 on TACO, and pass@4 improves on both as well. Note that LiveCodeBench and LiveCodeBench-Pro are two separate benchmarks. Our main results are on LiveCodeBench-Pro; LiveCodeBench v6 Hard is a second, separate evaluation.

Each configuration was trained once, because a run costs about a day on 16 H200 GPUs. The revision does contain six full-parameter runs from the same base model on different task pools. All six beat the base model on LCB-Pro easy, and the synthetic run is the best of them. The main comparison also holds on the full 391-problem easy split under paired statistics (synthetic minus originals, +5.2 points). Our reply to Reviewer 6TW1 has the details.

Correctness of generated tasks. Reviewers ZmpC and 22qt asked how we know the generated tasks are correct. Every task passes AutoCode's automated test construction. In the AutoCode study, tests built this way agreed with official Codeforces verdicts on 98.7% of LLM submissions to 720 problems. In addition, the solver itself wrote an accepted program for 140 of the 147 synthetic tasks during training. For those tasks, a second program written independently of the reference passes every test the reference produced. For the revision, an ICPC World Finalist also manually reviewed five simplified and five enhanced problems and found no correctness or test-data issues (Appendix L). Our replies to both reviewers give the details.

We will upload the revised PDF before the end of the discussion period. It adds CodeContests and TACO results (Appendix C, summarized in §5), the rewrite-operator ablations (Appendix K, and Appendix I for the 63 harder originals) and a manual audit of rewritten problems (Appendix L). It also redraws Figure 3 and merges the former Figures 4 and 5 into a single four-panel Figure 4.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.