- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The research background of this paper addresses the issue of extreme difficulty polarization in competitive programming datasets used for model training: when problems are too simple, agents solve them almost universally, whereas when they are too difficult, agents rarely succeed. Neither scenario provides a positive learning signal for reinforcement learning (RL). To resolve this, the authors propose a paradigm called "AutoCode-RL." The core approach involves rewriting existing problems—enhancing the complexity of overly simple problems and simplifying overly complex ones—to generate medium-difficulty problems that are more conducive to training. This method achieves superior results compared to baselines while requiring a smaller training dataset. Using original problems from recent Codeforces contests, the authors employed a fixed generator and two sets of prompts to produce rewritten problems that pass test-case validation, forming the synthetic component of the "FRESH-CF" dataset. During the training phase, the GPT-OSS-20B model was trained using the GRPO algorithm and subsequently evaluated on LCB Pro and LiveCodeBench v6 Hard; the results demonstrate that training on these synthetic problems outperforms training on the original, unmodified problems. To verify the generalizability of this paradigm, the authors replicated the experiments by substituting the GPT-OSS-20B model with Qwen3.5-9B, confirming that the approach yields performance gains across different base models. Furthermore, the authors extended their evaluation to include mathematical reasoning datasets, demonstrating that models trained via AutoCode-RL also exhibit improved performance in the domain of mathematics.
Critical strengths
This paper proposes a new paradigm for training reinforcement learning models specifically for competitive programming problems. The authors identify a key challenge: the difficulty of such problems tends to be highly polarized. To address this, they employ a problem-generation model to perform bidirectional problem rewriting, resulting in the creation of the FRESH-CF dataset. Compared to the baseline "BenchEvolver" mentioned in the original work, the proposed approach—AutoCode-RL—enhances model capabilities while requiring a smaller training set and fewer inference steps. Given its low computational overhead, AutoCode-RL offers a valuable reference for training intelligent agents to solve competitive programming problems.
Critical weaknesses
- This paper has certain limitations regarding methodological innovation. Its contributions are primarily focused on the combination of algorithms and the construction of a dataset. Specifically, the authors employ a fixed generator—guided by prompts provided in the appendix—to modify original problem statements, while utilizing the existing "AutoCode" framework for the construction and verification of these adapted problems. For model training, the paper relies on the established GRPO algorithm. The primary incremental contribution lies in the creation of the FRESH-CF dataset and the subsequent evaluation of the solver’s performance; however, as a new research paradigm, the work lacks a significant degree of novelty.
- The validation of the proposed innovations is insufficient. While the authors claim that adapting competitive programming problems facilitates the solver's learning process, they fail to specify the individual contributions of the bidirectional difficulty adjustments.
- The training process utilizes only a single random seed, failing to account for the variance that might arise from different random seeds. Consequently, the paper does not adequately demonstrate the reproducibility of the reported performance gains. Furthermore, the trained solver exhibits a relatively low success rate on difficult problems.
- The presentation and content of the tables and figures are flawed. The visualizations are overly simplistic and lack clear labeling of key information, which leads to ambiguity for the reader.
Questions for authors
- During the training process, the paper utilizes both "hard" (more challenging) and "simplified" problems. However, the respective contributions of these two types of training sets to the solver's learning process are not explicitly quantified. Could the authors provide supplementary experiments—or other results—that help distinguish the impact of each?
- Regarding the solver's out-of-distribution (OOD) generalization: while the paper demonstrates that the FRESH-CF paradigm improves performance on mathematical reasoning tasks and on "hard" problems derived from "simplified" ones, the evaluation for programming tasks itself remains incomplete. Specifically, the performance on algorithm categories not covered in the training set has not been explicitly assessed; there remains a possibility that the test problems overlap with the algorithms present in the training data. Further clarification on this point is requested.
- Why was Qwen3.5-9B selected as the second solver for the supplementary experiments, rather than other open-source models? Was this choice primarily driven by model capabilities, parameter scale, computational resources, or other factors? Furthermore, given that only one additional Qwen-based model was used for cross-model validation, could the authors elaborate on whether the AutoCode-RL paradigm possesses broader generalizability?
2 comments
(Continued from the previous comment.)
(W4) Figures and tables. Thank you. We have redrawn the main result figures. Figure 3 now shows pass@1 on Easy391 and Medium251 for all five training configurations, with every value labeled and the base model marked, next to Medium56 pass@k with labeled endpoints. Figure 4 now combines the analysis of generated tasks with the evaluation on the 63 harder originals (formerly Figure 5) in four panels: how the algorithm tags change from seed to rewrite for each operator, the distribution of rating changes with its mean, pass@k on the harder originals, and the result on each of them. Each new caption states the metric, the problem set and, where relevant, the number of samples per problem, and the tables added in the revision follow the same rule. If a particular figure or table is still unclear, we would be glad to revise it further.
(Q2) Algorithm categories outside training; overlap. We checked this in three ways.
First, no problems are shared. Training seeds come from Codeforces contests held after the LCB-Pro collection window (October 2025 to June 2026, against September 2025 or earlier), so no problem or contest appears in both.
Second, we looked at performance by algorithm category. The annotator used in §6 (a frozen GPT-5.5) tags every training and evaluation problem with algorithm categories. We split the 630 tagged LCB-Pro problems (Easy391 and Medium251) by whether they carry a tag that appears on fewer than 5% of the 147 training tasks. These rare tags are Geometry (0%), Two Pointers (2.1%), Brute Force (2.8%) and Constructive (3.5%).
| LCB-Pro problems | N | base pass@1 (%) | AutoCode-RL pass@1 (%) | difference [95% CI] |
|---|---|---|---|---|
| with a tag rare in training | 130 | 44.6 | 53.5 | +8.8 [+5.0, +12.9] |
| all tags common in training | 500 | 49.6 | 56.4 | +6.8 [+4.8, +8.8] |
The gain on problems with a rare tag is at least as large as on the rest, and this also holds with a 10% threshold (+8.3 [+5.3, +11.3] against +6.4 [+4.2, +8.7]). Per tag, pass@1 rises by 12.5 points on Two Pointers (20 problems), 9.3 on Constructive (89) and 9.3 on Graphs (54), all of which appear in fewer than 9% of training tasks. On Geometry, which no training task has, accepted samples go from 14 to 20 out of 36 (9 problems).
Third, we compared problem statements. In statement-embedding space, the rewrites are not closer to the evaluation problems than their seeds are. Taken alone, the rewrite pool is further from the evaluation set than the seed pool: the mean cosine similarity between each evaluation problem and its nearest training problem is 0.037 lower on easy and 0.020 lower on medium. Yet the rewrites train the stronger model.
These checks indicate that the gains do not come from overlap between training and test categories. We will add the per-category analysis to the appendix.
(Q3) Why Qwen3.5-9B; generality across models. We chose Qwen3.5-9B to test AutoCode-RL on a different model family, architecture and scale. GPT-OSS-20B is a mixture-of-experts model, while Qwen3.5-9B is a dense model from another family with a native thinking mode. At 9B parameters, full-parameter RL also fits our hardware. We trained Qwen3.5-9B mainly on AMD GPUs, whereas GPT-OSS-20B was trained on NVIDIA H200s, so the second backbone also checks that the gains are not tied to one GPU platform.
Trained on the same 147 synthetic problems, Qwen3.5-9B improves Easy96 pass@1 from 62.6% to 69.4% and Medium56 pass@16 from 21.4% to 33.9%, solving 19 problems instead of 12 (Appendix J).
The task construction and the execution reward make no assumption about the solver's architecture, and the method improves both model families, on two different GPU platforms.
We hope this clarifies our contribution and the new evidence, and we are happy to answer further questions. If our replies address your concerns, we would be grateful if you would consider raising your score.
Thank you for the detailed summary and the questions. We reply to each weakness and question below.
(W1) Methodological novelty. Our main contribution is to show that, for code RL, rewriting real problems in both directions of difficulty gives better training data than the real problems themselves. A frozen setter, decoupled from the solver, makes each problem harder or easier, toward the range the solver can learn from. At a matched rollout budget, the rewrites train a stronger solver than the original problems they came from (Easy391 pass@1 77.2 against 72.0; base 68.0), and the new ablation (reply to W2) shows that neither operator alone matches using both. Simplified problems also act as stepping stones: they improve the solver on their harder originals, which it never trains on, while training on those originals directly does not help them. Finally, we release a contamination-clean corpus with verified tests, and RL on it beats supervised fine-tuning on a stronger model's accepted solutions.
Compared with BenchEvolver, which also generates problems for RL, AutoCode-RL uses about a tenth of the training problems and nominal rollouts. It scores higher on the 96 easy problems BenchEvolver evaluates (74.2 against a reported 72.9). Under our protocol, BenchEvolver's released checkpoint also trails ours on all 391 easy problems (73.5 against 77.2) and on Medium56 (12.2 against 14.3).
(W2, Q1) Contributions of the two adjustment directions. Reviewer ZmpC raised the same point, and Table A in the general response has the full results. We trained enhancement-only (84 problems) and simplification-only (63 problems) arms with the same 10,240-rollout budget as the main runs.
The two directions help on different problems. Among the arms trained on a single kind of problem, simplification-only has the highest Easy391 pass@1 (72.6, against 72.0 for originals and 70.3 for enhancement), and enhancement-only has the highest Medium56 pass@1 (12.8, against 12.7 for originals and 12.2 for simplification; base 9.8). Using both gives the best result on both sets (77.2 and 14.3). Simplified problems also act as stepping stones to harder ones. On the 63 harder originals of the simplified problems, which the synthetic arms never train on, the simplification-only model solves 32 within 16 samples, against 25 for the base model and 23 for the enhancement-only model.
These results are in Appendices K and I of the revision, and our reply to Reviewer ZmpC (W1/Q1 and W2/Q2) discusses them in more detail.
(W3) Single training seed; low success on hard problems. A full-parameter run takes 16 H200 GPUs for about a day, and our compute budget does not cover another training seed. For post-training at this cost, we do not think repeated seeds are the only way to show that a result is reliable, and the paper offers two other kinds of evidence.
First, the main comparison uses paired statistics over problems, and it holds on the full 391-problem easy split, four times the size of the subset in the main comparison. Synthetic minus originals is +5.2 points (95% CI [+3.1, +7.4]).
Second, the revision contains six full-parameter runs from the same base model, each on a different task pool: originals, synthetic, their union, synthetic plus 79 originals, enhancement-only and simplification-only. All six beat the base model on LCB-Pro easy (Easy391 70.3 to 77.2 against 68.0 for the five runs evaluated there; Easy96 72.3 against 65.0 for the sixth), and the synthetic run is the best. These runs differ in data as well as in randomness, so they do not replace repeated seeds of one configuration. They do show that the improvement over the base model does not depend on a single run.
On hard problems, low absolute success is expected. The base model solves none of them (0 of 1,024 samples, 16 per problem), and neither does the model trained on the originals. Ours solves two distinct hard problems, each with a program that passes all 60 official tests. For this reason we report the hard split as the number of problems solved, from zero to two, instead of a pass@1 percentage.
(Continued in the next comment.)