acceptodds
Under review as a conference paper at ICLR 2027

CodeMidas: Scaling Agentic Coding RL Environments from Code Itself

Abstract

Training capable coding agents via reinforcement learning (RL) requires diverse tasks with reliable verifiers. Open-source codebases offer a rich source of such tasks, while existing methods typically rely on development artifacts such as issues and commits, limiting the range of tasks that can be extracted. To better scale RL environments, we present CodeMidas, an agentic pipeline that turns implemented functionality in existing codebases into executable RL environments using source code as its only task-specific input. CodeMidas allocates agentic compute to every stage of environment construction: agents explore implemented functionality to formulate behavioral specifications, construct tests grounded in execution of the original code, and validate and filter candidate tasks through execution checks and repeated solution rollouts. The resulting dataset has 5,545 training tasks from 3,185 open-source codebases spanning 23 programming languages and 15 technical domains. Training MiMo-V2.5 on these tasks with GRPO improves performance on all five diverse benchmarks, covering issue repair (DeepSWE + 11.7%), whole-program construction (ProgramBench +17%), and terminal work (Terminal-Bench v2.1 +8.5%). Ablations show that increasing the number of high-quality training tasks improves performance. Trajectory analysis shows the RL-trained agent demonstrates better behaviors like increasing codebase exploration and more diverse self-verification. These results establish source code as a scalable foundation for constructing RL environments that improve coding agents.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agentic coding, reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_sqf1OpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

CodeMidas constructs 5,545 coding RL environments by removing functionality from existing repositories, generating execution-grounded tests, and filtering candidate tasks. Training improves performance on five external benchmarks and outperforms training on public PR-derived tasks under shared settings. The contribution is the complete construction and filtering pipeline.

Critical strengths
  1. I particularly value the checks beyond reference execution: leakage probes and alternative-solution audits address distinct reward failures. The human audit also documents residual errors (Appendix E).
  2. Transfer spans issue repair, program construction, translation, and terminal work; the smaller-model results further support the tasks' training value (Figure 5; Table D7).
Critical weaknesses
  1. The executable environments are the central contribution, but the 500-task supplement provides statements and test patches without repository revisions, modified starting states, or build specifications. These missing components prevent reconstructing the supplied tasks as executable training environments, limiting reuse of the demonstrated method.
  2. Outcome filtering retains tasks with both successful and failed attempts, yet §3.4 specifies only “several” attempts from “a frontier model.” The model and attempt budget determine which tasks survive and therefore the training difficulty distribution. This part of the tested method needs a precise specification.
Questions for authors
  1. Can you provide or point to repository/revision mappings, starting states, and build specifications or containers for the supplied tasks?
  2. Which model/checkpoint, number of attempts, and rollout budget define outcome filtering? Does §3's default model also apply here?
1 comment
AuthorsOpenReview ↗

Following Up on Our Response

Dear Reviewer sqf1,

Thank you again for your positive assessment and constructive suggestions. We would greatly appreciate your feedback on whether these additions address your concerns, or if any substantial issues remain.

Best regards,

The Authors

Reviewer_L7dPOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that source code itself, rather than development artifacts (issues, PRs, commits), can supply RL environments for coding agents. A four-stage agentic pipeline designs tasks from observable public entry points, synthesizes tests anchored on the reference implementation's execution, removes the target implementation, and filters the result (leakage, verifier false positives/negatives, all-pass/all-fail rollouts). The funnel yields 5,545 tasks from 22,575 candidates across 3,185 repositories and 23 languages. Training MiMo-V2.5 with GRPO on this pool is compared against a 30k+ PR-derived pool under the same configuration, with gains on five external benchmarks (e.g., SWE-bench Pro 50.3→54.4, DeepSWE 10.0→21.7, Terminal-Bench 2.1 63.7→72.2). Supporting evidence includes a second (9B) model, a quality-vs-quantity ablation, and a training-time reward audit.

Critical strengths
  1. Verifier quality is treated as a first-class problem. The solution audit found verifier issues in 3,757 of 11,930 candidates; leakage and alternative-implementation misjudgements are explicitly checked. This is more rigorous than the common "reference solution passes" criterion.
  2. Value is demonstrated on external benchmarks, not a self-built leaderboard, and from a strong starting point (SWE-bench Pro 50.3), so the margin is not an artifact of a weak baseline.
  3. The quality-vs-quantity ablation is unusually informative: a clean 5k pool outperforms an unfiltered 8k pool, which directly challenges "more data is better" pipelines.
  4. Complementary robustness checks: repository-level non-overlap, a second model, and an honest statement of the audit's limitations.
Critical weaknesses
  1. The comparison is not compute-matched, and this is the main threat to the central claim. Equal optimization steps are not equal sampling cost: the CodeMidas run retains 68.2% of groups, and trajectory lengths and environment latency differ across pools. The near-flat PR baseline (Pro +1.0, DeepSWE −1.0) is equally consistent with that pool being under-trained. Without a token- or interaction-matched curve, "source code is the better task source" cannot be separated from "more gradient signal per unit of compute".
  2. The core factor is not isolated. Source type, implementation size (median 142 vs 22 lines), and filtering strength vary together. As written, the evidence supports "this full pipeline trains better agents", not specifically "tasks derived from implemented functionality outperform artifact-derived tasks".
  3. Construction cost is not accounted for, although scalability is part of the claim (22,575 candidates processed by agents). Cost per retained task, and how it scales with repository size, is needed to evaluate the framing.
  4. Checkpoint selection uses CodeMidasVal, which is drawn from the same construction distribution as training. This is better than selecting on test sets, but a fixed-step or uniformly-selected checkpoint would make the reported gaps easier to attribute.
  5. Single backbone family; the 9B result is same-family evidence only.
Questions for authors
  1. Can you report external-benchmark performance at matched cumulative generated tokens (or matched environment interactions), rather than matched optimization steps? If the PR pool were trained to the same token budget, does the gap persist?
  2. Can you add a PR-derived pool subjected to the same filtering strength and difficulty range, so that the "source code vs artifacts" effect is separated from the filtering effect?
  3. What is the measured cost per retained task (agent tokens, wall-clock, GPU hours)? How does it scale with repository size?
  4. How much of the reported gap depends on CodeMidasVal checkpoint selection? What happens at a fixed step count or a shared, distribution-independent validation set?
1 comment
AuthorsOpenReview ↗

Following Up on Our Response

Dear Reviewer L7dP,

Thank you again for your positive assessment and constructive suggestions. We would greatly appreciate your feedback on whether these additions address your concerns, or if any substantial issues remain.

Best regards,

The Authors

Reviewer_84GROpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The authors of this work propose an agentic pipeline called CodeMidas to convert existing codebases into RL environments for training and improving coding agents. The key aspect of the pipeline is that it creates environments not from development artifacts like commits, pull requests etc. but directly from the source code itself, in turn providing better coverage of the codebase for agents to explore and solve the given behavioral task expectations. With a resulting set of 5,545 training tasks used to train the MiMO-based agent, an improvement across external benchmarks, namely, SWE-bench Pro, DeepSWE, ProgramBench and TerminalBench was observed. Further the improvement observed was more than that of the agents trained on tasks derived from Public-PRs.

Critical strengths
  • The generated tasks span a wide range of programming languages and technicality
  • The pipeline’s strength is that it simply acts on input codebases and thus can be used for ones which lack sufficient commit histories, PRs
  • Performance gains on external benchmarks for agents trained on CodeMidas is more than those trained on PR-derived tasks.
  • Stages of the pipeline are clearly explained with intuitive reasoning of why each is essential.
Critical weaknesses
  • W1. Limited Novelty. Key contributions of using agents for exploration during task construction, broader interface scope (CLI tools, stateful APIs), more languages, and post-rollout filtering are incremental to existing task-creation ideologies in Table-1. For instance, R2E-Gym also extracts functions/methods from the repository, generates equivalence tests for them by relying on the ground truth implementation to get the expected outputs, CodeMidas’s removing existing code and ask the agent to rebuild it and execution-grounded test construction are common ways that have been explored in the prior-art. Fail-to-pass validation also comes from SWE-bench. What is the fundamental novelty in task construction that CodeMidas is bringing that existing works have not considered?

  • W2. Claim regarding task diversity is overstated. The resulting training tasks are diverse in terms of programing languages and technical domains but objective-wise each task seemingly requires agent to re-implement the removed functionality. Many other SWE tasks including but not limited to debugging, error localization, code refactoring, performance gauging etc. are not covered by pipeline.

  • W3. Missing empirical validation of why each filtering stage is required. While the performance of agent trained on 8k unfiltered tasks was reported, agent’s performance if trained on tasks filtered by each stage separately – namely, environment cleaning, execution consistency checks and post-rollout filtering is missing. Can you isolate the contribution of each stage empirically?

  • W4. Missing generalization guarantees beyond MiMo model family. Popular codebases often serve as pre-training data for language models. Can you report if the performance improvements hold for models belonging to other families like deepseek, llama etc.?

  • W5. When evaluated on SWE-Bench Pro, the high-quality data trained agent beats vanilla 8K by 0.6 pt. This improvement seems statistically insignificant.

  • W6. Missing per-codebase per-task cost spent by CodeMidas in terms of agentic compute like token count, LLM calls etc. Without such cost analysis, it is difficult to determine how reasonable the pipeline is for creating new training tasks.

  • W7. How are you determining if the codebase that you start with is high-quality? It is likely the reference behavior is buggy by virtue of bugs present in the starting codebases which the agent is asked to recover during training since assertions use reference execution to establish expected values.

Questions for authors
  • What is the effective batch size for training? It seems with batch size of 32 and reported performance for checkpoint after 70 training steps, agent has only seen less than half of the training task pool.
  • For others, see weaknesses section.
3 comments
AuthorsOpenReview ↗

Following Up on Our Response

Dear Reviewer 84GR,

Thank you again for your detailed review. We would greatly appreciate your feedback on whether these additions address your concerns, or if any substantial issues remain.

Best regards,

The Authors

I thank the authors for taking the time and effort to address my concerns. Their rebuttal has addressed majority of my concerns, including but not limited to the most critical ones - justifying how they are novel, empirically justifying why training with their tasks results in better models than the existing work, empirically justifying why each stage in the pipeline exists and finally the cost associated with creating such tasks using agentic approach. This has been really useful and I would encourage the authors to update the manuscript to include all these crucial details. I have updated my score accordingly. Thanks.

Thank you for your thoughtful engagement. We are greatly encouraged by your positive feedback. As you suggested, we have updated the manuscript to include all these details, highlighted in blue in the uploaded revision.

Comments on the paper

1 comment

Dear Area Chair and Reviewers,

We sincerely thank the Area Chair and all reviewers for their time and constructive feedback. We are encouraged that reviewers appreciate the core idea of CodeMidas: turning source code itself, without relying on issues, PRs, or commits, into verified RL environments for coding agents. Reviewers highlighted several key strengths:

  • Tasks built from source code alone, applicable to codebases that lack sufficient commit histories or PRs and spanning a wide range of languages and domains (84GR).
  • Verifier quality treated as a first-class problem, with checks beyond reference execution that are "more rigorous than the common 'reference solution passes' criterion" (L7dP, sqf1).
  • Transfer to external benchmarks spanning issue repair, program construction, translation, and terminal work, from a strong initial policy (84GR, L7dP, sqf1).

Following the reviewers' suggestions, we provided additional analyses that further support our findings. We are particularly encouraged that Reviewer 84GR found that our rebuttal "has addressed majority of my concerns, including but not limited to the most critical ones" and updated the score accordingly. We have incorporated these analyses into the revision (highlighted in blue):

  • We clarify how CodeMidas relates to prior pipelines in task scope and verifier validation (Section 1; Appendix B.7);
  • We show that the advantage over PR-derived training persists under the same filtering, at fixed steps, and at matched generated tokens (Sections 1 and 4.2; Appendix C.1);
  • We add a stage-wise ablation of the pipeline (Section 5.1; Table R3) and report the construction cost (Appendix A.8);
  • We extend the results to Qwen3.6-27B (Section 4.2; Appendix D.1) and further detail outcome filtering and the capabilities the tasks exercise (Sections 3.4 and 3.5).

Thank you all again for helping us strengthen our manuscript! We are happy to address any further questions.

Sincerely,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.