acceptodds
Under review as a conference paper at ICLR 2027

OpenMLE: Training LLMs for Evolutionary Search in Machine Learning Engineering

Abstract

Large language models have progressed from conversational assistants to reasoners and autonomous agents, and are now entering a stage in which they contribute to discovery itself, including the development of AI: AI that trains AI. Frontier labs have begun pursuing automated AI research, yet the open community lacks the training data and testbeds needed to study it: existing machine learning en- gineering (MLE) resources contain at most a few hundred tasks and are built for evaluation rather than training. We present OpenMLE, an open full-stack frame- work for AI-train-AI in executable MLE. First, OpenMLE-Gym provides, to our knowledge, the first large-scale AI-train-AI testbed: 5,758 quality-gated exe- cutable tasks spanning broad machine learning problems, with isolated execution, task-specific evaluators, and verified evolutionary trajectories. Second, OpenMLE-Train post-trains a single model for the discovery loop, where an agent repeatedly drafts, improves, debugs, and recombines solutions under delayed, noisy feedback: execution-verified SFT, then RL on per-task calibrated execution rewards, teaches it to serve as the shared program-revision operators. Third, OpenMLE-Evo deploys the trained model as the variation engine of long-horizon evolutionary search. On MLE-Bench Lite, post-training Qwen3.6-35B-A3B into OpenMLE-35B raises Medal Average from 39.39% to 60.61% with the harness fixed. With the model fixed, OpenMLE-EVO raises it from 53.03% under original AIRA-Evo to 60.61%. Adding benchmark-disjoint experience priors and asynchronous parallel search to OpenMLE-EVO, at the same total sandbox budget, brings the system to 71.21% with 3B active parameters, on par with GPT-5.5 with Codex (68.18%) under the same budget. Results on held-out NatureBench Lite show transfer to scientific reproduction tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
machine learning engineering agents, evolutionary search, execution-grounded reinforcement learning, LLM agents, code generation, AutoML, AI for AI

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_qSqMOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents OpenMLE, a full-stack framework for improving LLMs' capabilities on MLE tasks. It has three parts:

  1. OpenMLE-Gym: a collection of 5,758 executable, quality-checked MLE tasks.
  2. OpenMLE-Train: a post-training recipe that teaches a model four program-editing operators (draft, improve, debug, crossover) through execution-verified SFT followed by RL.
  3. OpenMLE-Evo: a redesigned AIRA-Evo-style search harness that uses structured experience records to guide parent selection and memory.

On MLE-Bench Lite, both post-training and the new harness improve results. The enhanced system improves a model with 3B active parameters to a level comparable with frontier proprietary agents under the same compute budget, and preliminary results also suggest generalizability to other modalities and scientific tasks.

Critical strengths
  1. The release is a substantial contribution to the open research community. The task corpus is much larger than existing executable MLE resources, and releasing tasks, sandbox, training trajectories, code, and weights together is rare in this area. This gives the community a valuable foundation for studying how to train models for iterative ML engineering rather than only evaluating them.

  2. The experiments are carefully designed and also broad. For example, the authors separate the effect of the model from the effect of the harness by holding one fixed while varying the other, replicate the post-training gain on a second backbone, and evaluate many models under a common protocol. The analysis showing that the new harness achieves its gains with far fewer tokens is also a useful empirical finding.

Critical weaknesses
  1. The evaluation is a bit noisy to support several claims made by the authors. MLE-Bench Lite has only 22 tasks, and with three runs the standard deviation of Medal Average is about 7 to 9 points (according to Table 2). Several reported gains are within that range, including each component of the harness ablation and the comparison with frontier proprietary agents, whose numbers come from single runs. Another concern is that the selected program is run three times on the test split and the best result is kept, which might optimistically inflate reported scores, and it is unclear whether the external baselines received the same treatment.

  2. The paper offers limited insights into which design choices matter. Most components adapt existing techniques, so the value lies largely in the overall engineering and the release. However, probably due to budget constraint the training recipe is not ablated: for instance, there is no comparison of the reward processing, progress bonus, or operator mix against simpler alternatives. Also, about 75% of the SFT data is for the draft operator, so it is unclear how much the model actually learns to improve, debug, and recombine existing solutions.

Questions for authors
  1. OpenMLE-Evo-Max adds experience priors, lessons distilled from public Kaggle competition materials and given to the search before it starts. What exactly are these materials? Beyond exact name matching, did you check for dataset-level or near-duplicate overlap between MLE-Bench and either the OpenMLE-Gym training tasks or the sources of these "experience priors"?
6 comments
AuthorsOpenReview ↗

Response to Reviewer qSqM (1/3)

We thank the reviewer for the detailed review and the concrete points on statistics and contamination. We agree with both main concerns. Below we (i) clarify the statistical support using Human Rank, (ii) quantify the effect of repeated test execution (with the mean of three executions instead of the best, OpenMLE-35B with OpenMLE-Evo scores 56.06% rather than 60.61%), (iii) add component-level evidence on the training recipe and on each operator, and (iv) describe the experience priors and our decontamination procedure in detail.

W1. Noise in the evaluation

We agree that Medal Average over the 22 MLE-Bench Lite tasks (Chan et al., 2024) is noisy: a medal is a binary outcome per task, so the metric is sparse and varies considerably across runs. For this reason we also report Human Rank, the fraction of human leaderboard participants that a submission surpasses, which is continuous and has much lower variance (Table 13). The post-training gains are far larger than this noise on both backbones: Human Rank rises from 0.5828 ± 0.0278 to 0.7647 ± 0.0376 on Qwen3.6-35B-A3B and from 0.5573 ± 0.0074 to 0.7055 ± 0.0505 on Qwen3-30B-A3B.

We agree that the Medal Average differences between ablation rows are within run-to-run noise. In Human Rank, the three search mechanisms each improve the result (Table 2): adding on-demand memory to the selection-only configuration raises Human Rank from 0.6986 ± 0.0370 to 0.7647 ± 0.0376; adding guided selection to the memory-only configuration raises it from 0.7168 ± 0.0263 to 0.7647; and adding asynchronous search raises it from 0.7283 ± 0.0103 to 0.8126 ± 0.0388. Each increase is larger than the run-to-run standard deviation of both configurations and agrees in direction with Medal Average; with three runs each, we treat these as consistent evidence, not individually significant results. The exception is the experience priors, which raise Medal Average slightly (60.61% → 62.12%) but lower Human Rank (0.7647 → 0.7283); we will state this explicitly in the revised text.

W1. Comparison with frontier proprietary agents

We describe OpenMLE-35B with OpenMLE-Evo-Max as on par with GPT-5.5 + Codex and do not claim that it is better. Human Rank supports this reading: our system reaches 0.8126 ± 0.0388 over three runs, and GPT-5.5 + Codex reaches 0.7833, about 0.76 standard deviations below our mean (Table 12). The frontier references are single runs because evaluating proprietary models costs considerably more, as noted in Appendix E.2; within our budget we ran each of them once.

W1. Repeated test execution and treatment of external baselines

You are right that keeping the best of three executions inflates the reported scores somewhat: model training inside the program is stochastic, so the same program scores slightly differently across executions. We recomputed OpenMLE-35B with OpenMLE-Evo using the mean over the three executions of all 66 task-runs:

OpenMLE-35B with OpenMLE-EvoMedal AverageHuman Rank
Best of three executions (reported)60.61%0.7647
Mean of three executions56.06%0.7416

The difference (4.5 points of Medal Average and 0.023 of Human Rank) does not change our conclusions: it is much smaller than the post-training gain measured under a single protocol (21.2 points and 0.182 with best of three). In 6 of the 66 task-runs (9.1%) the three executions differ in Human Rank by at least 0.10, and in 6 a medal is gained or lost. The revised PDF will add the mean-over-executions statistics for our models to the appendix.

Why three executions. Search and final selection never use test data: the program is selected by validation score alone, and that single program is then executed on the test package. We executed it three times because, in our shared evaluation environment, hardware and platform faults can make a correct program fail, which would heavily underestimate its Human Rank and medal. Two examples from our logs:

TaskHuman Rank of the three executionsCause of the failed execution
aerial-cactus-identification0.9992 / 0.8763 / 0.0000CUDA out of memory in the third execution (21.38 MiB free of 23.52 GiB)
detecting-insults-in-social-commentary0.0000 / 0.9800 / 0.9800OpenSandbox HTTP 500/502 platform error in the first execution

The Codex and Claude Code references were executed once. We manually inspected every failed reference submission and found none caused by hardware faults, so their scores were not affected by such faults.

References

  • Chan et al. (2024). MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. arXiv:2410.07095.
AuthorsOpenReview ↗

Response to Reviewer qSqM (2/3)

(Response to Reviewer qSqM, part 2 of 3.)

W2. Insights beyond the release

We agree that most components build on prior techniques: MLE-Smith-style task construction (Qiang et al., 2025), GSPO (Zheng et al., 2025) with an entropic advantage (Yuksekgonul et al., 2026), and an AIRA-Evo population loop (Toledo et al., 2025). What the stack adds is the ability to change one part while holding the others fixed, and the paper reports four findings from such comparisons: (1) at a fixed training budget, adding executable training tasks keeps improving the trained agent (Figure 6, both backbones); (2) SFT and RL each add to the result on two backbones (Table 1); (3) the harness gain comes from spending fewer tokens per node, not from evaluating more nodes: OpenMLE-Evo evaluates 12.4% fewer nodes with 42% fewer tokens, and the share of Improve calls that set a new best roughly doubles (Appendix E.3); and (4) training and search gains are complementary: OpenMLE-35B benefits from the improved harness, and the harness benefits the four other models we paired it with (Figure 7). The analyses below on the training signal and on individual operators add evidence at the component level.

W2. Ablating the training recipe

As you note, a full 200-step RL run costs about 2.5 × 10⁴ sandbox GPU-hours, so the submission ablates the recipe only at the stage level (SFT vs. SFT+RL, Table 1). We add component-level evidence from an offline analysis of the logged RL rollout groups of the released OpenMLE-35B; each group consists of 16 candidates that share the same task, operator, parent context, and prompt.

Reward processing. Compared with static leaderboard bounds, the task-adaptive bounds of Appendix C.4 increase the mean within-group reward range from 0.682 to 0.742 (+8.9%), so they separate candidates within a group more clearly.

Operator (candidates that beat their parent)CountMean reward, λ = 0Mean reward, λ = 0.5
Improve1,1530.5540.712
Debug1,4090.4150.622
Crossover1,0160.5980.696

Progress bonus. Holding the candidates, groups, parents, and base reward fixed and changing only the bonus coefficient, λ = 0.5 raises the mean reward of candidates that beat their parent for all three revision operators (table above), so verified improvements receive more credit.

Advantage. The mean of the largest within-group advantage is 6.04 with the entropic advantage, compared with 1.90 with the GRPO z-score (Shao et al., 2024), so the best candidate in each group receives a much larger update.

These numbers measure the learning signal each design produces, not its effect on final performance. Full-scale ablations remain future work, and the revised PDF will state this in Appendix C.4.

W2. Does the model learn to improve, debug, and recombine?

74% of the SFT examples are Draft (19,436 of 26,259), mainly because the parallel path collects complete solutions. Revision is trained mostly in the RL stage: RL samples Improve, Debug, and Crossover for half of all rollout groups (0.17, 0.17, and 0.16; Table 7), and RL contributes 12.1 of the 21.2 points on Qwen3.6-35B-A3B (48.48% → 60.61%; Table 1). To check this directly, we tracked the validation reward of each operator throughout RL on the held-out Gym validation set of Figure 15 (mean of the first and of the last three validation points):

OperatorValidation reward, start of RLValidation reward, end of RL
Draft0.1980.289
Improve0.1380.356
Debug0.2020.360
Crossover0.1300.297

All three revision operators improve, and by more than Draft. The search logs show the same trend (see our response to Reviewer P8LV): from the base model to SFT to OpenMLE-35B, the share of Improve calls that beat their parent rises from 17.7% to 30.4% to 32.4%, the share of Crossover calls that beat the stronger parent from 11.7% to 20.7% to 28.8%, and the share of Debug chains repaired within ten calls from 91.8% to 94.2% to 95.2%. The revised PDF will add the per-operator validation curves to Appendix C.4.

References

  • Qiang et al. (2025). MLE-Smith: Scaling MLE Tasks with Automated Multi-Agent Pipeline. arXiv:2510.07307.
  • Shao et al. (2024). DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models. arXiv:2402.03300.
  • Toledo et al. (2025). AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench. arXiv:2507.02554.
  • Yuksekgonul et al. (2026). Learning to Discover at Test Time. arXiv:2601.16175.
  • Zheng et al. (2025). Group Sequence Policy Optimization. arXiv:2507.18071.
AuthorsOpenReview ↗

Response to Reviewer qSqM (3/3)

(Response to Reviewer qSqM, part 3 of 3.)

Q. What the experience priors are

The experience priors are text guidance given to OpenMLE-Evo-Max before search starts, in three parts, all included in the supplementary code (skills/): (1) twelve category playbooks, one per modality × task type (e.g., image classification, tabular regression, text sequence-to-sequence), listing task routing, validation schemes, model families, and common failure modes; (2) one task skill per target task, generated offline from the task's public description, the matching playbooks, and the execution constraints, without web access and without any data, code, or solution from the target competition; and (3) one shared note on common runtime and submission-format errors. The playbooks were distilled from public Kaggle notebooks and write-ups of competitions outside MLE-Bench; the next answer describes how MLE-Bench material was excluded.

The priors are used only in the Evo-Max rows. Every post-training and harness result in Sections 5.1 and 5.2 is obtained without them. In Table 2, adding the priors to OpenMLE-Evo raises Medal Average by 1.5 points but lowers Human Rank (0.7647 → 0.7283), while adding asynchronous parallel search on top raises Medal Average by a further 9.1 points. The revised PDF will describe the priors in full in Appendix E.1.

Q. Overlap beyond exact name matching

Yes, at the dataset level, for both the training tasks and the priors. For OpenMLE-Gym, beyond exact name matching, we also filtered tasks by Kaggle competition ID, which removes different datasets derived from the same competition.

When building the experience library, we excluded at the source all material of every MLE-Bench competition: data, discussions, notebooks, solutions, leaderboards, and write-ups. For the remaining competitions we used only notebooks and write-ups, and we further removed competitions or documents that share a source with, or explicitly refer to, an MLE-Bench competition, based on task names and descriptions. For example, for dogs-vs-cats-redux-kernels-edition we excluded not only the competition with that name but also dog-vs-cat-classification and dogs-vs-cats, which use the same dataset. No web search was allowed during library construction or during the experiments, so the information sources are fixed and auditable.

The revised PDF will describe both procedures in Appendices B.1 and E.1.

We hope these responses address your concerns, and we would be glad to clarify anything further.

Reviewer_P8LVOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper is on improving the MLE task by post-training. It decomposes the framework into task testbed, training recipe, and deployment. The framework aims to train four atomic operators that can be applied across search controllers. The evaluation includes controlled ablations, backbone replication, and scientific-task transfer.

Critical strengths
  • openMLE provides a comprehensive system that includes: (1) training-scale verifiable testbed where each task is an environment instance for MLE development; (2) a post-training recipe on atomic operators for program search; (3) an AI-for-AI long-horizon search framework
  • The experiments includes the results on how post-training (SFT and RL) and evolutionary search (with fixed model) improves the performance on MLE-bench Lite split
Critical weaknesses

please see my questions

Questions for authors
  • What is the difference on task type distribution between OpenMLE-Gym Task and MLE-Bench Lite task? and also the performance on MLE-Bench Lite grouped by task types.
  • Figure 7 is unclear to me. For the OpenMLE-35B part? Is that means post-training in section 5.1 improves the search performance under the same searching policy?
  • The system defines four atomic operators. Are they used in both section 5.1 and 5.2? Currently evaluation reports the general performance across tasks, how can we evaluate the separate contribution on each of them?
5 comments
AuthorsOpenReview ↗

Response to Reviewer P8LV (1/2)

We thank the reviewer for describing OpenMLE as a comprehensive system that spans a training-scale testbed, a post-training recipe, and a long-horizon search framework, and for three concrete questions. In short: (Q1) OpenMLE-Gym is mostly tabular and MLE-Bench Lite mostly image, and Human Rank on Lite rises for every modality and task type; (Q2) Figure 7 varies the harness with the model fixed, and post-training does improve search under a fixed search policy (39.39% → 60.61% Medal Average under OpenMLE-Evo, and 34.85% → 42.42% under Codex); (Q3) all four operators are used in both sections, and training raises the valid rate of every operator. Details and the corresponding manuscript changes follow.

Q1. Task distribution of OpenMLE-Gym and MLE-Bench Lite, and performance by task type

Figure 2 (right) shows the task distribution of OpenMLE-Gym by modality as well as by task type.

The table below shows task distribution of OpenMLE-Gym and MLE-Bench Lite (Chan et al., 2024) by modality; the last column gives the Lite Medal Rate of the base model and of OpenMLE-35B under OpenMLE-Evo (Figure 8).

ModalityOpenMLE-Gym (5,758)MLE-Bench Lite (22)Lite Medal Rate (%), base → OpenMLE-35B
Tabular44%4 (18%)42 → 50
Image18%9 (41%)44 → 52
Time series13%0—
Multimodal11%2 (9%)33 → 83
Text9%5 (23%)33 → 60
Audio2%2 (9%)33 → 100
Video / other1% / 2%0—

The two sets differ mainly in modality. Gym is mostly tabular (44%, plus 13% time series), while Lite is mostly image (41%) and has proportionally more text (23% vs. 9%) and audio (9% vs. 2%). Task types are closer: classification and regression make up 87% of Gym (56% and 31%) and 19 of the 22 Lite tasks (86%), although Lite has more classification and less regression. All 75 MLE-Bench competitions are excluded from Gym (Appendix B.1), so Lite is a held-out set whose modality mix differs from the training distribution.

Grouped by task type, under OpenMLE-Evo:

Task type#Lite tasksMedal Rate (%), base → OpenMLE-35BHuman Rank (×100), base → OpenMLE-35B
Binary classification858.33 → 70.8373.90 → 86.95
Multi-class / ordinal classification638.89 → 77.7867.61 → 90.42
Multi-label classification311.11 → 55.5640.68 → 71.85
Regression250.00 → 50.0048.48 → 64.75
Sequence-to-sequence20.00 → 16.679.10 → 22.90
Image restoration (denoising)133.33 → 0.0048.15 → 53.29
All2239.39 → 60.6158.28 → 76.47

Several task types contain only one or two tasks, and Medal Rate is a sparse metric, so per-type Medal Rates fluctuate strongly; for example, Medal Rate does not increase for the regression tasks and drops for the single denoising task. Human Rank, however, increases for every task type, consistent with the overall result. The revised PDF will add both tables to Appendix B.1 and Appendix E.

Q2. Reading Figure 7

Yes: with the search policy fixed, post-training improves search performance, but this is shown in Section 5.1 rather than in Figure 7. With OpenMLE-Evo fixed, post-training raises Medal Average from 39.39% (base) to 48.48% after SFT and 60.61% after RL (Table 1, Figure 5). The same holds under a different harness: with Codex, post-training raises Medal Average from 34.85% to 42.42% (three runs each; see our response to Reviewer JNiZ).

Figure 7 answers a different question: with the model held fixed, does OpenMLE-Evo outperform original AIRA-Evo (Toledo et al., 2025) and general-purpose coding-agent harnesses? Each group fixes the model and changes only the search harness: the gray bar is a reference harness, the cyan bar is OpenMLE-Evo, and the hatched cap is the further gain from OpenMLE-Evo-Max. For OpenMLE-35B, the reference is original AIRA-Evo rather than a general coding agent, so its group reads 53.03% (original AIRA-Evo) → 60.61% (OpenMLE-Evo) → 71.21% (OpenMLE-Evo-Max). Figure 7 therefore measures the harness, not post-training.

References

  • Chan et al. (2024). MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. arXiv:2410.07095.
  • Toledo et al. (2025). AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench. arXiv:2507.02554.
AuthorsOpenReview ↗

Response to Reviewer P8LV (2/2)

(Response to Reviewer P8LV, part 2 of 2.)

Q3. Are the four operators used in both Sections 5.1 and 5.2?

Yes. Every OpenMLE-Evo and OpenMLE-Evo-Max run in Sections 5.1 and 5.2 uses the same four operators with the same schedule: the first generation consists of Draft calls; later individuals use Crossover with probability 0.5 from the third generation on and Improve otherwise; and a child that fails to execute receives up to ten consecutive Debug calls (Table 10). The original AIRA-Evo uses the same four operators. Training uses the same operators: all four appear in the SFT corpus, and RL samples Draft, Improve, Debug, and Crossover with probabilities 0.50, 0.17, 0.17, and 0.16 (Table 7). The only runs that do not use these operators are the Claude Code and Codex references in Figure 7, which use their own tool-calling loops.

Q3. Contribution of each operator

We analyzed the search logs offline, comparing the base model, the SFT checkpoint (after SFT, before RL), and OpenMLE-35B, all under OpenMLE-Evo with 66 task-runs each. A new best is counted when a node beats the best score found earlier in the same search trajectory.

OperatorValid rate (base / SFT / OpenMLE-35B)Share of callsShare of new-best updates
Draft11.3% / 27.8% / 45.7%10.6% / 11.0% / 8.5%9.5% / 16.8% / 19.1%
Debug31.3% / 39.8% / 48.0%67.0% / 52.8% / 38.4%69.0% / 44.7% / 34.6%
Improve35.6% / 58.1% / 67.4%14.5% / 23.2% / 30.5%16.4% / 29.2% / 28.9%
Crossover42.7% / 63.1% / 76.6%7.9% / 13.1% / 22.6%5.2% / 9.3% / 17.5%
New-best updates (count)116 / 161 / 246

Training raises the valid rate of every operator. Because the model writes correct code more often, Debug takes a shrinking share of calls (67.0% → 38.4%) and of new-best updates (69.0% → 34.6%), while the total number of new-best updates grows from 116 to 246. The search is thus increasingly driven by Draft, Improve, and Crossover rather than by repairs.

MetricBaseSFTOpenMLE-35B
Improve beats its parent17.66%30.43%32.44%
Improve beats the current global best4.90%7.30%6.25%
Crossover beats the stronger parent11.74%20.66%28.84%
Crossover beats the current global best2.82%4.13%5.10%
Debug chains repaired within 10 calls91.83%94.18%95.15%
Mean attempts per successful Debug chain2.712.221.82

Improve and Crossover are meant to produce a child better than its parent(s), and Debug to make a failing program valid. Except for Improve surpassing the global best, every rate rises with training; for OpenMLE-35B, more than a quarter of Improve and Crossover calls beat their (stronger) parent. Debug repairs nearly all failing programs and needs fewer attempts after training. During RL, the held-out validation reward of each operator also rises: Improve from 0.138 to 0.356, Debug from 0.202 to 0.360, Crossover from 0.130 to 0.297, and Draft from 0.198 to 0.289 (details in our response to Reviewer qSqM). The search-log rates are observational measurements, since each model's search produces different parents. The revised PDF will add this analysis to Appendix E.

Summary of the changes we will make in the revised PDF: (i) Appendices B.1 and E will add the two task-distribution tables; (ii) Appendix E will add the per-operator analysis.

We hope these answers make the corresponding parts of the paper easier to follow, and we would be glad to clarify anything further.

AuthorsOpenReview ↗

Revision uploaded — we would appreciate your feedback

Dear Reviewer P8LV,

Thank you again for your review. We have uploaded the revised PDF, with all changes marked in blue, and an updated supplementary archive. The revision adds the task distribution of OpenMLE-Gym and MLE-Bench Lite with results by modality and task type (Appendix E.6), the Codex experiment, which keeps the search policy fixed and changes only the model (Appendix E.2), and a per-operator analysis with per-operator validation curves during RL (Appendix E.7 and Figure 16). We would be glad to know whether our response addresses your concerns. If it does, we hope you will consider updating your assessment; if anything is still unclear, we are happy to clarify before the discussion period ends.

Reviewer_JNiZOpenReview ↗
Rating
4(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces OpenMLE a recipe for training LLMs to better perform an evolutionary search process for Machine Learning Engineering (MLE). They first create OpenMLE-GYM with 5,758 task environments sourced from Kaggle datasets and Kaggle competitions as well as a small seed set of "curated anchors" pulled from papers and benchmarks. They craft a harness that emphasizes a discovery loop where the model drafts, improves, debugs, and recombines solutions. Then they train using SFT and RL in their custom harness with the OpenMLE-GYM tasks. Ultimately they produce a model that achieves performance on par with GPT-5.5 at MLE-Bench Lite but with only 3B active parameters.

Critical strengths

The OpenMLE-GYM contribution feels like a major contribution towards training models to be better at MLE tasks. This reminds me of a SWE-Smith style contribution (I suppose building on top of MLE-Smith), but the scale and diversity of the resource as pictured in Figure 2 is what sold me on the importance of the contribution. If released this can be a major resource for enabling people to train better MLE agents. I think this is a standalone contribution independent of the modeling/harness contribution.

The paper uses reasonable baselines and computes confidence intervals to prove significance. (See Table 2)

The trained model seems like it could be a somewhat useful for MLE tasks although it may have been fit heavily to the custom harness.

The paper is super clear to read (aside from Section 3 where I was finding myself questioning the harness. However the rest of the paper was quite clear. There is also a lot of detail in the appendix which would help with reproduction.

Critical weaknesses

I found myself rereading the harness section a bit trying to understand what the motivation of sampling the operators and parents rather than letting the model do selection over memory itself as in a more general harness. I think I can be convinced this is important especially for small models, but this definitely feels a bit like a custom pattern that may not be necessary with better models. It felt a bit like an adhoc solution that works for optimizable objectives (so you are training a model that can be quite good at hillclimbing) but that may be quite specific to this setting. Overall the harness was the least convincing to me that it would be an important contribution beyond this particular setting, and since the model was heavily trained on this harness (and never evaluated on a general harness) I got the feeling the model was fit to this particular harness/setting too.

I also found Figure 1 and several of the diagram figures to have ai vibes. This isn't necessarily a bad thing outright since the figures are legible and interpretable, but it is always something noticeable. Things like subtext on a figure in gray are giveaways.

Questions for authors

Questions:

  • Do you believe your model is fit to your OpenMLE-EVO harness or would it transfer to other harnesses?
  • How did you pick your atomic actions: "draft, improve, debug, crossover"?

Suggestions:

  • For Figure 6 perhaps use steps on the x-axis instead of fraction of SFT corpus because just by looking at the figure you already know percentage completion. Steps is strictly more informative.
4 comments
AuthorsOpenReview ↗

Response to Reviewer JNiZ (1/2)

We thank the reviewer for the careful reading and for recognizing OpenMLE-Gym as a contribution in its own right. The 5,758-task corpus (complete packages where the source license permits redistribution, and the preparation and metric code that rebuilds the remaining packages), the SFT trajectories, and both post-trained checkpoints are already available at the anonymous repository linked in Appendix F.2, and the method code is in the supplementary material. On your main concern, whether the model is fit to OpenMLE-Evo, we have evaluated it under Codex, a general-purpose coding agent it was never trained in: post-training still raises Medal Average from 34.85% to 42.42% (three runs each; Part 2). Below we answer each point in turn.

W1. Why the controller, not the model, selects the operator and the parent

We made this choice for three reasons, which we will state at the beginning of Section 4 in the revised PDF.

(1) Research context. OpenMLE follows the evolutionary-search line of AIRA-dojo (Toledo et al., 2025), AlphaEvolve (Novikov et al., 2025), and OpenEvolve (Sharma, 2025), in which a controller maintains the population and the language model serves as the variation operator. Our aim is to improve how well the model evolves code within this framework. In OpenMLE-Evo, what we change is how the controller uses execution evidence (structured records, three-factor parent selection, and operator-conditioned memory); within each call, the model still decides what to change and why.

(2) A clearer training target. Keeping the two roles separate lets training focus on one capability, revising code, and this capability carries over across parent-selection strategies. If the model also selected parents, it would have to learn a second skill, a search policy rewarded by the best score at the end of a 12-hour run, alongside program revision, which is rewarded by the execution of each call. The two objectives receive signals on different time scales, and training them jointly would make it hard to tell which one an improvement came from. As you suggest, this matters most for smaller models.

(3) Letting the model make these decisions remains an open problem. The closest evidence we know of concerns parent selection. In Hyperagents (Zhang et al., 2026, Appendix E.5), a Claude Sonnet 4.5 meta-agent was allowed to rewrite its own parent-selection mechanism, starting from random selection. On held-out tasks it reached 0.491, above random selection (0.428) but below the handcrafted rule (0.532), and the authors kept parent selection fixed in all of their main experiments. The Darwin Gödel Machine (Zhang et al., 2025) likewise keeps parent selection fixed and leaves making it modifiable to future work. We consider learned control a direction worth pursuing, but in this work we keep it fixed so that training targets a single skill whose effect we can measure.

W1. Scope: is the harness specific to small models and hill-climbing objectives?

We agree that OpenMLE-Evo is designed for problems with an executable, scalar evaluator; the revised PDF will state this scope in Section 4. This class is broad, however: it covers MLE, AlphaEvolve-style algorithm discovery, and the scientific-reproduction tasks of NatureBench (Wang et al., 2026), where each task has its own paper-specific metric. On NatureBench, with the base model fixed, replacing original AIRA-Evo by OpenMLE-Evo raises Match-SOTA from 2/10 to 5/10 (Appendix E.5).

Within this class, the structured loop is not only a crutch for small models. Under the same sandbox and 12-hour budget, replacing Claude Code or Codex by OpenMLE-Evo raises Medal Average for all four larger open models we tested: Kimi K2.6 from 59.09% to 66.67%, GLM-5.2 from 59.09% to 62.12%, MiniMax M3 from 54.55% to 59.09%, and MiniMax M2.7 from 45.45% to 50.00% (Figure 7; the general-harness references are single runs). We did not run GPT-5.5 or GPT-5.6 Sol inside OpenMLE-Evo, so we make no claim about the strongest closed models.

References

  • Novikov et al. (2025). AlphaEvolve: A Coding Agent for Scientific and Algorithmic Discovery. arXiv:2506.13131.
  • Sharma (2025). OpenEvolve: An Open-Source Implementation of AlphaEvolve. https://github.com/codelion/openevolve.
  • Toledo et al. (2025). AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench. arXiv:2507.02554.
  • Wang et al. (2026). NatureBench: Can Coding Agents Match the Published SOTA of Nature-Family Papers? arXiv:2606.24530.
  • Zhang et al. (2025). Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents. arXiv:2505.22954.
  • Zhang et al. (2026). Hyperagents. arXiv:2603.19461.
AuthorsOpenReview ↗

Response to Reviewer JNiZ (2/2)

(Response to Reviewer JNiZ, part 2 of 2.)

W1 / Q1. Is the model fit to OpenMLE-Evo, and does it transfer to other harnesses?

In short, the post-training gain carries over to a general-purpose agent, although it is about a third as large there (7.6 vs. 21.2 points of Medal Average). We have now evaluated both the base model and OpenMLE-35B under Codex, a general-purpose coding agent that runs the model in its own multi-turn tool loop with its own prompts and context management, and uses none of OpenMLE-Evo's operators, parent selection, or memory. The protocol matches our other runs: the same sandbox, the 22 MLE-Bench Lite tasks (Chan et al., 2024), a 12-hour per-task budget, and three runs per model, reported as mean ± population standard deviation as in Table 13.

Model (under Codex)Medal AverageHuman Rank
Qwen3.6-35B-A3B (base)34.85% ± 2.14%0.5205 ± 0.0316
OpenMLE-35B42.42% ± 2.14%0.6089 ± 0.0183

Post-training therefore improves the model under a harness it was never trained in: Medal Average rises by 7.6 points and Human Rank by 0.088. The gain is smaller than under OpenMLE-Evo (39.39% → 60.61%). We attribute this to two factors: both training stages use single calls that each return a complete program rather than multi-turn tool use, and the structured search loop itself adds to the result (Figure 7). The revised PDF will report this experiment in Appendix E.2.

W2. Figures

Thank you for the comment, and for noting that the figures remain legible. We take the point on their appearance and may simplify them in a future version, for example by removing the gray annotation text you pointed out.

Q2. How we chose the four operators

Draft, Improve, Debug, and Crossover are the four whole-program transitions that recur in executable search for ML code: starting a solution, refining a working one, repairing a failing one, and recombining two. Draft, Improve, and Debug are the operators of AIDE (Jiang et al., 2025); Crossover is the recombination step of AIRA-dojo's evolutionary solver (Toledo et al., 2025) and of evolutionary search in general. Because every call emits a complete program, a single operator can change preprocessing, features, model, validation, or ensembling, so we did not need a separate action for each MLE subtask. A small vocabulary also gives one typed interface across SFT data, RL rollouts, and search (Table 6). We do not claim the set is complete: multi-file, tool-using, or long-running workflows may need additional actions.

Our search logs show that each operator fulfils its role in OpenMLE-35B runs: Improve children beat their parent in 32.4% of calls and Crossover children beat the stronger parent in 28.8%, and 95.2% of Debug chains turn a failing program into a valid one within ten calls. Draft, Improve, and Crossover together produce 65.4% of new-best updates. Our response to Reviewer P8LV gives the full per-operator analysis.

Suggestion. The x-axis of Figure 6

Thank you for the suggestion; it showed us that the figure did not make one point clear. Every checkpoint in Figure 6 is trained for the same number of optimizer steps at the same batch size (Appendix C.2), so training steps are constant along the x-axis. What varies is how much distinct data those steps cover, from 12.5% of the SFT tasks to all of them, with smaller subsets revisited more often. The figure therefore measures data scaling at a fixed training budget, not training progress. In the revised PDF, the x-axis will show the percentage of SFT tasks used for training (0% being the untrained backbone), and the caption will state that every checkpoint is trained for the same number of optimizer steps.

Summary of the changes we will make in the revised PDF: (i) Section 4 will open with the design rationale above and state the scope of OpenMLE-Evo; (ii) Appendix E.2 will add the Codex experiment; (iii) Figure 6 will label its x-axis as the percentage of SFT tasks, and its caption will state that every checkpoint uses the same number of optimizer steps.

We hope these responses address your concerns, and we would be glad to clarify anything further.

References

  • Chan et al. (2024). MLE-bench: Evaluating Machine Learning Agents on Machine Learning Engineering. arXiv:2410.07095.
  • Jiang et al. (2025). AIDE: AI-Driven Exploration in the Space of Code. arXiv:2502.13138.
  • Toledo et al. (2025). AI Research Agents for Machine Learning: Search, Exploration, and Generalization in MLE-bench. arXiv:2507.02554.
AuthorsOpenReview ↗

Revision uploaded — we would appreciate your feedback

Dear Reviewer JNiZ,

Thank you again for your review. We have uploaded the revised PDF, with all changes marked in blue, and an updated supplementary archive. The revision adds the Codex experiment, where post-training still raises Medal Average from 34.85% to 42.42% under a general-purpose agent (Appendix E.2), opens Section 4 with the rationale for letting the controller choose operators and parents and with the scope of the harness, and relabels the x-axis of Figure 6 as the percentage of SFT tasks at a fixed number of optimizer steps. We would be glad to know whether our response addresses your concerns. If it does, we hope you will consider updating your assessment; if anything is still unclear, we are happy to clarify before the discussion period ends.

AuthorsOpenReview ↗

Follow-up before the discussion closes

Dear Reviewer JNiZ,

As the author–reviewer discussion closes tomorrow, we would like to follow up once more on your comments. In brief: under Codex, a general-purpose coding agent the model was never trained in, post-training still raises Medal Average from 34.85% to 42.42% and Human Rank from 0.5205 to 0.6089 (Appendix E.2); Section 4 now opens with why the controller chooses operators and parents and states the scope of the harness; our response explains how we chose the four operators, with search-log evidence that each fulfils its role (Appendix E.7); and Figure 6 now labels its x-axis as the percentage of SFT tasks at a fixed number of optimizer steps. Your review also noted that OpenMLE-Gym could be a major resource if released: the 5,758-task corpus, the SFT trajectories, and both checkpoints are already available at the anonymous repository (Appendix F.2).

If these answers resolve your concerns, we would be grateful if you would consider raising your score. If something still keeps the paper at its current rating, we would very much like to know, so that we can address it in the remaining time.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all three reviewers for their constructive reviews. We were glad that the reviewers found OpenMLE-Gym a contribution in its own right (JNiZ, qSqM), the experiments "carefully designed and also broad" (qSqM), and the release of tasks, trajectories, code, and weights rare in this area (qSqM). Below we summarize the new evidence added during the discussion. Each point is detailed in the individual responses, and each listed change will appear in the revised PDF.

Transfer beyond our harness (JNiZ, P8LV). Under Codex, a general-purpose coding agent that uses none of OpenMLE-Evo's operators, parent selection, or memory, post-training raises Medal Average from 34.85% ± 2.14% to 42.42% ± 2.14% and Human Rank from 0.5205 to 0.6089 (three runs each). The gain is smaller than under OpenMLE-Evo (39.39% → 60.61%), and we report it as such.

Repeated test execution (qSqM). Scoring OpenMLE-35B with OpenMLE-Evo by the mean of its three test executions instead of the best gives 56.06% Medal Average and 0.7416 Human Rank, against 60.61% and 0.7647 as reported, so the conclusions do not change. The revised appendix will report mean-over-executions statistics for our models.

Statistical support (qSqM). Human Rank, which is continuous and varies much less across runs than Medal Average, shows post-training gains on both backbones far beyond run-to-run noise. In Table 2, adding each of the three search mechanisms (guided selection, on-demand memory, and asynchronous search) raises Human Rank by more than the run-to-run standard deviation of both configurations, in the same direction as Medal Average; with three runs per configuration, we present these as consistent evidence rather than individually significant results. The experience priors are the exception: they raise Medal Average slightly but lower Human Rank (0.7647 → 0.7283), which the revised text will state.

What training changes in each operator (P8LV, qSqM, JNiZ). In search logs of 66 task-runs per model, from base to SFT to OpenMLE-35B the share of Improve calls that beat their parent rises from 17.7% to 30.4% to 32.4%, Crossover calls that beat the stronger parent from 11.7% to 20.7% to 28.8%, and new-best updates from 116 to 246. During RL, the held-out validation reward of Draft Improve, Debug, and Crossover rises notably.

Training-recipe signal (qSqM). An offline analysis of the logged RL rollout groups shows that task-adaptive bounds widen the within-group reward range, the progress bonus gives more credit to candidates that beat their parent, and the entropic advantage gives the best candidate a much larger update. We present this as evidence about the learning signal, not as full ablations, and the revised Appendix C.4 will say so.

Task distribution (P8LV). OpenMLE-Gym is mostly tabular and MLE-Bench Lite mostly image. Human Rank on Lite rises for every modality and task type.

Experience priors and decontamination (qSqM). We describe the priors in full and explain how MLE-Bench material was excluded at the source from both the priors and OpenMLE-Gym. The priors are used only in the Evo-Max rows.

Changes we will make in the revised PDF: Section 4 will open with the design rationale and state the scope of OpenMLE-Evo; Figure 6 will label its x-axis as the percentage of SFT tasks used; and the appendix will add the Codex experiment (E.2), mean-over-executions statistics, the task-distribution tables (B.1, E), the per-operator analysis (E), the per-operator validation curves (C.4), and the priors and decontamination procedures (B.1, E.1).

We thank the reviewers again for their time, and we would be glad to clarify anything further during the discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.