acceptodds
Under review as a conference paper at ICLR 2027

M1: Towards a Generalist Agentic Model for Autonomous Machine Learning Engineering

Abstract

Large language models (LLMs) are expanding from assisting scientific research and engineering to autonomously developing and improving AI systems, advancing AI for AI (AI4AI) as a pathway toward recursive self-improvement (RSI). Workflow-driven machine learning engineering (MLE) systems provide a practical path toward this pursuit through complex orchestration that manages long-horizon experimentation, while recent MLE systems adopt model-directed interaction, allowing models to autonomously organize experiments and adapt optimization strategies based on execution feedback. However, feedback from each experiment reflects several preceding decisions, and a long MLE run contains many such costly experiments. Training must therefore select useful supervision from long interaction trajectories and learn from intermediate outcomes as they become available. To address these challenges, we propose a post-training method for autonomous long-horizon MLE optimization and apply it to train M1, a 35B Mixture-of-Experts agentic model. We design the Kaggle-MLE Factory to construct executable tasks and an interaction harness for model-directed experimentation. Hierarchical agentic SFT transfers general tool-use and coding capabilities to MLE through outcome-guided supervision. Windowed Interaction RL (WIRL) treats the multi-turn decisions between successive experiment results as an execution window and assigns it a shared outcome-based reward. Asynchronous collection and policy correction support policy updates before the trajectory ends. On MLE-Bench Lite, M1 improves the average medal rate of the base model from 37.9% to 72.7%, achieving performance comparable to larger frontier models such as Kimi-K3 and GLM-5.3. Further experiments show that M1 improves performance on NatureBench Lite, agent scaffold optimization, and algorithm optimization, suggesting that the learned optimization capability generalizes to other tasks.

Primary area:
foundation or frontier models, including LLMs
Keywords:
machine learning engineering, LLM agents, long-horizon agents, agentic post-training, supervised fine-tuning, reinforcement learning, AI4AI

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_SQ4jOpenReview ↗
Rating
3
Confidence
4
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents M1, an MLE agent trained through general and domain-specific SFT followed by Windowed Interaction RL (WIRL). It introduces a task construction pipeline and an interaction harness, and evaluates the trained model on MLE and related optimization tasks.

Critical strengths
  1. The performance gains over the base model are substantial.

  2. Evaluation under both the proposed harness and Claude Code provides useful evidence of transfer.

  3. Comparisons with RL baselines and ablations of trend rewards and policy correction help assess the training method.

Critical weaknesses
  1. The connection between window-level optimization and long-horizon decision making is insufficiently established. WIRL assigns a shared reward based on the current execution outcome to decisions within each window. This enables training from intermediate feedback, but it is unclear how it credits exploratory actions whose benefits emerge only in later windows.

  2. The methodological novelty is not sufficiently established. While the empirical gains are promising, the paper reads more like a technical report describing an effective training pipeline than a research paper establishing a distinct methodological advance. The combination of a custom harness, staged SFT, and RL involves substantial data collection and training effort, but the paper does not convincingly establish what new, generalizable insight the proposed recipe provides. In particular, it remains unclear how much of the improvement depends on the distinctive methodological choices rather than the overall investment in task-specific data and post-training. My concern is therefore not simply the absence of additional ablations, but the limited justification of the method’s novelty beyond a successful integration of existing ideas.

  3. The need for the custom harness is unclear. The paper does not clearly explain what essential capability the M1 harness provides beyond a general-purpose coding harness such as Claude Code, potentially augmented with context management. Could comparable or better results be obtained by collecting trajectories and performing the same post-training directly within that environment? Evaluating the trained M1 model in Claude Code demonstrates transfer, but does not answer this training-time question. Without a clearer account of which limitations of existing harnesses necessitate the proposed design, it is difficult to assess the custom harness as a research contribution.

Questions for authors

See weaknesses. The concerns above are substantive and could change my recommendation.

4 comments
AuthorsOpenReview ↗

Official Comment to Reviewer SQ4j (Part 1)

[W1] Connection between window-level optimization and long-horizon decision making, and credit assignment for exploratory actions.

We thank the reviewer for raising this question. We have made the following additions to the revised manuscript:

  • Section 4.2: We add a paragraph entitled "Within-window credit assignment and cross-window continuity" to clarify how window-level optimization supports long-horizon decision-making.
  • Section 7: We identify multi-window return estimation and long-horizon value modeling as directions for future work.

WIRL decomposes long-horizon optimization into decision units bounded by experimental feedback and uses reward design and interaction history to align optimization across windows with the objective of the entire trajectory.

(1) Window boundaries align with the feedback structure of long-horizon MLE. In long-horizon MLE, the environment provides verifiable feedback only when an experiment finishes, and each experimental outcome depends on multiple preceding decisions involving analysis, modifications, and configuration. WIRL groups all decisions between two consecutive experimental outcomes into an execution window and assigns the outcome of the corresponding experiment as a shared reward. Exploratory decisions such as data analysis, feature exploration, and hypothesis testing typically occur before the next experiment is launched. They therefore belong to the same window as the experiment they support and receive credit from its outcome.

(2) Window rewards are aligned with the final objective. Window rewards are primarily determined by the absolute quality of the solution, using the same task-specific medal thresholds as the final evaluation. The reward tiers are −1.0, −0.6, −0.4, 1.0, and 2.0, while the local trend term TmT_m contributes only ±0.2 to reflect progress relative to the previous valid submission. Since the final submission is selected from the candidate solutions produced during experimentation, improving solution quality within individual windows is consistent with improving the final outcome. The trend term has a small magnitude: within the same quality tier, a temporary score decrease receives a reward only 0.2 lower than an unchanged score. Furthermore, executing an experiment does not change the current submission candidate, which is updated only when the model selects a node through update_answer. Historical solutions are preserved in the experiment tree, so exploration does not overwrite the current submission.

(3) Information from earlier experiments is not lost when a window ends. Window boundaries determine which decisions receive a reward, but do not interrupt the interaction context. The trajectory continues across windows, and information from previous experiments, including feature performance, rejected hypotheses, and analysis findings, is carried into subsequent windows. Therefore, when the value of an earlier exploratory action becomes apparent in a later experiment, the model can use the information retained from that exploration. The improved outcome in the later window is achieved using this information, and the corresponding reward is assigned to the subsequent decisions that use it.

(4) Empirical comparison. In Table 4, trajectory-level GRPO and PPO-style optimization assign the final trajectory outcome as a shared reward to all decisions along the trajectory. With the same MLE-SFT initialization and training tasks, and the shared optimization settings listed in Table 10, GRPO and PPO-style achieve Any Medal rates of 56.1% and 60.6%, respectively, both below the MLE-SFT starting point of 62.1%, whereas WIRL reaches 72.7%. Removing the trend term reduces WIRL's performance to 66.7%. These results indicate that, in long-horizon MLE, execution-window optimization based on completed experiments improves long-horizon decision-making more effectively than assigning a shared reward to the entire trajectory.

Explicit multi-window return estimation, such as discounted returns across windows or long-horizon value functions, could further assign credit to exploratory decisions whose benefits emerge over multiple experiments. We identify this as a direction for future work in Section 7.

We again thank the reviewer for the question, which has helped us clarify the connection between window-level optimization and long-horizon decision-making.

AuthorsOpenReview ↗

Official Comment to Reviewer SQ4j (Part 2)

[W2] Methodological novelty: how much of the improvement comes from method design versus data and training effort, and what generalizable insights the recipe provides.

We thank the reviewer for this question. We have made the following additions in the revised manuscript:

  • Section 5.3: We add an MLE SFT comparison without outcome-guided trajectory filtering and supervision truncation, with the mean and SEM over three evaluations reported in Appendix D.1.
  • Section 3.2: We clarify how experiment-level outcomes recorded by the Harness determine the supervision boundaries in SFT and the execution windows in WIRL.
  • Appendix D.1: We add an SFT comparison in which MLE teacher trajectories collected using M1 Harness are replaced with trajectories collected using Claude Code (Table 12).
  • Introduction and Conclusion: We clarify the unifying principle underlying the three method components.

(1) M1 addresses the challenging problem of autonomous optimization in real-world long-horizon agentic MLE.

Real-world MLE tasks involve data analysis, code modification, model training, and evaluation, with costly execution and delayed feedback, and the agent must maintain experimental states and adjust optimization directions over thousands of interaction steps. Existing studies advance this direction with different trade-offs in task scale, training granularity, and optimization organization. ML-Agent [1] collects expert trajectories limited to 15 steps and 30 minutes and mainly uses single-step RL on intermediate decisions. SandMLE [2] uses miniature synthetic environments to reduce execution costs and enable trajectory-level RL. Frontis-MA1 [3] trains predefined program evolution operators and organizes long-horizon search through an external evolutionary framework.

M1 instead trains a model to organize and advance the complete, open-ended MLE experimentation process on real historical competition tasks, rather than optimizing local actions or predefined operators. Individual teacher trajectories reach about 3,000 steps over 12–24 hours, so the model learns long-horizon optimization from real execution feedback.

(2) M1's methodological contribution lies in the coordinated design of training mechanisms for this challenge.

① Model-directed Interaction Harness. We design a dedicated Interaction Harness that gives the model control over experimental state management, solution branching, and optimization directions. This enables the model to autonomously organize open-ended, long-horizon experiments rather than relying on predefined workflows.

② Outcome-guided Hierarchical SFT. Through Hierarchical Agentic SFT, we use actual experimental outcomes to select valuable interaction trajectories and determine supervision boundaries, rather than treating complete trajectories as uniformly positive demonstrations.

③ Feedback-aligned Policy Optimization (WIRL). We organize the multi-turn decisions associated with an experimental outcome into a single optimization unit and combine trend signals, window-level length normalization, and asynchronous policy correction, allowing the model to continuously learn from intermediate experimental feedback.

These designs address environment organization, behavioral supervision, and policy optimization respectively, and together constitute a post-training approach for real-world MLE.

(3) Additional experiments further validate the effectiveness of key methodological designs.

① Effect of the Interaction Harness. We replace MLE teacher trajectories collected using M1 Harness with trajectories collected using Claude Code; the scientific teacher trajectories were already collected using Claude Code, and all other configurations are identical. Each resulting model is evaluated three times under both harnesses (Any Medal, mean ± SEM):

MLE Trajectory SourceM1 Harness (%)Claude Code (%)
Claude Code53.0 ± 4.056.1 ± 3.0
M1 Harness (Ours)62.1 ± 1.560.6 ± 4.0

The model trained with M1 Harness trajectories outperforms the one trained with Claude Code trajectories under both evaluation harnesses, so the Harness design contributes to performance rather than merely providing an execution environment. Its performance under Claude Code (60.6%) is also close to that under M1 Harness (62.1%), indicating that the learned capabilities transfer to a general-purpose coding harness.

General-purpose coding harnesses do not distinguish experiments from other operations or record each experiment's outcome, whereas M1 Harness records them explicitly, and these records define the supervision boundaries in SFT and the execution windows in WIRL. The three designs are therefore tightly coupled.

AuthorsOpenReview ↗

Official Comment to Reviewer SQ4j (Part 3)

② Effect of Outcome-guided Supervision. We use the same raw teacher trajectories, initial checkpoint, and training configurations, removing only outcome-guided trajectory filtering and supervision boundary selection:

Supervision StrategyAny Medal (%)
Outcome-guided filtering and truncation (Ours)62.1 ± 1.5
Without filtering and truncation56.1 ± 3.0

Removing outcome-guided filtering and truncation reduces performance by 6.0 points. Organizing long-horizon supervision by experimental outcomes therefore preserves effective exploration and failure recovery, and avoids treating subsequent degraded behavior as positive demonstrations.

③ Effect of WIRL. As shown in Table 4, under the same MLE-SFT initialization, trajectory-level GRPO and PPO-style reach 56.1% and 60.6% Any Medal, whereas WIRL reaches 72.7%. Removing the local trend reward or policy correction reduces it to 66.7% and 65.2%, respectively.

M1 also improves in other environments and tasks, including Claude Code, AIME scaffold optimization, and representative MLS-Bench-Lite algorithm optimization tasks, so the learned optimization capabilities are not limited to the training harness.

(4) M1 provides three generalizable insights for training long-horizon agents.

First, autonomous experiment organization can itself become a post-training objective in real-world MLE tasks. Models should learn not only to generate code or execute local operations, but also to maintain experimental states, explore alternative solutions, analyze execution feedback, and autonomously determine subsequent optimization directions throughout the complete experimentation process. This suggests that the ability to organize long-horizon tasks can be transferred from externally designed workflows into learnable strategies within the model itself.

Second, long-horizon trajectory supervision should not treat complete interactions as uniformly positive demonstrations. Instead, experimental outcomes should guide the selection of effective exploration, failure-recovery behaviors, and supervision boundaries. This principle is also relevant to scientific experimentation, algorithm optimization, and other tasks where verifiable feedback becomes available only after multiple rounds of interaction.

Third, the granularity of RL optimization should align with environmental feedback events when outcomes depend on multiple decisions, rather than being restricted to individual actions or complete trajectories. Rewards should reflect both the absolute quality of solutions and local optimization progress. Under asynchronous execution, window-level length normalization and policy-lag correction are also needed to control the effects of interaction units with different lengths and samples collected from outdated policies. These considerations provide a useful approach to policy optimization for other long-horizon agent tasks with costly execution feedback.

(5) The role of data and training effort.

We acknowledge that domain-specific task construction and staged training provide an important foundation (Table 3). Beyond it, the controlled experiments above show that the selection of supervision content, the intermediate feedback structure of the interaction environment, and the granularity of policy optimization each add further gains.

We again thank the reviewer for the constructive feedback.

References

[1] Liu, Zexi, et al. "ML-Agent: Reinforcing LLM agents for autonomous machine learning engineering." arXiv preprint arXiv:2505.23723 (2025).

[2] Zhou, Yuhang, et al. "Synthetic sandbox for training machine learning engineering agents." arXiv preprint arXiv:2604.04872 (2026).

[3] Yang, Junlin, et al. "Frontis-MA1: Training an AI4AI model towards recursive self-improvement in machine learning engineering." arXiv preprint arXiv:2607.28568 (2026).

AuthorsOpenReview ↗

Official Comment to Reviewer SQ4j (Part 4)

[W3] Necessity of the custom interaction harness.

We thank the reviewer for this question. We have made the following additions in the revised manuscript:

  • Section 3.2: We further clarify how experiment-level outcomes recorded by the Harness determine the supervision boundaries in SFT and the execution windows in WIRL.
  • Appendix D.1: We add an SFT comparison replacing M1 Harness MLE teacher trajectories with Claude Code trajectories (Table 12).

M1 Harness is designed for M1 and provides a unified experiment structure for trajectory collection, post-training, and evaluation; both outcome-guided SFT and WIRL build on it.

(1) Limitations of general-purpose harnesses and the key capabilities of M1 Harness. In general-purpose coding harnesses, an experiment is one of many shell commands: the harness does not distinguish experiments from other operations or record each experiment's outcome. Compaction and summarization ease the length limit, but experimental states remain text in the conversation, and a summarized history may lose the code and execution results of previous solutions.

M1 Harness instead turns candidate executions and submission updates into explicit events through execute_code and update_answer. Candidate code, execution logs, and metadata are stored in solution nodes forming an experiment tree by parent–child revision, so the model can retrieve any historical solution to compare alternatives, backtrack, or change direction, while a separate submission tool keeps exploration from overwriting the selected submission (Appendix B).

This difference also determines how effectively a model learns from the trajectories. Both harnesses compact history near the context limit, but M1 Harness keeps candidate code and execution logs in solution nodes independently of compaction. For a small model such as M1, the supervision signal therefore shifts from inferring over a long history which operations were effective to learning complete analysis–modification–execution–outcome units, making it easier to learn what to do at each optimization stage and which changes lead to improvement.

(2) Controlled SFT comparison using trajectories collected with Claude Code. To answer the training-stage question directly, we replace the MLE teacher trajectories collected with M1 Harness by Claude Code trajectories during MLE SFT, keep all other data and configurations unchanged, and evaluate each SFT model three times under both harnesses (Any Medal, mean ± SEM):

MLE Trajectory SourceM1 Harness (%)Claude Code (%)
Claude Code53.0 ± 4.056.1 ± 3.0
M1 Harness (Ours)62.1 ± 1.560.6 ± 4.0

The model trained with M1 Harness trajectories performs better under both harnesses. Even when both collection and evaluation use Claude Code, its 56.1% remains below the 60.6% of the model trained with M1 Harness trajectories.

(3) Experiment-level outcomes provide execution windows for WIRL. WIRL treats the multi-turn decisions between two consecutive execute_code outcomes as an execution window and uses that outcome as a shared reward for the decisions within it; a completed window can be used immediately for asynchronous updates. With the same MLE-SFT initialization and training tasks, WIRL reaches 72.7% Any Medal against 56.1% for GRPO and 60.6% for PPO-style optimization, both trained on terminal rewards over complete trajectories (Table 4). Building these windows requires explicit records of experiment events and outcomes, which M1 Harness provides and general-purpose harnesses do not record.

(4) Complementary use of general-purpose harnesses. M1 also uses data from general-purpose environments: the general agent initialization stage includes Claude Code-style trajectories, and the scientific teacher trajectories are collected with Claude Code (Appendix C.1). They provide general programming, tool-use, and scientific problem-solving experience, complementing the MLE trajectories from the M1 Harness.

(5) Additional comparison on algorithm optimization tasks. We evaluate the same M1 model with both harnesses on 12 representative MLS-Bench-Lite tasks, one per domain:

Evaluation HarnessAverage Verifier Reward (Representative MLS-Bench-Lite Tasks)
Claude Code0.351
M1 Harness (Ours)0.425

M1 Harness addresses these limitations, brings measurable gains under the same SFT configuration, and gives outcome-guided SFT and WIRL the experiment-level structure they require.

Reviewer_CiDDOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces M1, an agentic model for autonomous machine learning engineering. The authors build executable tasks and a model-directed interaction environment, then combine hierarchical SFT with WIRL to train M1 to improve solutions using experimental feedback. Under the M1 Harness, M1 outperforms the evaluated same-scale baselines and approaches the performance of larger models.

Critical strengths
  1. The motivation is clear. By training on completed execution windows, the method can use experimental feedback before the entire trajectory is finished.
  2. The paper includes useful ablations on the main training components. Table 3 studies the effects of general agent SFT, MLE-domain SFT, and WIRL, while Table 4 compares WIRL with GRPO, PPO-style optimization, and several WIRL variants.
  3. The performance gains transfer across interaction harnesses. M1 outperforms its base model under Claude Code, suggesting that the improvements are not specific to the M1 Harness.
Critical weaknesses
  1. The methods in Table 4 share the same MLE-SFT initialization but differ in reward processing and data collection. The training configurations of these RL baselines need to be reported more fully.
  2. WIRL uses the current experiment's outcome as the window reward, without explicitly incorporating future returns. The paper does not explain how this objective handles exploratory decisions that yield no immediate improvement but benefit later experiments.
Questions for authors
  1. Could the authors provide the main training settings for GRPO, PPO-style optimization, and WIRL in Table 4, including learning rate, batch size, number of update steps, and data collection details?
  2. How does WIRL handle credit assignment for exploratory decisions whose benefits only emerge in subsequent windows?
3 comments
AuthorsOpenReview ↗

Official Comment to Reviewer CiDD (Part 1)

[W1, Q1] Training configurations of the RL methods in Table 4: learning rate, batch size, update steps, and data collection.

We thank the reviewer for pointing out the need for more complete reporting of the RL baseline configurations. We have made the following additions to the revised manuscript:

  • Appendix C.4: We add Table 10, which provides a side-by-side comparison of GRPO, PPO-style, and WIRL, including their shared training configurations and method-specific differences.
  • Section 5.3: We clarify that the three methods share the same training tasks, reward tiers, optimization hyperparameters, update steps, and computational resources, with a reference to Table 10.

The differences among the three methods in Table 4 primarily concern reward handling and data collection, which are the variables examined in this comparison. Reward handling includes the unit to which rewards are assigned and how reward weights are constructed: WIRL directly weights execution windows using window-level rewards, whereas GRPO and PPO-style optimize complete trajectories. For data collection, WIRL uses asynchronous sampling with policy correction to account for policy lag, while GRPO and PPO-style use synchronous sampling. All other settings, including initialization, training tasks, interaction environment, reward tiers, optimization hyperparameters, update steps, and computational resources, are kept consistent.

(1) Shared training configurations.

SettingValue
Initialization and KL referenceSame MLE-SFT checkpoint
Training tasksSame Kaggle-MLE Factory and MLE-Dojo tasks (Appendix C.1)
Interaction environmentM1 Harness and the same execution sandbox; the analyze sub-agent uses the current training model
Learning rate (initial / minimum)1e-6 / 1e-7
Global batch size32
Update steps69
Policy clipping parameters (ε−, ε+)(0.3, 3.0)
KL / entropy coefficients1e-3 / 1e-4
Reward tiersEq. (2): −1.0 / −0.6 / −0.4 / 1.0 / 2.0
Rollout samplingTemperature 0.6 / top-p 0.95 / top-k 20
Rollout concurrency64 trajectories, 4 concurrent trajectories per task
Execution limit per experiment90 minutes
Trajectory terminationUp to 3,000 steps, 12 hours, or 128K context; early stopping upon obtaining a medal
Maximum generation length per call65,536 tokens
Token-level implementationOriginal prompts and output tokens are preserved for each model call; the loss is computed only on model-generated tokens
Training resources2 nodes × 8 GPUs, allocated separately to parameter updates and rollout inference
AuthorsOpenReview ↗

Official Comment to Reviewer CiDD (Part 2)

(2) Method-specific differences. (continued from above)

SettingGRPOPPO-styleWIRL
Optimization unitComplete trajectoryComplete trajectoryExecution window
RewardQuality tier of the final trajectory solutionQuality tier of the final trajectory solutionWindow quality tier + trend term$T_m$
WeightingGroup-relative normalization over 4 trajectories from the same taskDirect reward weighting without a criticDirect window-reward weighting
Batch samples32 complete trajectories32 complete trajectoriesExecution windows from 32 distinct trajectories
Data collectionSynchronous: updates after the entire batch of trajectories finishesSynchronous: updates after the entire batch of trajectories finishesAsynchronous: completed windows enter the training queue
Policy correctionNone (samples collected from the current policy)None (samples collected from the current policy)Token-level interval correction [0.2, 4.0]

GRPO uses group-relative advantages, whereas PPO-style uses a clipped objective weighted by terminal trajectory rewards without training an additional critic, following the reward-weighted form used in WIRL. This allows us to compare the effects of different optimization units and data collection procedures. The three methods share the same learning rate, batch size, and policy clipping bounds to maintain controlled comparison conditions. For the two WIRL ablations, we remove only the corresponding component from the full WIRL configuration.

All three methods are trained for the same number of update steps. Synchronous collection keeps the samples of GRPO and PPO-style on the current policy, so at equal updates asynchronous collection adds policy lag for WIRL rather than extra updates; and since each of their samples is a complete trajectory, they also consume more interaction data under the same number of updates. The gains of WIRL in Table 4 therefore do not come from more updates or more training data. The learner also waits for training data 87.6% of the time with GRPO against 39.3% with WIRL (Appendix D), so asynchronous window collection reduces training overhead.

We again thank the reviewer for the constructive suggestion. The additional configurations clarify the comparison conditions in Table 4 and improve the reproducibility of the experiments.

AuthorsOpenReview ↗

Official Comment to Reviewer CiDD (Part 3)

[W2, Q2] Credit assignment in WIRL for exploratory decisions whose benefit appears only in later windows.

We thank the reviewer for raising this question. We have made the following additions to the revised manuscript:

  • Section 4.2: We add a paragraph entitled "Within-window credit assignment and cross-window continuity" to clarify how credit is assigned within windows and how information is carried across windows.
  • Section 7: We identify multi-window return estimation and long-horizon value modeling as directions for future work.

WIRL assigns credit to exploratory decisions within each window based on experimental outcomes, while information from earlier experiments is carried forward to subsequent decisions through the interaction history. Rewards are not propagated backward across windows.

(1) Exploratory decisions typically belong to the same window as the experiment they support. An execution window contains all multi-turn decisions between two consecutive experimental outcomes. Data analysis, feature exploration, hypothesis testing, and code modifications typically occur before the next experiment is launched. Their effects are reflected in the outcome of that experiment, so these decisions share the same window reward. Exploratory decisions whose benefit only becomes visible in the experiment result that closes the window are therefore credited within that same window.

(2) Exploration does not overwrite the current submission, and temporary score decreases incur only a small additional penalty. Executing an experiment does not change the current submission candidate, which is updated only when the model selects a node through update_answer. Historical solutions are preserved in the experiment tree, so exploratory experiments do not overwrite the current submission, and the model can return to a better solution at any time.

The window reward is primarily determined by the absolute quality of the solution, with reward tiers of −1.0 (model-caused execution error), −0.6 (no scorable submission), −0.4 (valid submission below the medal threshold), 1.0 (Bronze or Silver), and 2.0 (Gold). The local trend term TmT_m contributes only ±0.2. Within the same quality tier, a temporary score decrease receives a reward only 0.2 lower than an unchanged score. The main reward differences therefore reflect solution quality, consistent with the objective of the final evaluation.

(3) Information from earlier experiments is carried forward through the interaction history. Each window is conditioned on the accumulated interaction history, which preserves information obtained from earlier experiments, including feature effectiveness, failed hypotheses, and analysis results. When subsequent decisions use this information to achieve better experimental outcomes, the corresponding window receives a reward. The reward directly updates the subsequent decisions that use this information, allowing the policy to learn how to use earlier exploration to advance optimization.

(4) Empirical comparison. In Table 4, trajectory-level GRPO and PPO-style optimization assign the final trajectory reward to all decisions along the trajectory, so early decisions also receive credit from the final outcome. With the same MLE-SFT initialization and training tasks, and the shared optimization settings listed in Table 10, GRPO and PPO-style achieve Any Medal rates of 56.1% and 60.6%, respectively, both below WIRL's 72.7%. Removing the trend reward reduces WIRL's performance to 66.7%.

For long-horizon MLE tasks, where execution is costly and each experimental outcome depends on many intermediate decisions, assigning credit at the boundaries of completed experiments provides a more effective learning signal. It also allows asynchronous policy updates without waiting for the entire trajectory to finish.

Explicit multi-window return estimation, such as discounted returns across windows or long-horizon value functions, could further assign credit to exploratory decisions whose benefits emerge over multiple experiments. We identify this as a direction for future work in Section 7.

We again thank the reviewer for the question, which has helped us clarify WIRL's credit assignment mechanism.

Reviewer_W1XGOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces M1: An Agentic Model for MLE, alongside the data collection and RL method for producing this model. First the authors produced Kaggle-MLE Factory, a set of task environments for training models to solve MLE tasks. The authors scraped Kaggle for environments which they converted into task environments using a multi-agent pipeline for task construction validation, and repair. This produced 154 environments. They introduce an interaction harness for MLE which enables simpler experiment analysis and easier backtracking. They start with a Qwen3.6-35B-A3B model and start with General SFT on agent trajectories. Then they SFT on MLE tasks from their Factory and MLE Dojo. Finally they use RL with a RL method which they call "Windowed Interaction Reinforcement Learning" where they take update steps every time the model calls the "execute_code" tool. They introduce a custom reward based on the quality of the submission on the Kaggle leaderboard and introduce a policy correction for weighting (and excluding) certain tokens in the policy update based on the difference in the logits of those tokens in the current policy (before update) and the policy that produced the rollout. Together this training process produces the M1 model which attains similar performance on MLE tasks to frontier models (GPT-5.5, GLM 5.3) but on a much smaller model.

Critical strengths

The paper was well written and aside from some missing details about data collection could be a reproducible recipe for training an MLE agent, especially improving the MLE capabilities of smaller models.

I found the Windowed Interaction RL (WIRL) algorithm to be clean and interesting. In particular I liked the idea of windowed updates when the agent executes code, and applying intermediate signal as windowed rewards in the update step.

The M1 model does seem to be performant on Kaggle tasks, so the model itself could be useful for researchers studying recursive self-improvement on limited resource hardware where they would benefit from a 30B model that can write effective ML software.

Critical weaknesses

I will write my concerns as an addressable list:

  1. I am uncertain if this paper proposes a recipe for MLE agent training that hasn't already been discussed commonly in the literature. The data collection is largely similar to the work done in MLE Dojo [1], and the training recipe is quite similar to those covered in ML-Agent [2] or AceGRPO [3] with the exception of the Windowed Interaction RL algorithm. However the policy correction of WIRL matches the work done in Ring-1T [4] for their IcePop RL update, so the contribution is primarily in the (domain specific) reward function, and the windowed update.
  2. If you deem it similar enough, perhaps cite IcePop [4] when talking about the policy correction rule.
  3. I feel there is a lack of detail in the construction of Kaggle-MLE Factory across both section 3.1 and the appendix. The extent of detail shared is that a multi-agent system processes the tasks and a high level description of the goals of each of the agents in the system (the discovery agent, the analyzer agent, the generator agent, and the execution/validator repair agent). The high level description of the agents and their objectives is not enough to reproduce this process. The paper would benefit from an appendix section dedicated to exact prompts about how these agents/llms were used, which models were used, and how their outputs were validated. Without this detail the paper is missing a significant component needed for reproduction.
  4. I don't feel that the need to create a custom MLE harness was justified carefully (empirically) enough. It was unclear to me if this made any significant impact on the results versus just using a generic harness on MLE tasks for SFT and RL.
  5. The choice to only evaluate on 10 tasks for NatureBench in Table 2 perplexes me, since so much of agent evaluation comes down to high variance within a rollout. Already we see the Nex-N2.5-mini baseline nearly matching performance of M1, but I don't know how much stock to place in this eval because of the small sample size. Can you evaluate on more instances from NatureBench or at least run multiple trials with confidence intervals?

[1] https://arxiv.org/abs/2505.07782 [2] https://arxiv.org/abs/2505.23723 [3] https://arxiv.org/abs/2602.07906 [4] https://arxiv.org/pdf/2510.18855

Questions for authors

I was very concerned about decontamination since both the evaluation benchmark (MLE-Bench) and your RL task/SFT source (Kaggle-MLE Factory) come from Kaggle. I know you mention on L316-317 "Any training tasks overlapping with MLE-Bench or NatureBench are excluded" and reiterate a similar line in Appendix C.1, but I was interested in some more detail on how you performed this decontamination, since it such a crucial step, and if done incorrectly would weaken the findings of the rest of the paper. Could you clarify how you checked this?

Also would you be able to include more trials in your NatureBench evaluation in Table 2? I am not sure how to interpret these results with a single run on just 10 tasks.

5 comments
AuthorsOpenReview ↗

Official Comment to Reviewer W1XG (Part 1)

[W1, W2] Similarity to prior work: data collection, training recipe, and policy correction (IcePop).

We thank the reviewer for pointing out these related works and for suggesting IcePop. We have made the following additions to the revised manuscript:

  • Section 4.2: We cite IcePop in the discussion of policy correction and clarify the relationship between its correction mechanism and ours.
  • Appendix A: We provide detailed prompts and design descriptions for each stage of Kaggle-MLE Factory to clarify how the tasks are constructed.

(1) Task construction. MLE-Dojo provides interactive MLE environments and competition-specific data preparation and evaluation scripts. Our training tasks come from two sources: 134 tasks selected from MLE-Dojo and 154 newly constructed tasks from Kaggle-MLE Factory. Factory organizes competition discovery, data analysis, data preparation and evaluation code generation, execution-based validation, and error repair into an automated pipeline, in which the code is generated by an LLM and then checked and repaired through actual execution. Appendix A provides detailed prompts and design descriptions for each stage.

(2) Training targets. ML-Agent and AceGRPO both divide an interaction into many short segments and use a single action or tool call as the optimization unit, so the model learns what to do at the current state while experiment organization across stages is left to the sampled state distribution. M1 instead trains the complete autonomous optimization process, with teacher trajectories from full runs of up to about 3,000 steps over 12–24 hours. At this scale, how to organize experiments becomes part of what is learned: when to continue along the current direction, when to recognize an ineffective approach and backtrack, and how to determine subsequent actions from execution results. WIRL matches this length by treating the multi-turn decisions between two consecutive experiment outcomes as one execution window, using that outcome as their shared reward, and updating asynchronously while the trajectory continues.

Therefore, M1 is not simply an application of existing approaches at a larger scale, but an integrated design for long-horizon autonomous MLE. Task construction provides executable real-world tasks, hierarchical SFT teaches the model to organize long-horizon experiments, and WIRL enables continuous improvement from the model's own experimental outcomes. Together, these components target a common objective: training the model's ability to organize experiments autonomously, rather than only improving the correctness of individual decisions.

(3) Policy correction and IcePop. We thank the reviewer for pointing us to IcePop and have added the citation in Section 4.2. Both methods apply a two-sided interval constraint to the probability ratio, but they address different sources of mismatch. IcePop addresses implementation-level discrepancies between the training and inference engines, which operate on the same weights and context; the constraint mitigates differences in their numerical implementation. WIRL addresses temporal policy lag. Under asynchronous collection, a long trajectory may span multiple parameter updates, so later calls in the trajectory come from an outdated rollout policy. WIRL therefore aligns each call in a window with the policy version that generated it, computes ratios from the log probabilities recorded at collection time, and excludes tokens outside the interval from the policy-gradient update. IcePop concerns whether two implementations agree at the same moment, whereas WIRL concerns which policy version produced the data used for an update.

(4) Methodological contribution of M1. In long-horizon tasks with costly execution, verifiable feedback arrives as intermediate results that experiments produce repeatedly, rather than only at the end of the trajectory. The learning unit should therefore align with where feedback is generated, rather than with individual interaction steps or the entire trajectory. This premise shapes three levels of the design: the Harness makes each execution an explicit event with an observable outcome; SFT uses these outcomes to decide which trajectory segments are effective demonstrations and where supervision is truncated; and WIRL uses them to define optimization windows and assign rewards. All three act on the same object, the feedback events generated by experiments, which distinguishes the design from a combination of existing components.

We again thank the reviewer for the constructive suggestions.

AuthorsOpenReview ↗

Official Comment to Reviewer W1XG (Part 2)

[W3] Insufficient details for reproducing Kaggle-MLE Factory, including the exact prompts, models, and output validation procedures.

We thank the reviewer for the attention to reproducibility. We have made the following additions in the revised manuscript:

  • Section 3.1: We clarify the division of responsibilities between rule-based procedures and the LLM at each stage, and specify the model used in the pipeline.
  • Appendix A: We describe construction stage by stage, with Table 6 summarizing what rules and the LLM handle at each stage; report the model and call configuration together with the prompts of the five LLM stages (Figures 5–9); and give the validation layers and the repair procedure.

(1) Use of agents and LLMs. Decisions that can be made by inspecting data, executing code, or comparing numerical values are handled by deterministic rules, while decisions requiring an understanding of competition descriptions are made by the LLM. The discovery and analyzer agents first perform rule-based metadata filtering and data profiling, and the LLM then determines semantic information such as the technical direction, evaluation metric, and submission format; when an LLM judgment conflicts with measured data, the measured value is retained. The generator agent uses the LLM to write data-preparation and scoring code, the repair agent uses the LLM to revise code based on validation feedback, and execution and validation are entirely rule-based.

(2) Model and prompts. All agents use the same LLM, gemini-3.1-pro-preview, with single-turn, zero-shot prompting; the configuration is in Appendix A.2, and Figures 5–9 give the prompts for discovery, analysis, data-preparation code generation, scoring code generation, and scoring code repair.

(3) Output validation. Validation relies entirely on actual execution. Generated code first undergoes syntax and entry-point checks. The data-preparation code is then run on real data, after which the task package is checked for its structure, labels, and identifier overlap. The scoring code is tested with shuffled rows, invalid submissions, and oracle submissions constructed from ground-truth labels. Tasks that fail are routed to the repair stage, which rewrites the relevant code according to the error source, and all checks are rerun after repair; tasks for which ground-truth labels or a faithful scoring implementation still cannot be obtained are excluded. After a final human review, 154 tasks are retained.

We again thank the reviewer for the constructive suggestion.

[Q1] Data decontamination: How was overlap between the training tasks and MLE-Bench or NatureBench checked?

We thank the reviewer for the attention to decontamination. We have made the following additions in the revised manuscript:

  • Section 5.1 and Appendix C.1: We state that overlap is checked using competition identifiers for MLE-Bench and source-paper DOIs for NatureBench, and describe the training data sources and both decontamination procedures in a dedicated subsection.

(1) MLE-Bench. We exclude overlapping tasks by Kaggle competition identifier at three stages. Before construction, we establish an exclusion list covering all 75 competitions of the full MLE-Bench, although evaluation uses only the 22 competitions in MLE-Bench Lite, and we remove all MLE-Dojo tasks overlapping with MLE-Bench at selection, retaining 134 tasks. During construction, the discovery agent checks candidates against the list and discards any match, and the pipeline entry checks every task before construction, including manually specified tasks. After construction, we cross-check all 288 training tasks (154 from Kaggle-MLE Factory and 134 from MLE-Dojo) against the full MLE-Bench by exact identifier matching, case- and punctuation-normalized matching, and substring matching. No overlaps are found, and the results are confirmed by manual review. RL uses the same MLE tasks and is covered by the same checks.

(2) NatureBench. We check scientific tasks at the source-paper level. Each task records the DOI and title of its source paper, which we compare against the source papers of all 90 NatureBench evaluation tasks, including the 10-task Lite subset, and no paper is shared. Title matching also covers the same paper appearing under different DOIs, such as preprint and published versions, and the results are confirmed by manual review.

(3) Trajectory inspection. We inspect all collected training trajectories and find no task or data contamination related to the evaluation benchmarks, and no access to evaluation answer files.

These checks confirm that the training tasks do not overlap with the MLE-Bench or NatureBench evaluation tasks. We again thank the reviewer for the question.

AuthorsOpenReview ↗

Official Comment to Reviewer W1XG (Part 3)

[W4] Value and necessity of a custom MLE harness.

We thank the reviewer for raising this question. We have made the following additions in the revised manuscript:

  • Section 3.2: We clarify how experiment-level outcomes recorded by the Harness determine the supervision boundaries in SFT and the execution windows in WIRL.
  • Appendix D.1: We add an SFT comparison in which MLE teacher trajectories collected using M1 Harness are replaced with trajectories collected using Claude Code.

M1 Harness is not intended to replace general-purpose coding harnesses such as Claude Code for MLE tasks. It is designed for experiment organization, state maintenance, and feedback-driven optimization in long-horizon MLE, and provides a shared experiment structure for trajectory collection, post-training, and evaluation.

(1) M1 Harness provides structured long-horizon interaction for experiment-level optimization. In a general-purpose harness an experiment is one of many shell commands: the harness does not separate experiments from other operations and does not record each experiment's result separately. Both harnesses compact history near the context limit, but what survives differs. In a general-purpose harness, experiment state exists as dialogue text, so after summarization the code and execution results of past solutions are no longer fully kept and the link between decisions and results weakens. The M1 Harness stores candidate code, execution logs, and metadata in solution nodes, links them into an experiment tree by parent–child revision, and keeps submission candidates in a separate tool (Appendix B), so this state outlives compaction and any past solution can be retrieved. For a model of M1's size, this changes the supervision signal from inferring over a long history which operations worked to learning complete analysis–modification–execution–outcome units step by step.

The same records define the execution windows of WIRL. The multi-turn decisions between two execute_code events share the outcome of that experiment as their reward, and a completed window can be used directly for asynchronous updates. Under the same MLE-SFT initialization and training tasks, WIRL reaches 72.7% Any Medal, compared with 56.1% for trajectory-level GRPO and 60.6% for PPO-style optimization (Table 4), showing that the granularity of optimization matches the granularity of the feedback the harness records.

(2) M1 Harness and general-purpose harnesses provide complementary training data rather than replacing one another. M1 also uses trajectories from general-purpose environments: the general initialization stage includes Claude Code-style trajectories, and during MLE adaptation the M1 Harness trajectories collected on Kaggle/MLE-Dojo tasks are combined with Claude Code trajectories collected on tasks built from scientific papers (Appendix C.1). The two sources provide complementary supervision.

(3) A controlled comparison directly evaluates the additional SFT value of trajectories collected with M1 Harness. At the MLE-SFT stage, from which RL is initialized, we replace the M1 Harness trajectories for MLE tasks with Claude Code trajectories and keep everything else unchanged, then evaluate each model three times under both harnesses (Any Medal, mean ± SEM):

MLE Trajectory SourceEvaluated with M1 Harness (%)Evaluated with Claude Code (%)
Claude Code53.0 ± 4.056.1 ± 3.0
M1 Harness (Ours)62.1 ± 1.560.6 ± 4.0

Even when the model is evaluated in Claude Code, it still outperforms the model trained on Claude Code trajectories, so the gain comes from the training value of these trajectories rather than from the harness used for collection or evaluation.

(4) Results on other algorithm optimization tasks provide additional evidence of its applicability. We evaluate the same M1 model with both harnesses on 12 representative algorithm optimization tasks, one per domain of MLS-Bench-Lite, one run per task:

Evaluation HarnessAverage Verifier Reward (Representative MLS-Bench-Lite Tasks)
Claude Code0.351
M1 Harness (Ours)0.425

Under Claude Code, M1 also reaches 69.7% Any Medal on MLE-Bench Lite against 42.4% for the base model, indicating that what is learned in the M1 Harness transfers to a general-purpose harness.

We again thank the reviewer for the constructive suggestion.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions and Author Responses

Dear Area Chairs, 

Thank you for overseeing the review of our paper. In response to the reviewers' comments, we have substantially revised the manuscript and conducted additional experiments and analyses. The main revisions are summarized below. 

1. Methodological Contributions and SFT Design. 

We clarified the methodological contributions of M1 and their relationship to prior work, highlighting the distinctive optimization objective of MLE. To validate the contributions of our training designs, we added two controlled SFT comparisons: removing outcome-guided filtering and truncation reduces Any Medal from 62.1% to 56.1% (Table 3), while models trained on M1 Harness trajectories outperform those trained on Claude Code trajectories under both evaluation harnesses (Table 12). 

2. WIRL and RL Training Configuration.

 We expanded the explanation of WIRL's execution-window construction, credit assignment, and policy correction, including its relationship to IcePop. We also added a comprehensive comparison of GRPO, PPO-style, and WIRL, reporting their shared training configurations and method-specific differences (Table 10).

 3. Reproducibility and Data Decontamination. 

We documented the Kaggle-MLE Factory construction process, including prompts, model configurations, and execution-based validation procedures (Appendix A). We also detailed the decontamination checks against MLE-Bench and NatureBench (Appendix C.1). 

4. Evaluation Reliability and Generalization. 

We conducted three evaluation runs on NatureBench Lite for M1 and the same-scale baselines, reporting means, SEMs, and per-run Surpass-SOTA counts. Across the three runs, M1 achieved Surpass-SOTA on 4, 6, and 5 of the 10 tasks, respectively. The highest count in any run by a same-scale baseline was 3 (Table 2 and Appendix D.2). In the rebuttal, we also reported results on three PostTrainBench domains and compared M1 Harness with Claude Code using the same M1 model on 12 representative MLS-Bench-Lite tasks.   We appreciate the reviewers' engagement throughout the discussion. Reviewer SQ4j acknowledged that our additional experiments strengthened the paper's soundness, while maintaining a concern about methodological novelty. We further clarified the motivation behind our designs in a follow-up response. 

We sincerely thank the reviewers and Area Chairs for their time and consideration. We hope these revisions and additional results provide a clearer picture of the contributions and empirical evidence of M1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.