acceptodds
Withdrawn from ICLR 2027

Priming Base Models for Agentic Post-Training: An Open Midtraining Recipe

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Frontier models, general-purpose and domain-specific alike, now include a midtraining stage between pretraining and post-training, but little has been published about how this stage is built. We present an open midtraining recipe that primes base models for agentic post-training, and we show its effectiveness on two Nemotron models of different sizes. Pretraining typically runs at short context and sees almost no agent interactions, while SFT data is mostly synthetic, and very long SFT stages can hurt the model's later RL trainability. Our midtraining stage sits between the two. It trains at 256K context for hundreds of billions of tokens, before the short final 1M extension stage, on a diverse mixture of natural software-engineering data like packed repositories, pull requests, commit histories, and build logs together with a small share of agentic traces. On Nemotron 3.5 Lightning, 500B midtraining tokens improve SWE-bench Verified from 60.7 to 64.5, SWE-bench Multilingual from 55.8 to 62.8, and SWE-bench Pro from 32.0 to 35.5 after identical full SFT. On a larger model based on Nemotron 3 Super, 280B midtraining tokens improve DeepSWE, a harder benchmark of long-horizon tasks, from 11.7 to 15.7 and SWE-bench Verified from 72.9 to 74.3 after identical full SFT. Midtraining also improves robustness across agent harnesses on both models. We report the ablations behind the recipe and an evaluation protocol built on short fixed SFT runs, since base-model proxy metrics did not correlate reliably with post-training results.

Primary area:
foundation or frontier models, including LLMs
Keywords:
midtraining, large language models, software engineering agents, agentic coding, SWE-bench, long-context training, post-training

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mmnHOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a 256K-context midtraining recipe for preparing base language models for agentic coding, combining natural software-engineering data, a small proportion of agent trajectories, and pretraining replay. After identical SFT, the recipe improves Nemotron 3.5 Lightning on SWE-bench Verified, Multilingual, and Pro by 3.8, 7.0, and 3.5 percentage points, respectively, and improves the larger Nemotron 3 Super on DeepSWE by 4.0 points, with additional gains across agent harnesses.

Critical strengths

• Understanding how to construct the intermediate training stage between pretraining and agentic post-training is valuable. The descriptions of data construction, filtering, serialization, and sampling provide useful implementation guidance. • Applying identical short and full SFT to the starting and midtrained checkpoints helps establish that the added midtraining stage can improve the resulting agents. Evaluating after 100B SFT tokens is particularly useful for checking whether the improvements survive substantial downstream training.

Critical weaknesses

• The paper shows that the complete midtraining pipeline improves downstream performance, but it does not establish that the proposed natural-data-heavy recipe is responsible for these gains. In Table 1, most of the downstream improvement comes from adding agent trajectories, yet there is no trace-only or simpler trace-based control under a comparable training budget. Therefore, it remains unclear whether the proposed data composition provides meaningful benefits beyond additional agentic training. • A key motivation for limiting agent trajectories to 10–15% is that excessive SFT-like training may hurt subsequent RL trainability. However, the RL experiment contains only 20 updates and, as the authors acknowledge, does not demonstrate improved RL trainability. Without comparisons across trace proportions or against longer SFT under a meaningful RL budget, this central justification for the proposed mixture remains unverified.

Questions for authors

The topic is valuable, and the results show that midtraining can improve downstream agent performance. However, I do not think the current experiments sufficiently validate the specific recipe and its key motivations. These concerns are the main reason for my current score, and I would be willing to raise it if they can be addressed in the rebuttal.

Reviewer_kob9OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a 256K-context midtraining stage between pretraining and post-training, mixing repository, issue–PR, diff, log, and agent-trace data with replayed pretraining data. On Nemotron 3.5 Lightning and Nemotron 3 Super, the recipe improves post-SFT performance on SWE-bench, DeepSWE, Terminal-Bench, and multiple agent harnesses, with ablations covering trace inclusion, masking, replay, learning rate, and context extension.

Critical strengths
  • The paper addresses a practically important but under-documented stage of agent-model training and provides unusually concrete data-construction, filtering, weighting, and training details.
  • Matched base and midtrained checkpoints receive identical SFT pipelines, directly testing whether midtraining changes downstream post-training potential.
  • Experiments scale token budget and model size and evaluate several benchmarks and harnesses rather than relying on one metric.
  • The ablations cover trace inclusion, tool-response masking, replay, learning rate, and long-context behavior, and the paper states its limitations explicitly.
Critical weaknesses
  • Agentic Midtrain already has plenty of open-source work, like davinci-dev, discussing its data recipe. I want to know what novelty and advantages this paper has over other papers discussing agentic midtraining, beyond just the data volume.
  • The paper uses multiple data sources such as PR, diff data, CI logs, and agent traces, but it doesn't give a detailed explanation of how these data types are constructed, and it doesn't even provide a corresponding pipeline diagram. There is a lack of detailed description regarding data processing.
  • If the authors want to emphasize the superiority of their mid-training recipe, they should at least compare experimental results across three groups with the same data sources but different data ratios. However, I didn't find that in the paper.
  • Finally, the benchmark built on the 500B dataset shows a 4-point improvement on SWE-V, with only marginal gains on the rest. By comparison, the performance gap against davinci-dev is significant.
Questions for authors

see above

Reviewer_3WakOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a practical midtraining recipe for coding agents, which has yielded encouraging results across two model scales and various agent evaluation frameworks. The details regarding data construction and the analysis of the training process offer significant value, and the open nature of this approach will facilitate further development by the community. I have some comments on the paper, and I am willing to adjust my score based on the authors' response

Critical strengths
  1. This paper presents an open midtraining recipe to prepare base models for agentic post-training, offering useful insights for developing coding agents.

  2. This paper evaluates performance across varying model scales, training budgets, and agent frameworks.

  3. The paper provides useful analyses of training choices, including tool-response masking, learning rates, and pretraining replay, while discussing negative results and limitations.

  4. The paper is well-written.

Critical weaknesses
  1. The baseline and midtrained models in Tables 2 and 3 receive different total training budgets. It is therefore unclear to what extent the observed improvements stem from the proposed approach versus the additional training budget.

  2. The ablations demonstrate the value of agentic traces, but the contribution of the software-engineering data beyond these traces remains unclear.

  3. The comparison between the Super 65B and 280B checkpoints would benefit from clarification, since they use different long-context recovery procedures.

Questions for authors
  1. Regarding Tables 2&3, could the authors include a baseline with a comparable computational budget, where the additional budget is instead allocated to extending SFT or continuing code pre-training?

  2. Could the authors add a control experiment to Table 2 in which the amount of natural software engineering data is reduced, and the model is trained primarily using agent trajectory data? This would help determine whether the natural data provides any performance gains.

  3. Could the authors clarify whether the additional long-context recovery stage affects the comparison between these checkpoints?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.