acceptodds
Under review as a conference paper at ICLR 2027

Tmax: A simple recipe for terminal agents

11 views

Abstract

Terminal-using agents have quickly become a popular and important downstream application of language models (LMs). Despite their prevalence, relatively little academic work has examined RL-based training of these models, due to difficult benchmarks, a lack of data, and a lack of simple baseline recipes. We present TMAX, an open RL recipe for terminal agents that brings small open models closer to the frontier. While simple, our recipe achieves 33% on Terminal-Bench 2.0 with only 9B parameters, outperforming much larger models from prior work. Our recipe first involves generating data using a novel taxonomy, combining difficulty control, personas, and verifier diversification, which allows us to cheaply generate large amounts of terminal environments for RL and SFT training. We then train open weight models using RL with our data, using a simple, outcome-only recipe, with small adjustments to improve stability. Our recipe works for models from 2B to 27B and across model families, dominating the Pareto curve for open terminal-agent performance at smaller sizes. We will publicly release our data, models, and code as a strong baseline for future open academic work on terminal agents

Primary area:
foundation or frontier models, including LLMs
Keywords:
agent, terminal, rl, lm

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_JEivOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces TMAX, a simple and open recipe for training terminal-using language-model agents. The authors construct TMAX-13K, a dataset of 13,565 synthetic terminal environments generated through a compositional taxonomy that explicitly controls domain diversity, task and command complexity, personas, fixtures, and verifier types, followed by an LLM-based verifier auditing and repair stage. They then train open-weight models using a relatively simple outcome-based DPPO recipe with several stability improvements, including an FP32 LM head and larger rollout groups. Experiments show that TMAX-13K outperforms a broad set of existing terminal-agent RL datasets under the same training setup, and TMAX-9B reaches 33% on Terminal-Bench 2.0, the strongest reported result among models below 10B parameters. The gains largely transfer across model sizes, Qwen model variants, agent harnesses, and additional tasks such as SWE-Bench Verified and AIME, suggesting that the proposed recipe improves general terminal/tool-use capabilities rather than only fitting a single benchmark.

Critical strengths
  • The paper presents a simple yet well-designed recipe for generating large-scale terminal-agent RL environments, with explicit control over task diversity and difficulty, together with verifier auditing and repair. The resulting TMAX-13K dataset is substantially more balanced and challenging than several existing terminal-agent datasets.
  • The proposed recipe consistently improves multiple model sizes and families, and the gains transfer across different agent harnesses and to additional tasks such as SWE-Bench Verified and AIME. In particular, TMAX-9B achieves strong Terminal-Bench performance relative to similarly sized open models.
Critical weaknesses
  • It remains unclear whether TMAX-13K is intrinsically a better RL dataset or is better aligned with Terminal-Bench. The 13-gram decontamination analysis rules out direct lexical overlap, but does not exclude higher-level similarity in task archetypes, required skills, environment structure, domain composition, or difficulty. Since the main comparison against alternative RL datasets is conducted on TB-Lite/TB2.x, such distributional alignment could partly explain TMAX-13K's advantage. Although the SWE-Bench Verified and AIME experiments demonstrate transfer over the base model, they do not compare against models trained on alternative RL datasets.
  • Only 7.7% of the initially generated environments are classified as clean, while 76.4% are judged to have overly weak verifiers. The authors repair the dominant failure modes and show that training on the repaired data improves downstream performance, but they explicitly do not perform a second rubric-based audit after repair. Therefore, the actual validity rate of the final TMAX-13K dataset remains unclear, including whether repairs fully resolve the original problems or introduce new ones.
  • Dataset difficulty is primarily characterized using a single evaluator, Gemini-3-Flash-Preview, so it is unclear how well this calibration transfers across model strengths and families. More importantly, the gains become much smaller for the strongest tested model: Qwen 3.6 27B improves only modestly on TB2.1 and slightly degrades on TB-Lite, whereas substantially larger gains are observed for smaller models. This raises the possibility that TMAX-13K can only provide useful learning signal for relatively weak starting policies and may become saturated and less useful for stronger ones.
Questions for authors
  • The paper reports that RL training often collapses after roughly 300 steps and presents several practical mitigations, including an FP32 LM head, larger rollout groups, and reduced infrastructure noise. While the ablations provide useful evidence that these interventions can improve stability, they do not identify the underlying failure mode that actually triggers collapse. for example, whether collapsing runs exhibit abnormal shifts in trajectory length or content, reward/advantage distributions, rollout diversity, tool-use behavior, log-probability mismatch, filtering rates, or other training statistics. Without such diagnostics, it is difficult to tell whether the proposed interventions address the same underlying pathology or merely suppress different symptoms. A targeted analysis comparing trajectories and training signals immediately before and after collapse, ideally across stable and unstable runs, would make the stability claims substantially more informative and help determine whether the proposed recipe is robust beyond the specific engineering setup.
  • The authors state that DPPO appeared more stable than GRPO in early pilot experiments, but no quantitative comparison is provided. As a result, it is unclear whether the advantage of TMAX-13K is specific to its interaction with DPPO or whether the dataset remains superior under other commonly used RL objectives. At minimum, a controlled vanilla-GRPO comparison would substantially strengthen the generality of the conclusions.
  • Although the authors acknowledge that the pipeline is completely synthetic and may struggle with settings such as large-scale codebases, the paper does not directly characterize how well the synthetic task distribution reflects real terminal-agent workloads. In particular, roughly uniform coverage across the nine predefined domains is a useful design choice, but does not by itself establish that the generated tasks match realistic user intents, workflows, failure modes, or environment constraints. Have you considered grounding the synthetic generation pipeline in real public issues/tasks while retaining its controllable composition?

Please also see the Weaknesses section for additional questions and concerns.

4 comments
AuthorsOpenReview ↗

Rebuttal - Addressing Reviewer Concerns

We thank the reviewer for noting that our recipe is simple and well-designed, and that our setup transfers across model sizes, families, and tasks. We start by addressing your concerns directly, and answer the questions in a second comment.

Is TMax a better dataset or aligned better with TB?

To investigate this, we evaluate the other models trained on other datasets on SWE-Bench Verified and AIME, and show the results in Table 4 (and full results in Table 16). We find that TMax has the highest AIME performance, and only falls behind on SWE-Bench Verified compared to SWE-Smith, which explicitly was constructed for SWE-Bench. As such, this suggests that the diversity and difficulty of Tmax can lead to improvements on non-terminal bench evaluations relative to other datasets. This also suggests that TMax’s diversity makes it a better choice relative to other datasets for a general set of tasks.

We also compare the distribution of domains between evaluation datasets and RL datasets (including TMax) in Appendix F.4. We find that TermiGen better matches Terminal-Bench Lite and 2.1 distributions, despite underperforming TMax, although the balanced distribution does result in TMax having more data science and scientific computing tasks than other baseline datasets.

Importantly, we primarily compare to other works that also target TB (e.g., TermiGen, OpenThoughts) as their primary evaluation. In this light, our purely synthetic data generation pipeline and ability to tightly control the distribution of our samples to ensure diverse tasks is a strength of our work. Future users of TMax could adjust the sampling ratios of task domains to better match particular distributions of interest to them (e.g., based on real-world user analyses).

Do the repairs fully resolve the original problems or introduce new ones?

We run the same auditor over our repaired dataset, and find that roughly 70% of the ‘verifier-too-weak’ samples now are classified as ‘clean’ (in the entire dataset, 9,014 of 13,565 are judged clean). Overall, in Tmax-13k, 66% samples are classified as ‘clean’, 21% have ‘verifier too weak’, and only ~4% now have answer leakage. We do not apply the repair stage to already-clean tasks, so it cannot make already-good tasks worse. Around 5% of tasks receive a different issue label after repair, but this is fairly small relative to the number of completely fixed tasks. We do not apply further repair rounds due to cost. We show in Table 6 that training on the post-repair data results in significantly stronger performance, showing that the repair stage is effective.

How well does dataset difficulty transfer across model strengths, especially Qwen 3.6 27B?

We first note that we found a minor bug around chat templates when evaluating Qwen 3.6 27b (we forgot to turn on preserved thinking). When we fix this, we find that Tmax training also improves Qwen 3.6 27b on TBLite and TB2.1:

ModelTB-LiteTB2
Qwen 3.6 27B74.2 ± 1.543.6 ± 1.9
TMAX-27B77.4 ± 1.843.9 ± 1.6

Inspired by the reviewer, we also do additional analyses on the difficulty of TMax for Qwen 3.6 27B (and the other model sizes) by looking at the pass rates of groups during RL training (i.e., number of 0/32, 8/32, … , 32/32 correct groups). We add this analysis to Appendix F.3 and find that tmax-13k is not trivially solved by Qwen 3.6 27B, and in fact over 30% of tmax-13k prompts still provide signal during RL training (i.e., we have mixed-outcome rollouts). This suggests that TMax is still difficult enough for stronger models. We also note that our core focus is designing a strong academic setting for small-scale terminal agent training, and as such using smaller, weaker policies is in some ways a feature of our setting, not a bug. We do further investigation (see App. F.3 in the revised copy) and find that the 27B model does improve on many tasks, but also degrades on a substantial number due to it losing some ability to self-verify results. We leave investigating why this occurs to future work.

AuthorsOpenReview ↗

Rebuttal - Addressing Reviewer Questions

Here, we directly answer your questions:

What is the underlying reason for training collapsing? Can you perform a targeted analysis comparing trajectories and training signals?

We provide further analysis and discussion of why training collapses in Appendix E.6. In general, we observed in earlier experiments that collapses in performance were often preceded by large spikes in logprob differences, and as such fixes to reduce such mismatch aided in reducing these crashes. We show that crashes in reward are preceded by logprob and gradient norm spikes, which the FP32 head, DPPO, and larger group size target fixing. We provide ablations and diagnostic figures in Appendix E.6 to ablate these changes (see Figures 8, 11, 12). Surprisingly, training on more stable infrastructure also reduced these spikes. This suggests they are partly caused by noisy rollouts (e.g., tool calls hanging due to slow sandboxes), and that non-robust infrastructure can appear as optimization issues. Reward drops were accompanied by shorter model outputs: in the 8-group size case, 11% of rollouts were cut short before the reward drop, and 38% were cut short afterward. The model also goes from about 20 to 13 bash calls per rollout.

Any experiments comparing DPPO and GRPO?

We provide a comparison in Appendix E.6, Figure 11 comparing GRPO and DPPO under matched settings from an earlier pilot experiment. While DPPO did not completely stop collapse, it did collapse later than GRPO, and when it did collapse, it did not collapse as severely, based on train reward. This provides quantitative evidence for our swap to DPPO. We did not compare all datasets using GRPO due to compute limitations, and we note that we believe that DPPO and GRPO are not very different (DPPO primarily improves the training-inference mismatch issue in GRPO training), suggesting that results would likely transfer to GRPO-like algorithms.

Have you considered grounding the synthetic generation pipeline in real public issues/tasks while retaining its controllable composition? How well does it match realistic user behaviours?

We consider grounding the pipeline in real repositories or tasks is an interesting extension for future work, potentially allowing for more complex tasks while controlling the composition of the dataset.

In Appendix F.4, we do a detailed comparison of Tmax’s distribution to Terminal-Bench Lite/2/4, 1k random samples from SWE-Chat (a dataset of user prompts to coding agents), and several baseline RL datasets. We find that Terminal bench has a significantly different distribution to both TMax and SWE-Chat, and that user tasks seem to focus mostly on software engineering tasks over e.g. data processing tasks. This suggests that focussing more on software engineering tasks could be useful for real-world users, although keeping a diverse set of tasks is useful for evaluations like Terminal Bench. We also note that one can easily vary the distribution of the dataset in our pipeline by simply adjusting the sampling weights when choosing axis values, and that a uniform prior is a fairly reasonable diversity-maximising choice.

AuthorsOpenReview ↗

Follow up to Reviewer JEiv

Thank you again for your detailed and thoughtful review. As the discussion period closes today (Nov 18), we wanted to kindly follow up.

Your questions led to several new results / analyses:

  1. Comparisons against models trained on other RL datasets on SWE-Bench Verified and AIME (Table 4).
  2. A post-repair audit of TMax-13K.
  3. Corrected 27B results with a pass-rate analysis (Appendix F.3).
  4. Collapse diagnostics and a GRPO comparison (Appendix E.6).
  5. A comparison with real user tasks (Appendix F.4).

We would really appreciate it if you could take a look at our response. If it addresses your concerns, we would be grateful if you would consider raising your score. We are happy to clarify anything before the deadline!

Thank you for the detailed rebuttal and additional experiments. The cross-dataset evaluations together with the additional RL stability diagnostics substantially address several of my original concerns. Remaining questions:

  1. I remain concerned about the reliability of the learning signal provided by TMAX-13K. approximately 34% of the final TMAX-13K environments are still not classified as clean, including around 21% with overly weak verifiers. Could the authors clarify the implications of this residual error rate for reward reliability?
  2. The first concern may also be related to the limited gains observed for the 27B model, the improvement on TB2.1 is only 0.3 percentage points, substantially smaller than the reported evaluation uncertainty. Could the authors clarify whether the gains for 27B are statistically distinguishable from noise?

I appreciate the authors' additional efforts, but I decide to maintain my original score.

Reviewer_k8wNOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents TMax, an open recipe for training small terminal agents with RL. It has two parts:

  1. TMax-13K, a dataset of about 13.5K synthetic terminal tasks. The tasks are built by sampling across nine axes (domain, skills, persona, difficulty, verifier type, etc.), and then an LM judge flags and repairs weak or leaky verifiers.
  2. A simple outcome-only RL setup: DPPO, an FP32 LM head and larger group sizes for stability.

TMax-9B reaches 33% on Terminal-Bench 2.0, the best reported result for models under 10B parameters. Trained on the same recipe, TMax-13K matches or beats a dozen prior terminal datasets. The gains hold from 2B to 27B, carry over to Qwen 3, and show up on other harnesses and on SWE-bench Verified and AIME. The dataset and training recipe are both very valuable assets to the community, that's why I highly recommend acceptance.

Critical strengths
  1. Valuable and strong open recipes. RL for terminal agents has had few simple, reproducible open recipes. Previous works have mostly focused on Qwen3-8B which is much weaker in coding/terminal tasks, where this paper still show improvement upon Qwen3.5, that starts stronger in both capability. The authors compare against many prior datasets under one fixed training setup (Table 2) and plan to release data, code and checkpoints. These assets have already contributed a lot within the community. That makes this a solid starting point for academic work, and it is the main reason I lean toward acceptance.
  2. Careful dataset design with supporting evidence. The explicit difficulty and diversity controls, plus the repair pass, are well motivated. The repair ablation (Table 6), the balance and pass@k analysis (Table 1, Fig. 6), the self-generated data experiment and the cross-harness results (Table 5) all support the claims.
Critical weaknesses

Overall I think this paper itself is already well enough for the scope of a conference paper, I wrote the following just because the review require weakness. This doesn't mean I recommend reject or think the author should fix it within this paper. Instead I would like to hear the authors' thought on this only.

  1. It's unclear whether the recipe scales beyond small models, and the difficulty control is never really used. Gains shrink as the base model gets stronger. At 27B, TB-Lite slightly drops and TB 2.1 improves only modestly. The pipeline's main selling point is explicit difficulty control, and Section 3.1 says per-bucket up-weighting can "match specific model capabilities or induce a curriculum". But every model, from 2B to 27B, is trained on the same uniform mix, so the paper never shows that this control produces more useful data when the policy needs harder tasks. A curriculum, or difficulty matched to model size, seems like the natural fix for the 27B result, and it would show whether the recipe stays useful as base models get stronger.
Questions for authors
  1. Clarification: Why do you think the 27B model regresses on TB-Lite? Is it task difficulty, saturation of the stronger Qwen 3.6 base, or something else? Did you try up-weighting the harder complexity buckets, or a difficulty curriculum, for the larger models?
  2. Future work: Following the previous question, how do you see this pipeline scaling to tasks hard enough to train frontier models? Is a strong generator plus compositional sampling enough, or would it need more realistic data, such as real repositories, real user workflows or other external sources?
  3. Clarification: Do you have results at smaller dataset sizes (e.g., 3K or 6K tasks)? It would help show whether generating more data keeps helping or has already plateaued.
3 comments

We thank the reviewer for their careful reading and positive review, especially for noting that “this paper is already well enough for the scope of a conference paper”! We address your questions and concerns below:

Why do you think the 27B model regresses on TB-Lite? Did you try up-weighting the harder complexity buckets, or a difficulty curriculum, for the larger models?

We first note that we found a small bug in our 27B evaluation (forgetting to set preserved thinking to true), and fixing it we find that we do get improvements on both TB2 and TB Lite at the 27B scale:

ModelTB-LiteTB2
Qwen 3.6 27B74.2 ± 1.543.6 ± 1.9
TMAX-27B77.4 ± 1.843.9 ± 1.6

However, these gains are indeed smaller than with the 4B or 9B models. We do further investigation (see App. F.3 in the revised copy) and find that the 27B model does improve on many tasks, but also degrades on a substantial number due to it losing some ability to self-verify results. We leave investigating why this occurs, and generating more complex tasks to train on, to future work. We also note that in some ways our RL training results in an implicit curriculum as we filter completely solved or completely failed prompts via active sampling - as such, as the model improves, easier prompts become fully solved and are filtered out, so training naturally shifts towards harder prompts. Incorporating further improvements such as dynamic group sizes could further help for training on hard tasks [1], which we leave to future work.

[1] Learning to Solve Hard Problems in RL for LLMs by Never Giving Up, Noukhovitch et al. 2026. https://arxiv.org/abs/2609.13443.

It's unclear whether the recipe scales beyond small models. How do you see this pipeline scaling to tasks hard enough to train frontier models?

We did not examine larger models as our aim with this work is to show a recipe that allows working on terminal agents at the smaller scale. We consider how to develop even more complex tasks as an area for future work, e.g. incorporating existing repositories or tasks into the generation pipeline, or iterative task generation.

We also think that improving the dataset generator (i.e., using larger or more recent frontier models) could make generating more complex tasks easier, especially since Qwen 3.5 9B is able to design environments that improve its own performance. As such, if we work with frontier models, their improved capabilities may transfer well to RL environment generation, and so allow our setup to scale.

Do you have results at smaller dataset sizes (e.g., 3K or 6K tasks)?

We compare to a small dataset of ~3k samples generated by Qwen 3.5 9B in Table 6, although it is unclear if the drop in performance is due to Qwen 3.5’s lower capabilities or the smaller dataset size. However, this does show that smaller datasets can provide significant improvements in performance.

The authors’ response fully addressed my concerns, and thank you for the additional analysis. I continue to strongly recommend acceptance. I disagree with the assessment in the other two reviews, particularly Reviewer HVwg’s assessment of novelty. Its review is extremely low quality and seems like last minute review. The paper’s contribution is not just a new dataset: it provides a practical recipe for training small terminal agents. I encourage the ACs to reconsider and ignore its reviews. Overall, I believe this paper is a strong contribution to ICLR and should be accepted.

AuthorsOpenReview ↗

Thank you to Reviewer k8wN

Thank you for your continued support and strong recommendation!

We are glad our response addressed your concerns. Your careful questions led to the additional 27B analysis in Appendix F.3, and we will keep exploring difficulty curricula and harder task generation in future work. Cheers

-- Authors of "TMax"

Reviewer_HVwgOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper describes a dataset called Tmax-13k, which contains RL environments for diverse domains, skills and difficulty levels. The dataset is designed to be challenging for terminal use tasks. The authors then use this dataset to train open-weight terminal agents and get good scores for small models on TB 2.0. The authors release their recipe and their datasets, and then show the approach can generalize across harnesses and tasks.

Critical strengths

Terminal skills are important, and the community needs more RL environments to use to train models for them. I also appreciated that the authors did post-train a variety of Qwen models to show their environments are helpful.

Critical weaknesses

My biggest concern with this paper is that I don't think the approach is novel enough, nor do I think the results are strong enough for ICLR. This paper is a great tech report that describes good work, and I think this paper needs to exist so people can cite the dataset when they use it. I just don't see it as a research advance that moves the field forward. There's not enough insight here. So we're left with the results. Which although decent, don't seem to redefine the field. TB2.1 is pretty old at this point.

Questions for authors

Can you bring out the novelty in this work more? Have you tried on TB4.0?

2 comments

We thank the reviewer for their review, especially for noting that our paper is good work that deserves to exist. However, we respectfully disagree that our paper does not contain novel elements, nor that it is not fit for a conference paper.

Novelty

First, we believe our work contains a number of novel elements around data generation and RL training, and we view the development of a SOTA recipe for small-scale models as an important academic contribution. Our novel findings include:

  • We show that explicit validation of RL envs is not required for data generation. Previous methods relied on teacher validation to verify RL envs, which is expensive and less scalable.
  • A stable recipe for training terminal agents, including novel aspects like a larger group size and using DPPO for improved stability.
  • We show models can design their own coding environments to self-improve on terminal tasks, with Qwen 3.5 9B constructing environments that improve its own performance.
  • We show that RL on one harness in our setup generalises across tasks and harnesses, as opposed to gains being limited to the harness. We observe that our data generalises to OOD tasks (AIME, SWE-Bench Verified) better than baseline datasets.
  • While Terminal-Bench 2 may be considered saturated by frontier models, it came out in January 2026 (https://arxiv.org/abs/2601.11868), this calendar year, and we are the first to show significant gains on these styles of tasks at small (2B, 4B) model sizes, enabling further academic development.

We have adjusted our introduction to highlight these novel findings more.

Second, we note that the ICLR CFP explicitly states ‘datasets and benchmarks’ as being a subject area of interest (https://iclr.cc/Conferences/2027/CallForPapers). We view our work as being a contribution similar to those asked for in the NeurIPS datasets and benchmarks track, which explicitly asks for new datasets to enable novel research (e.g., in the space of small-model terminal agents, which our dataset and work enables).

Results on TB4

Following your suggestion, we evaluated on Terminal-Bench 4.1. Full task completion was rare, so we report partial credit as the average percentage of tests passed. If we look at the percentage of test cases passed, TMax training results in significant improvements. We find this especially striking considering that TB4.1 requires much longer horizons than used during TMax training (200 turns and 131k context lengths):

Base modelBase scoreAfter TMAX RLGain (pp)
Qwen 3.5 2B16.1 ± 0.629.7 ± 0.9+13.6
Qwen 3.5 4B27.5 ± 1.336.3 ± 1.0+8.8
Qwen 3.5 9B28.7 ± 1.239.1 ± 0.8+10.4
Qwen 3.6 27B41.4 ± 1.545.2 ± 1.3+3.8

We have added these results to Table 3 with further details in Appendix F.5. We also provide additional evaluations, analyses, and results in our response to reviewer JEiv, which further show that our recipe and data generalise to evaluations beyond TBLite/TB2, and provide further insights into agentic post-training.

AuthorsOpenReview ↗

Follow Up from Authors to Reviewer HVwg

Thank you again for your review, and for recognizing the value of our dataset and training recipe to the community. As the discussion period closes today (Nov 18), we wanted to kindly follow up.

Following your suggestion, we evaluated on Terminal-Bench 4.1, where TMax RL improves 3.8-13.6 points across all four model sizes, and we revised our introduction to highlight our novel findings. We would greatly appreciate it if you could take a look at our response.

If it addresses your concerns, we would be grateful if you would consider updating your score. We are happy to clarify anything before the deadline.

-- Authors of "TMax"

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Rebuttal Update

We thank the reviewers for all their helpful reviews and feedback. We especially thank the reviewers for noting:

  • Our recipes contribute important data and setups in an area with few simple setups (HVwg, k8wN).
  • Our recipe is simple, well-designed, and well-motivated (k8wN, JEiv), with ablations providing evidence for both our dataset design choices and training stability choices (JEiv).
  • We consistently validate our setups across multiple model sizes and families, and across multiple harnesses and tasks (JEiv)

We thank reviewer HVwg for saying our paper is a “great tech report”, and reviewer k8wN for saying that “[the] paper itself is already well enough for the scope of a conference paper”.

We have updated our paper in a number of ways to address reviewer concerns, with changes highlighted in blue to make them clear. We summarise some of the changes below:

  • Improved 27B results. We fix a minor chat template issue found when evaluating Tmax-27B and 2B (turning on preserved thinking). This improves both models’ scores, such that TMax training now improves the model even at the 27B scale. We also further analyse the 27b training dynamics (App. F.3).
  • Validating repair quality. We re-classify Tmax-13k after the repair stage and find that most instances are fixed by the repair stage. About 69% of tasks originally flagged for weak verifiers were judged clean after repair. Table 10 and Appendix C.2.
  • More baseline evaluations. We evaluate models trained on baseline datasets on AIME and SWE-Bench Verified, and find that TMax-9B had the highest observed AIME score (93.3%) and ranked second to SWE-Smith on SWE-Bench Verified (55.3% vs. 57.8%). We consider this strong performance as SWE-Smith is explicitly designed for performance on SWE-Bench Verified. Tables 4 and 16.
  • Terminal Bench 4.1 Evaluation. We evaluate 4 TMax differently-sized models on TB4.1 and find all tested model sizes improved by 3.8 to 13.6 percentage points. Table 3 and Appendix F.5.
  • Additional analyses. In the appendix, we provide further detailed analyses of our dataset composition (F.4), training instabilities (E.6), 27B training and performance (F.3).

We believe these edits have improved the work and have addressed all reviewer concerns as best we can. We are happy to address any further questions or concerns in the remaining rebuttal period and thank the reviewers again for their time and feedback in helping to improve our work!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.