acceptodds
Under review as a conference paper at ICLR 2027

TraceGym: Failure-Driven Environment Synthesis for Training Terminal Agents

Abstract

LLM agents are increasingly used to solve tasks in terminal environments, yet they still make frequent mistakes. These failures are a valuable learning signal, because the same kinds of errors, such as declaring success without checking the output, recur across otherwise unrelated tasks. Prior work uses such failures as feedback at inference time, fine-tunes on corrected trajectories, or synthesizes new tasks around the capability that failed. However, none of them makes the agent revisit the situation in which it erred and learn to act correctly there. We introduce **TraceGym**, a fully automated pipeline that turns an agent's failures into executable training environments, each placing the agent back in a situation where it made a wrong decision so that it can learn the right one. It has three stages. *Failure diagnosis* distills the agent's errors into *failure contracts*, reusable specifications of a wrong decision and its correct alternative. *Environment synthesis* recreates each contract in tasks drawn from a separate training pool, through a minimal change that leaves the task's objective and verifier intact. *Training* then uses this existing verifier as the only reward, so no new reward design is needed. From failures of Qwen3.6-27B on Terminal-Bench 2.1 and a task pool disjoint from the benchmark, TraceGym produces 225 training environments. Training the same model on these environments, without a stronger teacher, raises its Terminal-Bench 2.1 success from 48.8% to 56.2%, compared with a 4.4-point gain reported for reinforcement learning on the same pool. The gain generalizes to TerminalWorld-Verified, an out-of-distribution benchmark, where success rises from 32.0% to 37.5%.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, failure-conditioned environment generation, terminal agents, reinforcement learning, synthetic training environments, agent failure diagnosis

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sy5JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents TraceGym, a failure-driven environment synthesis framework for training terminal agents. It diagnoses recurring errors in agent trajectories and abstracts them into failure contracts specifying erroneous decisions, desired alternatives, and applicability conditions. These contracts guide the insertion of bounded textual artifacts into tasks from a separate training pool, while preserving the original task instructions and verifiers. Environments are filtered through static validation, execution checks, and failure-code matching. The main model is then trained using rejection fine-tuning on its own successful trajectories, followed by curriculum GRPO with task-verifier rewards. Using 225 constructed environments, TraceGym improves Qwen3.6-27B from 48.8% to 56.2% on Terminal-Bench 2.1, which also supplies the initial failure evidence, and from 32.0% to 37.5% on TerminalWorld-Verified. Additional experiments demonstrate improvements on Qwen3.5-4B and analyze changes in failure patterns and curriculum sampling efficiency.

Critical strengths
  1. Clear motivation and a structured formulation connecting failure diagnosis to training environment construction.
  2. Promising improvements across two benchmarks, with transfer to a smaller model.
  3. Informative failure-mode and curriculum analyses, supported by detailed reporting of training settings and computational costs.
Critical weaknesses
  1. Limited novelty and unclear empirical advantage. The pipeline substantially overlaps with related work such as TRACE. The added value of failure contracts and the performance advantage over existing approaches are not established through controlled comparisons.

  2. Missing training-stage ablation. The paper does not report RFT-only results, making it difficult to distinguish the contributions of RFT and subsequent GRPO training.

  3. Insufficient validation of targeted failure correction. Comparisons against unmodified source tasks are missing, and targeted errors do not show a statistically significant greater reduction than non-targeted errors. The specific benefit of failure-targeted environment construction therefore remains unclear.

Questions for authors
  1. Could you report RFT-only performance on both benchmarks to quantify the incremental contribution of GRPO?

2.Could you compare training on the original source tasks with training on their contract-modified versions under identical training settings? Does the modification improve benchmark performance and specifically reduce the targeted errors?

2 comments
AuthorsOpenReview ↗

Response to Reviewer sy5J

Thank you for the clear summary and focused questions. Following your suggestions, the revision adds RFT-only results, the original-task control and the uniform-curriculum control, and corrects a compute figure in App. D (changes in blue). We reply point by point. Below, TB2.1 = Terminal-Bench 2.1 and TW = TerminalWorld-Verified. The submitted TW numbers (27B: 32.0 → 37.5) were obtained with an incorrect evaluation setting; all TW numbers below come from the corrected re-run (Base and trained models alike).

W1. Novelty and the added value of failure contracts. You note overlap with TRACE and that the added value of failure contracts lacks a controlled comparison.

TRACE addresses the same problem with opposite design choices:

TRACETraceGym
Failures becomea new programmatic environment per capabilityone informational file added to an existing verified task
Rewardcapability-specific shaped rewardthe source task's own binary verifier; no reward design
Policyone LoRA per capability, composed by a learned MoE (v2)single full-parameter policy; inference unchanged
Evaluated onτ²-Bench, SWE-bench VerifiedTerminal-Bench 2.1, TerminalWorld-Verified

Revised §2 adds this contrast and SkillRL. For the added value of the contracts (instantiated as the added files), we added controls trained with the same procedure and rollout count (Table 3; Δ = TraceGym-27B − control):

Design choiceControlTB2.1 ΔTW Δ
Added file (contract instantiation)unmodified source tasks+4.3+3.0
Success-adaptive curriculumuniform sampling+2.2—

Both controls score below TraceGym-27B, consistent in direction; individual differences are within trial-to-trial variation, and the revision states this. In addition, with the same base model, task pool and protocol, TraceGym-27B is 5.8 points above Tmax-27B (56.2% vs. 50.3%). Revision: §2 (TRACE, SkillRL); §5.3, Table 3 (new); §5.6 (uniform curriculum); §1 (contributions) and §6.


W2 / Q1. RFT-only performance on both benchmarks. You ask for RFT-only results on both benchmarks to separate the contributions of RFT and GRPO.

We added RFT-only results on both benchmarks:

ModelTB2.1 (%)TW (%)
Qwen3.6-27B (Base)48.844.5
RFT only51.946.0
TraceGym-27B (RFT + GRPO)56.250.0
RFT over Base+3.1+1.5
GRPO over RFT+4.3+4.0

Both stages contribute, and GRPO adds more on both benchmarks (+4.3 / +4.0). The RFT arm is from a separate evaluation batch with aggregate counts; CIs will follow with the per-task records. Revision: §5.3, Table 3.


W3 / Q2. Unmodified source tasks; reduction of targeted errors. You ask for original vs. modified tasks under identical settings, and whether the modification improves performance and specifically reduces targeted errors.

Following your design, we added the original-task control. Evidence for the three sub-questions:

QuestionEvidenceStatus
Does the modification improve benchmark performance?Unmodified source tasks 51.9 / 47.0 vs. TraceGym-27B 56.2 / 50.0 (TB2.1 / TW)yes, higher on both (+4.3 / +3.0); individual differences within variation
Does the added file change the task for the Base model?71.4% vs. 72.2% success; file read in 28.7% of rolloutsno, so the gain does not come from easier tasks
Are targeted errors reduced more than non-targeted ones?absolute 4.2 vs. 2.1 points; per-code intervals for codes B.8 (−9.0) and C.1 (−7.9) exclude zero (§5.4); relative 17.0% vs. 22.8% (App. C.2)consistent in direction, not significant

Targeted errors fall more in absolute terms, but the difference is within noise and compatible with a uniform reduction in failures; the revision states it this way. Revision: §5.3, §4.3.

We also thank you for noting our compute reporting. On re-checking, we found an error in App. D: RFT used 204 H200-hours, not 102, so the training total is 1,455; the revision corrects this.

Thank you again for your feedback. We hope these additions address your concerns; if so, we would be grateful if you would take them into account in your evaluation. We are happy to continue the discussion.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer,

As the discussion period is closing in a few hours, we would like to check whether our response and the revised paper have addressed your concerns. We are happy to answer any further questions. If they have, we would be grateful if you would consider updating your score.

Thank you again for your time and feedback.

Best regards, The Authors

Reviewer_du6fOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents TraceGym, a pipeline that turns agent failures into executable training environments. It abstracts recurring errors into reusable failure contracts specifying a wrong decision and its correct alternative, then instantiates these contracts in separate training tasks through bounded informational interventions while preserving the original instructions and verifiers. The agent is trained on the resulting environments using task-completion rewards to encourage better decisions.

Critical strengths
  1. The paper studies an important question of how to make better use of agent failures. Instead of directly training on failed trajectories, it identifies recurring mistakes and uses them to construct new training environments. The idea is clear and worth exploring.

  2. The paper is easy to follow, and the reported performance gain sounds encouraging.

Critical weaknesses
  1. I am concerned about using failures from Terminal-Bench to develop the method and then evaluating on the same benchmark. Even though the training tasks are different, information from the evaluation set still guides environment construction. It would be more convincing to derive the contracts from independent training tasks and then evaluate on Terminal-Bench. Another option is to split the benchmark into diagnosis and evaluation sets in advance and keep the evaluation set untouched.

  2. The paper does not compare training on the original tasks with training on the modified versions under the same initialization, RFT and GRPO procedure, and training budget. Without this comparison, it is hard to tell whether the gains come from the proposed interventions or simply from training on the selected tasks.

  3. It is also unclear whether the added text actually leads the agent to face the intended decision. Finding the same error label in a failed run does not show that the intervention caused it. I would like to see repeated runs on the same tasks, with and without the added files, comparing how often the targeted errors occur. Inspecting whether the agent reads the files and acts on their content would also help support the claimed mechanism.

Questions for authors

See Weaknesses.

3 comments
AuthorsOpenReview ↗

Response to Reviewer du6f

Thank you for the constructive review and the very concrete experimental suggestions. Following them, we added the evidence-task split, the original-task control and a paired with/without-file analysis, plus two coding benchmarks beyond Terminal-Bench 2.1 (TB2.1). Changes are in blue in the revised PDF. We reply point by point.

W1. Failure evidence from TB2.1 and evaluation on TB2.1. You are concerned that failure evidence comes from TB2.1, the evaluation benchmark, and suggest a held-out split or contracts from independent tasks.

This is a fair concern; §5.5 states that TB2.1 is not a fully clean test, and the revision now quantifies this. Following your suggestion, we split the 89 tasks by whether they supplied failure evidence, applying the target-selection rule of §3.1 to the diagnosis records (per-code task counts match the paper) and freezing the groups before looking at any per-group result. Two trials per task (the cohort analyzed in §5.4):

GroupTasksBase (%)TraceGym-27B (%)Difference [95% CI]
Supplied failure evidence4122.040.2+18.3 [8.5, 29.3]
Other tasks4867.769.8+2.1 [−4.2, +8.3]
Other tasks, Base not at ceiling1918.426.3+7.9 [−5.3, +23.7]

The gain is concentrated on evidence tasks, but the groups differ in headroom: Base solves 29 of the other 48 tasks in both trials, and on the remaining 19 tasks, where Base is not at ceiling, the point estimate is +7.9. Evidence beyond TB2.1:

BenchmarkBaseTraceGym-27BDifference
TerminalWorld-Verified (TW; 200 tasks, 1 attempt)44.5%50.0%+5.5
SWE-bench Verified (200 tasks, 1 attempt)76.5%76.5%0.0 [−4.5, +4.5]
Aider Polyglot (225 tasks × 3)47.9%49.0%+1.2 [−1.5, +3.9]

TW took no part in diagnosis or construction and improves by +5.5; on two coding benchmarks outside the terminal domain we see no regression. Revision: §5.5 reports the split; Table 5 (new) reports SWE-bench Verified and Aider Polyglot; App. C.3 gives the full table.


W2. Original vs. modified tasks under the same training. You suggest comparing training on original vs. modified tasks under the same initialization, RFT+GRPO procedure and budget.

We ran this control as designed: the same 225 source tasks without the added files, with the same initialization, RFT + GRPO procedure and rollout count.

Training tasksTB2.1 (%)TW (%)
Unmodified source tasks51.947.0
TraceGym environments (added file)56.250.0
Difference+4.3+3.0

The version with the added file scores higher on both benchmarks. Because the file does not change task difficulty for the Base model (71.4% vs. 72.2% success, W3), this difference, if real, comes from training rather than from easier tasks. The control arm is from a separate evaluation batch with aggregate counts; individual differences are within trial-to-trial variation, and we will add CIs once the per-task records are compiled. Revision: §5.3, Table 3.


W3. Does the added file make the agent face the intended decision? You ask whether the added file actually leads the agent to the intended decision, suggesting with/without-file runs and checks of whether the file is read.

Following your suggestion, we compared the two versions of each environment on Base rollouts (8 per environment and version):

MeasureValue
Environments whose source task differs from ours only by the file198 (186 with valid rollouts on both)
Rollouts that read the file (name or path in an executed command)28.7% [23.5, 34.1]
Rollouts whose terminal output shows the file name50.2%
"Read" detector on versions without the file0 / 1,580
Base success with vs. without the file71.4% vs. 72.2% (−0.8 [−2.8, +1.3])

About 30% of rollouts read the file and half see it in their terminal output; with or without the file, Base success is nearly identical, so the training gain is not because the tasks became easier. Revision: §4.3 reports the read rate and the with/without-file success rates, states diagnostic agreement as an observational check, and restricts the mechanism claim to rollouts that read the file.

Thank you again; these analyses make the paper's argument more complete. We hope they address your concerns; if so, we would be grateful if you would take them into account in your evaluation. We are happy to continue the discussion.

Thank you for the detailed responses. The additional experiment is clear, but I am still concerned about the overall novelty and the limited improvement on OOD tasks where failure evidence has not been collected. Also, I am not fully convinced by the assumption that simply adding a file tied to a failure label is enough to put the agent into that specific decision scenario. The reported file access rates are weak evidence for this point. Thus, I will maintain my original rating.

AuthorsOpenReview ↗

Response to Reviewer du6f's Follow-up

Thank you for the follow-up.

The added file and the intended decision. We do not claim that every rollout faces the intended decision; the revised §4.3 limits the mechanism to rollouts that read the file and treats diagnostic agreement as observational. The method rests on a controlled comparison instead: under the same training setup, environments with the file outperformed the same 225 source tasks without it (TB2.1 56.2 vs. 51.9; TW 50.0 vs. 47.0), while the file leaves Base success unchanged (71.4% vs. 72.2%). Each difference is within trial-to-trial variation, but both point the same way, and the gain does not come from easier tasks. We will align the abstract, introduction and conclusion with this scope.

Tasks without failure evidence. Concentrating the gain on evidence tasks is the intended behavior: TraceGym targets the failures it observes. The other tasks also leave little headroom: Base solves 29 of the 48 in both trials, and on the 19 where it does not, the gain is +7.9 [−5.3, +23.7]. Out-of-domain coding is not hurt (SWE-bench Verified 76.5 → 76.5; Aider Polyglot 47.9 → 49.0).

Novelty. Unlike TRACE, which builds a separate environment for each capability, TraceGym changes one bounded file in an already-verified task and keeps its instruction and verifier fixed.

Thank you again for the concrete suggestions.

Reviewer_WWT4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces a new failure-driven environment synthesis method for terminal agents called TraceGym. Prior methods separate training environments around lacking capabilities from failed tasks. Instead, TraceGym recreates each recurring failure directly in compatible tasks drawn from the training pool. By placing the agent back in the situation in which it made an error and rewarding it correctly for acting there, TraceGym demonstrates improvements on TerminalBench-2.1 (+7.4 percentage points). The resulting environments deliver strong heldout gains on a different environment (TerminalWorld-Verified) and model (Qwen3.5-4B).

Critical strengths
  1. The paper demonstrates strong improvements on Terminal Bench 2.1 (+7.4 percentage points), which is a significant gain on a challenging agentic benchmark. Also, these gains are achieved without requiring a stronger-model’s demonstrations for training.
  2. The paper demonstrates transferable gains across environments and models, improving performance on TerminalWorld-Verified which isn’t used for the failure diagnosis nor environment construction. In addition, the resulting environments from Qwen3.6-27B’s failures transfer to improving Qwen3.5-4B on TerminalWorld-Verified, which is an exciting finding.
Critical weaknesses
  1. The main limitation is methodological novelty and insufficient evidence that the proposed differences confer improvements over related failure-driven methods. For instance, compared to TRACE (Kang et. al, 2026), the main difference appears to be making bounded changes to existing tasks rather than designing separate capability-targeted training environments. However, the authors do not empirically validate the effectiveness of this design compared to TRACE as well as other methods such as TermiGen (Zhu et. al, 2026) and SkillRL (Xia et. al, 2026).
  2. The comparison against the existing baselines is incomplete and insufficiently controlled. For instance, the authors do not use the same harness (Terminus-2) and training method (i.e., RFT+GRPO) for all baselines and their proposed method. Moreover, the authors do not report whether baselines were given the same computation budget across the number of rollouts, GPUs, and wall clock time. Also, the proposed method lacks comparison against these baselines on the generalization experiments to TerminalWorld-Verified and Qwen3.5-4B.

References: [1] Kang, Hangoo, et al. "TRACE: Capability-Targeted Agentic Training." arXiv preprint arXiv:2604.05336, 2026. [2] Zhu, Kaijie, et al. "TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agents." arXiv preprint arXiv:2602.07274, 2026. [3] Xia, Peng, et al. "SkillRL: Evolving Agents via Recursive Skill-Augmented Reinforcement Learning." NeurIPS 2026, Main Conference Track, 2026.

Questions for authors

The paper’s methodology largely overlaps with existing failure-driven agent improvement methods, particularly TRACE (Kang et. al, 2026). The main difference seems to be introducing bounded modifications to existing tasks rather than designing separate capability-targeted training environments. Could the authors please empirically validate this design against TRACE and other failure-driven agent improvement methods, such as TermiGen (Zhu et. al, 2026) and SkillRL (Xia et. al, 2026)?

There is a lack of proper ablations against baselines. For instance, the Tmax baseline uses a different harness and training method compared to TraceGym. Moreover, the authors do not establish that they used the same wall clock time, number of rollouts, and GPU compute across all evaluated baselines. Could the authors please evaluate all baselines under the same harness, training method, and computational setting?

Furthermore, for the TerminalWorld-Verified experiments, the authors do not include generalization results of other baseline training methods and only compare the base model versus TraceGym. Could the authors please add the baselines’ generalization results?

Only 9% (225 of 2,500) of environments are actually used for training. Given the computation cost of the pipeline, could the authors please compare the end-to-end efficiency of the proposed environment synthesis pipeline against relevant failure-driven environment synthesis baselines?

4 comments
AuthorsOpenReview ↗

Response to Reviewer WWT4 (1/3)

Thank you for the careful review and the very specific questions. Your points on baseline comparability, generalization baselines and pipeline efficiency led directly to the same-protocol Tmax comparison, the single-factor controls and the pipeline-yield comparison in the revision (changes in blue). We reply point by point in three comments.

W1 / Q1. Comparison with TRACE, TermiGen and SkillRL. You note that our main difference from TRACE is bounded edits to existing tasks, and ask for an empirical comparison with TRACE, TermiGen and SkillRL.

We did not run a direct empirical comparison with these methods (reasons below the table). The table summarizes how they differ; revised §2 covers the main differences from TRACE and SkillRL.

TRACETermiGenSkillRLTraceGym
Failures becomea new programmatic environment per capabilityerror-injected teacher trajectories (errors from a predefined taxonomy)a skill libraryone informational file added to an existing verified task
Rewardcapability-specific shaped rewardnone (SFT only)task reward (GRPO)the source task's own binary verifier
Policyone LoRA per capability, composed by a learned MoE (v2)single modelsingle policy + skills retrieved into the promptsingle full-parameter policy; inference unchanged
Stronger model used forwriting environmentstrajectories (Claude-4.5-Sonnet)skill distillation and cold-start SFT data (o3)diagnosis and file generation (GPT-5.6-Sol); no demonstrations
Evaluated onτ²-Bench, SWE-bench VerifiedTerminal-Bench 1.0ALFWorld, WebShop, 7 search-QA tasksTerminal-Bench 2.1, TerminalWorld-Verified

TRACE addresses the same problem with opposite design choices: it builds a new environment, a shaped reward and a separate adapter per capability, whereas we reuse the verifiers of verified tasks and train a single policy with inference unchanged. TRACE has no terminal-task implementation, so a fair comparison would first require reimplementing it for Docker tasks. SkillRL is complementary: it retrieves skills into the prompt, while we change the training environments and add nothing at inference, so the two can be combined. TermiGen trains by SFT on teacher trajectories and evaluates on Terminal-Bench 1.0, whereas our setting uses no demonstrations. To test whether our design choices help, we added single-factor controls trained with the same recipe (Table 3; Δ = TraceGym-27B − control; TB2.1 = Terminal-Bench 2.1, TW = TerminalWorld-Verified):

Design choiceControlTB2.1 ΔTW Δ
Added fileunmodified source tasks+4.3+3.0
Success-adaptive curriculumuniform sampling+2.2—
GRPO stageRFT only+4.3+4.0

All three controls score below TraceGym-27B on every benchmark where they were evaluated; individual differences are within trial-to-trial variation, and the revision states this. Revision: §2 adds SkillRL and the TRACE contrast; §5.3 and Table 3 (new) report the controls, §5.6 the uniform-curriculum control; §1 (contributions) and §6 are updated (§6 leaves a direct comparison with other methods to future work); "without a stronger teacher" is now "without teacher demonstrations".

AuthorsOpenReview ↗

Response to Reviewer WWT4 (2/3)

(Continued.)

W2 / Q2. Same harness, training method and compute. You note that the Tmax baseline uses a different harness and training method, and ask whether rollouts, GPUs and wall-clock time were matched.

We added the following controlled comparisons:

ConcernWhat we didResult
HarnessRe-evaluated Tmax-27B with Terminus-2 under our TB2.1 protocol (same base model and task pool, TMax-15K, as TraceGym-27B)Tmax-27B 50.3% vs. TraceGym-27B 56.2% (+5.8). Under its own harness, Tmax reports 40.5 → 44.9 (+4.4)
Training methodOur RFT + GRPO recipe and rollout budget on the same 225 source tasks from TMax-15K, without the added files51.9% vs. 56.2% (+4.3); TW 47.0% vs. 50.0%
RolloutsIdentical across RL controls1,800 base rollouts, 1 RFT epoch, 7 GRPO rounds × 225 groups × 8 = 12,600 rollouts
GPU hours (ours)Measuredtraining 1,455 H200-h (corrected RFT row); learner screening 1,163 H200-h
GPU hours (Tmax)Cited from its paperH100 nodes, typically 2 for training and 6 for inference, 2–3 days; 300 steps for Qwen3.6-27B

Tmax-27B is the only training method that shares our base model and task pool, so we re-evaluated it under our protocol; FACET and RST start from a different base model and remain listed with their published numbers for context. With the same base model, task pool, harness and protocol, TraceGym-27B is 5.8 points above Tmax-27B, and 4.3 points above the control that differs only by the added files. The Tmax-27B arm comes from a separate evaluation batch (aggregate counts); CIs will follow with the per-task records. Revision: Table 2 adds Tmax-27B under our protocol; §5.2 now compares methods under the same protocol instead of across papers ("+7.4 vs. +4.4"); §5.3, Table 3; App. C.1.

Q3. Baselines for TerminalWorld-Verified and Qwen3.5-4B. You note that the TW and Qwen3.5-4B generalization results compare only Base and TraceGym.

The revision adds Tmax baselines on TW at both scales:

ModelTB2.1 (%)TW (%)Source
Qwen3.6-27B (Base)48.844.5our protocol
Tmax-27B50.347.0our protocol
TraceGym-27B56.250.0our protocol
Qwen3.5-4B (Base)17.218.0our protocol
Tmax-4B19.9 (+5.7 over 14.2)21.3TB2.1: model card, own harness; TW: our protocol
TraceGym-4B20.2 (+3.0)22.5 (+4.5)our protocol

TW is one attempt per task for 27B and the mean of three attempts for 4B; all TW numbers come from the re-run with the corrected evaluation setting (Base and trained models alike). On TB2.1, Tmax-4B reports a larger gain under its own harness (+5.7, versus +3.0 for TraceGym-4B under ours); on TW, under the same protocol, TraceGym-4B (22.5%) is on par with Tmax-4B (21.3%). Our 4B environments were built from the 27B model's failures, not the 4B model's own. Revision: Table 4 (TW corrected for both sizes; Tmax-27B and Tmax-4B added), Table 2 (Tmax-4B), §5.5.

AuthorsOpenReview ↗

Response to Reviewer WWT4 (3/3)

(Continued.)

Q4. Share of environments used for training; end-to-end efficiency. You note that only 9% of 2,500 environments are used for training and ask for an end-to-end efficiency comparison.

2,500 is the number of construction attempts (250 per code), not environments; the 9% reflects the selection criterion of keeping only environments in which the current learner actually commits the designated error (share of attempts in parentheses):

Construction attemptsPass static checksRuntime-validLearner fails in one rolloutRe-diagnosis matches intended codeUsed for training
2,5002,110 (84.4%)1,946 (77.8%)438 (17.5%)231 (9.2%)225 (9.0%)
PipelineValidity yieldLearner-relative retentionReported cost
FACET77.4% (6,078 / 7,852 seeds, computed from its App. Table 5)——
Terminal-World83.1% (environment construction success)— (LLM-judge and verifier checks)$0.17 / trajectory
RST≈50–57% per attempt— (reference solution passes verifier)≈$0.05 / accepted task
CalibForge—19% initially; 96% after up to 50 revision rounds—
SETA—560 of 4,567 (12.3%), after removing environments too hard for the base model—
RIVER—< 30% of TMax environments—
TraceGym84.4% static, 77.8% runtime11.9% of runtime-valid (231 / 1,946) produce the designated error in one rolloutper environment 130K input (105K cached) + 4.4K output tokens; learner screening 1,163 H200-h (≈44% of construction and training GPU hours)

Our static and runtime validity rates (84.4% / 77.8%) are comparable to FACET and Terminal-World. As far as we can find, the failure-driven methods (TRACE, SkillRL, WebRL, HardGen) do not report yield, generation tokens or GPU hours, so an end-to-end comparison from published numbers is not possible; in the table above, only we report per-stage yield, generation tokens and GPU hours together. The lower retention comes from the stricter requirement that the current learner commit the designated error; revising rejected environments, as CalibForge does, could raise the yield further. Revision: App. A.5 (new).

Thank you again for your feedback. We hope these additions address your concerns; if so, we would be grateful if you would take them into account in your evaluation. We are happy to discuss any further questions during the discussion period.

AuthorsOpenReview ↗

Kind Reminder

Dear Reviewer,

As the discussion period is closing in a few hours, we would like to check whether our response and the revised paper have addressed your concerns. We are happy to answer any further questions. If they have, we would be grateful if you would consider updating your score.

Thank you again for your time and feedback.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response (to all reviewers)

We thank the three reviewers for their careful reading and constructive feedback. We have uploaded a revised PDF with all changes in blue. The main additions are single-factor controls trained with our recipe and a Tmax baseline evaluated under our protocol: every control scores below TraceGym-27B (second table below).

Summary of changes

ChangeWhereRequested by
Single-factor controls; Tmax-27B re-evaluated under our protocol§5.3, Table 3; Table 2, §5.2all
TerminalWorld-Verified (TW) corrected and re-run; Tmax-27B and Tmax-4B baselines addedTable 4, §5.5WWT4 (baselines)
Gain split by failure-evidence tasks; SWE-bench Verified, Aider Polyglot§5.5, Table 5, App. C.3du6f
Paired with/without-file analysis§4.3du6f, sy5J
Pipeline yield and cost vs. other pipelinesApp. A.5WWT4
Related work (SkillRL, TRACE) and wording fixes§1, §2, §6WWT4, sy5J
Compute table corrected (RFT: 204 H200-hours)App. Dsy5J

New controls (revision Table 3; success %, our protocol; TB2.1 = Terminal-Bench 2.1)

ArmChangeTB2.1TW
Qwen3.6-27B (Base)no training48.844.5
RFT onlyno GRPO51.946.0
Unmodified source tasksadded file removed51.947.0
Uniform curriculumuniform sampling53.9—
Tmax-27B (Terminus-2, our protocol)different method, same base model50.347.0
TraceGym-27B—56.250.0

Except for Tmax-27B (same base model, task pool and evaluation protocol), each control is the TraceGym-27B recipe with one change. Controls come from a separate evaluation batch (CIs will follow with the per-task records), and individual differences are within trial-to-trial variation (single-trial Base: 46.1–55.1%), so the revision describes them as consistent in direction.

Correction. The TW numbers in the submitted Table 3 (27B: 32.0 → 37.5; 4B: 22.5 → 27.0) were obtained with an incorrect evaluation setting. After correcting it, the re-run of Base and trained models gives 27B 44.5 → 50.0 (+5.5, one attempt per task) and 4B 18.0 → 22.5 (+4.5, three attempts per task).

Details are in our replies to each reviewer. Thank you again for your time and feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.