- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents TraceGym, a failure-driven environment synthesis framework for training terminal agents. It diagnoses recurring errors in agent trajectories and abstracts them into failure contracts specifying erroneous decisions, desired alternatives, and applicability conditions. These contracts guide the insertion of bounded textual artifacts into tasks from a separate training pool, while preserving the original task instructions and verifiers. Environments are filtered through static validation, execution checks, and failure-code matching. The main model is then trained using rejection fine-tuning on its own successful trajectories, followed by curriculum GRPO with task-verifier rewards. Using 225 constructed environments, TraceGym improves Qwen3.6-27B from 48.8% to 56.2% on Terminal-Bench 2.1, which also supplies the initial failure evidence, and from 32.0% to 37.5% on TerminalWorld-Verified. Additional experiments demonstrate improvements on Qwen3.5-4B and analyze changes in failure patterns and curriculum sampling efficiency.
Critical strengths
- Clear motivation and a structured formulation connecting failure diagnosis to training environment construction.
- Promising improvements across two benchmarks, with transfer to a smaller model.
- Informative failure-mode and curriculum analyses, supported by detailed reporting of training settings and computational costs.
Critical weaknesses
-
Limited novelty and unclear empirical advantage. The pipeline substantially overlaps with related work such as TRACE. The added value of failure contracts and the performance advantage over existing approaches are not established through controlled comparisons.
-
Missing training-stage ablation. The paper does not report RFT-only results, making it difficult to distinguish the contributions of RFT and subsequent GRPO training.
-
Insufficient validation of targeted failure correction. Comparisons against unmodified source tasks are missing, and targeted errors do not show a statistically significant greater reduction than non-targeted errors. The specific benefit of failure-targeted environment construction therefore remains unclear.
Questions for authors
- Could you report RFT-only performance on both benchmarks to quantify the incremental contribution of GRPO?
2.Could you compare training on the original source tasks with training on their contract-modified versions under identical training settings? Does the modification improve benchmark performance and specifically reduce the targeted errors?
2 comments
Kind Reminder
Dear Reviewer,
As the discussion period is closing in a few hours, we would like to check whether our response and the revised paper have addressed your concerns. We are happy to answer any further questions. If they have, we would be grateful if you would consider updating your score.
Thank you again for your time and feedback.
Best regards, The Authors
Response to Reviewer sy5J
Thank you for the clear summary and focused questions. Following your suggestions, the revision adds RFT-only results, the original-task control and the uniform-curriculum control, and corrects a compute figure in App. D (changes in blue). We reply point by point. Below, TB2.1 = Terminal-Bench 2.1 and TW = TerminalWorld-Verified. The submitted TW numbers (27B: 32.0 → 37.5) were obtained with an incorrect evaluation setting; all TW numbers below come from the corrected re-run (Base and trained models alike).
W1. Novelty and the added value of failure contracts. You note overlap with TRACE and that the added value of failure contracts lacks a controlled comparison.
TRACE addresses the same problem with opposite design choices:
Revised §2 adds this contrast and SkillRL. For the added value of the contracts (instantiated as the added files), we added controls trained with the same procedure and rollout count (Table 3; Δ = TraceGym-27B − control):
Both controls score below TraceGym-27B, consistent in direction; individual differences are within trial-to-trial variation, and the revision states this. In addition, with the same base model, task pool and protocol, TraceGym-27B is 5.8 points above Tmax-27B (56.2% vs. 50.3%). Revision: §2 (TRACE, SkillRL); §5.3, Table 3 (new); §5.6 (uniform curriculum); §1 (contributions) and §6.
W2 / Q1. RFT-only performance on both benchmarks. You ask for RFT-only results on both benchmarks to separate the contributions of RFT and GRPO.
We added RFT-only results on both benchmarks:
Both stages contribute, and GRPO adds more on both benchmarks (+4.3 / +4.0). The RFT arm is from a separate evaluation batch with aggregate counts; CIs will follow with the per-task records. Revision: §5.3, Table 3.
W3 / Q2. Unmodified source tasks; reduction of targeted errors. You ask for original vs. modified tasks under identical settings, and whether the modification improves performance and specifically reduces targeted errors.
Following your design, we added the original-task control. Evidence for the three sub-questions:
Targeted errors fall more in absolute terms, but the difference is within noise and compatible with a uniform reduction in failures; the revision states it this way. Revision: §5.3, §4.3.
We also thank you for noting our compute reporting. On re-checking, we found an error in App. D: RFT used 204 H200-hours, not 102, so the training total is 1,455; the revision corrects this.
Thank you again for your feedback. We hope these additions address your concerns; if so, we would be grateful if you would take them into account in your evaluation. We are happy to continue the discussion.