- Rating
- 2
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper presents ReTerm, a recovery-oriented training framework designed to improve terminal agents' ability to recover from execution failures in long-horizon tasks. The framework consists of two main components. First, the authors construct ReTerm-10K, a dataset of successful recovery trajectories generated by having a stronger teacher model continue from the terminal environment states of failed attempts produced by a weaker base model. These trajectories are filtered and used for SFT. Second, the authors propose ReTerm-GRPO, which extends standard GRPO by constructing recovery tasks from failed rollouts generated by the evolving policy and jointly optimizing original and recovery task groups. A recovery task queue is introduced to reuse failure states across training iterations.
Experiments using Qwen3.5-9B demonstrate improvements across several benchmarks, including 34.8% Pass@1 on Terminal-Bench 2.0 and 58.4% on SWE-bench Verified, outperforming the base model by 14.6 and 13.2 percentage points, respectively. Ablation studies show that ReTerm-GRPO improves Terminal-Bench 2.0 performance from 28.0% with standard GRPO to 34.8%. Additional evaluations across agent harnesses, out-of-domain benchmarks, and different error categories provide further evidence of the effectiveness and generalization of the proposed approach.
Critical strengths
- Strong empirical performance on terminal agent benchmarks.
The proposed framework achieves substantial performance improvements on challenging terminal agent benchmarks. In particular, ReTerm-9B improves Terminal-Bench 2.0 performance from 20.2% to 34.8%, with both recovery-oriented SFT and ReTerm-GRPO contributing meaningful gains. The improvements on SWE-bench Verified and OpenThoughts-TBLite further demonstrate the practical effectiveness of the proposed training framework.
- Comprehensive experimental evaluation.
The paper provides a relatively comprehensive evaluation, including ablation studies, cross-harness generalization, out-of-domain benchmarks, and recovery analysis across different error categories. These experiments help establish that the observed improvements are not limited to a single evaluation setting and provide useful evidence for the effectiveness of recovery-oriented training.
Critical weaknesses
- Limited methodological novelty.
The overall methodology appears relatively straightforward. The SFT stage follows a natural pipeline of collecting failed trajectories, using a stronger teacher model to generate successful recovery continuations, and filtering them for supervised learning. Similarly, ReTerm-GRPO mainly augments standard GRPO with recovery tasks constructed from failed rollouts, while retaining the original GRPO objective and advantage estimation. The recovery task queue also appears to be a relatively standard sampling mechanism. While these design choices are reasonable and practically effective, the paper does not clearly establish what constitutes a substantial methodological advance beyond existing recovery-oriented training and standard RL techniques.
- Insufficient analysis of the sources of performance improvements.
Although the reported improvements are substantial, the current experiments do not sufficiently isolate the factors responsible for these gains. In particular, ReTerm-GRPO changes the training task distribution by introducing recovery tasks, but the paper does not compare this strategy against alternative sampling approaches, such as training from ordinary intermediate states or oversampling difficult original tasks. It is therefore unclear whether the improvements arise specifically from learning to recover from failures or from more general benefits of task distribution augmentation. Similarly, the paper does not compare recovery-specific SFT data against conventional successful trajectories under matched settings, nor does it independently evaluate the contribution of the recovery task queue.
Questions for authors
- Fairness of the GRPO Comparison
Could the authors clarify whether the standard GRPO baseline uses the same number of rollouts, optimization steps, and overall training compute as ReTerm-GRPO? In particular, ReTerm-GRPO uses 8 original task groups and 2 recovery task groups per step. How is the standard GRPO baseline configured?
- Context Length and Computational Overhead
Since recovery rollouts are conditioned on the complete failed interaction history, could the authors report the typical length distribution of these histories and clarify how histories exceeding the context window are handled? How much additional computational overhead does this introduce?
- Attribution of Recovery Improvements
Since recovery rollouts receive a fresh generation and interaction budget after the original attempt has failed, could the authors clarify how they distinguish improvements in recovery capability from the benefits of additional execution opportunities? Is there a controlled comparison with standard GRPO under a comparable total execution budget?
4 comments
Q3: The paper does not compare recovery-specific SFT data against conventional successful trajectories under matched settings.
A3: Thanks for your comments. We have added an SFT control experiment using the same task set and sample count. We subsample 800 recovery trajectories from ReTerm-10K. For the same task set, we ask the teacher model to execute directly from the initial task environments and use rejection sampling to obtain 800 ordinary trajectories. Both models are initialized from the same Qwen3.5-9B base model and fine-tuned with identical settings: 2 epochs, a learning rate of , and a global batch size of 16. The results are as follows:
| SFT Data | Terminal-Bench 2.0 |
|---|---|
| Ordinary trajectories | 22.5% |
| Recovery trajectories | 24.7% |
Under these controlled settings, SFT on recovery trajectories improves Pass@1 by 2.2 points over SFT on ordinary trajectories, further supporting the effectiveness of recovery trajectories as fine-tuning data.
Q4: The paper does not independently evaluate the contribution of the recovery task queue.
A4: Thanks for your suggestion. We have added an ablation experiment removing the recovery task queue. In this setting, recovery tasks are constructed only from failure states generated by original tasks in the current training step and are not retained across steps. Each step still contains 8 original task groups and targets 2 recovery task groups. If the recovery groups are not filled, no additional sampling is performed, and training proceeds with the recovery groups actually obtained. The results are as follows:
| Method | Terminal-Bench 2.0 |
|---|---|
| ReTerm-GRPO | 34.8% |
| w/o recovery task queue | 30.3% |
Removing the recovery task queue reduces performance by 4.5 points, supporting the effectiveness of the recovery task queue in maintaining a stable supply of recovery tasks.
Q5: Could the authors clarify whether the standard GRPO baseline uses the same number of rollouts, optimization steps, and overall training compute as ReTerm-GRPO? In particular, ReTerm-GRPO uses 8 original task groups and 2 recovery task groups per step. How is the standard GRPO baseline configured?
A5: Thanks for pointing this out. The original manuscript did not fully specify the configuration of the standard GRPO baseline. We will explicitly include these details in the revised manuscript.
Standard GRPO and ReTerm-GRPO are initialized from the same SFT checkpoint and use the same number of training steps, optimizer, and learning rate. Standard GRPO replaces the two recovery task groups with two additional original task groups, using 10 original task groups per step. Each group contains 16 sampled rollouts, so both methods target 160 rollouts per training step. Both methods also use the same dynamic sampling strategy.
Q6: Since recovery rollouts are conditioned on the complete failed interaction history, could the authors report the typical length distribution of these histories and clarify how histories exceeding the context window are handled? How much additional computational overhead does this introduce?
A6: Thanks for raising this question. We have collected additional statistics on the lengths of the failed interaction histories:
| Statistic | Value |
|---|---|
| Mean length | 18,054 |
| Median | 14,753 |
| P90 | 34,705 |
| P95 | 42,175 |
| Maximum length | 56,812 |
| Histories shorter than 32K | 88.3% |
| Histories shorter than 48K | 98.0% |
Both original and recovery attempts use a 65K context window, and the inherited history counts toward this limit. We retain the history without additional summarization or sliding-window truncation. Once the context window is exhausted, generation stops and the attempt proceeds directly to evaluation.
Recovery training primarily introduces additional context-processing overhead for the inherited histories. With the same number of rollouts, ReTerm-GRPO has approximately 23.3% more cumulative input tokens than standard GRPO. To reduce repeated context computation, we enable prefix caching so that recovery attempts branching from the same failure state can reuse the KV cache of their shared prefix.
Q7: Since recovery rollouts receive a fresh generation and interaction budget after the original attempt has failed, could the authors clarify how they distinguish improvements in recovery capability from the benefits of additional execution opportunities? Is there a controlled comparison with standard GRPO under a comparable total execution budget?
A7: Thanks for your constructive comments. Within each training batch, both methods use the same number of rollouts, maximum generation length, and interaction-turn limit. Nevertheless, the original and recovery attempts together allow longer cumulative interaction on the same task. To examine this factor, we have added a standard GRPO control experiment with the interaction-turn limit doubled to 64, giving the baseline twice the per-rollout interaction budget of ReTerm-GRPO. The results are as follows:
| Method | Terminal-Bench 2.0 |
|---|---|
| GRPO (32 turns) | 28.0% |
| GRPO (64 turns) | 31.5% |
| ReTerm-GRPO (32 turns + 32 turns) | 34.8% |
Increasing the interaction-turn limit improves standard GRPO’s performance, but ReTerm-GRPO still outperforms this baseline by 3.3 points, supporting the benefit of recovery training beyond simply extending the interaction horizon. This also motivates us to explore recursive self-recovery in future work: extending the current single-round recovery to multiple rounds, allowing the agent to continue diagnosing and correcting errors when a recovery attempt still fails. Through this extension, we hope to further improve the agent’s performance on long-horizon terminal tasks.
Thank you again for your time in reviewing our work and for your constructive feedback.
We have responded to your concerns point by point and hope that our explanations and additional results have addressed them. As the discussion deadline of 19 Nov approaches, we would appreciate any further feedback or remaining questions you may have.
If our responses have resolved your concerns, we would also appreciate it if you could consider updating your score accordingly.
A1: Thanks for your comments. This work represents an early exploration of self-recovery training for terminal agents, aiming to improve their robustness in long-horizon terminal interactions.
Regarding existing recovery-oriented training, TermiGen [1] constructs recovery trajectories by injecting hand-crafted errors for supervised fine-tuning. These errors may differ in distribution from the failures actually encountered by the student model. To address this, we carefully construct ReTerm-10K from the student's own execution failures and extend this idea to the design of our online RL algorithm.
Regarding general RL techniques, works such as TMax [2] primarily adopt suitable general RL algorithms to improve terminal-task performance. Building on GRPO, we explore a recovery-training algorithm tailored to terminal environments, allowing recovery capabilities to adapt to the evolving failure distribution and co-evolve with task-solving capabilities. Our ablation studies and recovery evaluations further support the effectiveness of the proposed method.
We hope our work provides valuable insights and a practical framework for the research community, inspiring further research on improving terminal agents' capabilities in long-horizon tasks.
[1] TermiGen: High-Fidelity Environment and Robust Trajectory Synthesis for Terminal Agents.
[2] TMax: A Simple Recipe for Terminal Agents.
A2: Thanks for your constructive suggestion.
For training from ordinary intermediate states, we have added a control experiment that replaces the recovery tasks in ReTerm-GRPO with ordinary intermediate-state tasks. Within each original task group of 16 trajectories, we designate one trajectory at random in advance and use reservoir sampling to uniformly select an intermediate state at an interaction boundary. When the original task group is accepted for training, we add the selected environment state and history to the queue for subsequent training steps. All other training settings remain the same as those of ReTerm-GRPO. The results are as follows:
Training from ordinary intermediate states performs similarly to standard GRPO and does not reach the performance of ReTerm-GRPO.
For oversampling difficult tasks, we clarify that, following terminal-agent training work such as TMax [1], ReTerm-GRPO uses the dynamic sampling strategy introduced by DAPO [2]. Groups with identical rewards are filtered out, and sampling continues until the training batch is filled. Under this setting, ReTerm-GRPO constructs a recovery task from each original task group accepted for policy optimization. Rejected groups, including all-failure groups, do not contribute tasks to the recovery task queue. Therefore, we do not additionally increase the sampling proportion of difficult original tasks; instead, recovery training uses failed states from tasks within the model's current capabilities. We will explore combining difficult-task oversampling with recovery training in future work to further expand terminal agents' capabilities.
[1] TMax: A Simple Recipe for Terminal Agents.
[2] DAPO: An Open-Source LLM Reinforcement Learning System at Scale.