- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces EvoTerminal, a method for automatically adapting the training distribution of terminal agents as the agent capabilities evolve. It adjusts the distribution using feedback from past trajectories to decide both what capabilities to focus on, and how hard the problems targeting those capabilities should be. Their experiments on Terminal-Bench 2.0 show gains for both SFT model adaptation and harness adaptation. The paper also presents ablations on some of the method’s components, analyses of the resulting curriculum, and experiments that show SFT improvements transfer to SWE-Bench Verified and AIME 2025.
Critical strengths
- The presentation is overall good.
- The problem is interesting and well motivated.
- The expanded description of the method in appendix D, the example in appendix E, and the Prompt templates used in appendix F are useful and give clarity on the method.
Critical weaknesses
Method
- The method’s construction of the next iteration distribution has high complexity. It involves multiple steps, components, and hyperparameters. The evidence from the paper does not establish whether all this complexity is necessary to achieve the reported gains.
- The prioritization based on difficulty is stated to be controlled separately from the prioritization based on ‘what to practice’, using and respectively. However, the utility of skeletons, which influences , itself favors skeletons with an intermediate success rate. Should the roles be described differently? Is the overlap needed?
Experiments
-
The paper is on a curriculum method. I would like to see comparisons against other curriculum methods applied under all other conditions matched (same learner, budget, teacher LLM, etc). Most of the comparisons are against results from other papers where the differences extend beyond the methodology of the curriculum construction. This makes the benefits of the approach hard to disentangle from other potential causes of the seen differences.
-
I find table 4b as one of the most valuable results because it gives more direct evidence of the contributions of different method components. However, these ablations are constrained to the harness adaptation mode and to a single experimental setting, and do not show standard deviations.
Questions for authors
-
The method uses several different kinds of entities and predefined structures (e.g., families, domains, capabilities, L1-L5 difficulty levels, TaskIntent structure and Skeleton structure). Do you think all of them are necessary? How much work did it take to choose/develop these? Would you consider this dependence a weakness in transferring the approach to other settings?
-
Could you clarify to me what you see as the main novelty with respect to past work? I would also like a bit more clarity on the argument that difficulty based guidance is not enough. It is not fully clear to me the argument the paper makes for separating selection over families of tasks from favoring tasks at a reasonable level of difficulty within them. What underlying problem is it solving? Is it to more easily find tasks at the difficulty frontier, is it for diversity, or do you think it contributes in some other manner?
Additional (less important) feedback:
-
In the conclusion, the paper states ‘Our central insight is that the learning value of practice is learner-relative and non-stationary ‘. This seems to me like a very general statement and something recognized already by a lot of prior work in automatic curriculum learning. I would prefer to see something closer to what the paper uniquely contributes as the ‘Central insight’.
-
While I think the writing is good, I found the exact behavior of the algorithm (sections 3.2-3.4) and motivation for some design decisions could be clearer in the main text. For example, more clarity on the meaning of a TaskIntent, Skeleton, domain, and capability would have been useful to have without referring to the appendix.
-
The method makes the new distribution depend on insights extracted from the previous distribution. Can this lead to the distribution being too concentrated on certain aspects or directions of improvement and make it hard to uncover other unexplored regions of the task space where learning can also be fruitful?
6 comments
Reply to W3/W4 – Response to Reviewer ntmU (2/4)
W3/W4: Fair comparisons, per-run results, and standard deviations
1. W3: Fair comparisons under the same setting and against adaptive curriculum methods
Reply: Thank you for emphasizing fair comparisons. SETA-Evol is a difficulty-guided environment self-evolution method that uses task pass rates to choose between increasing difficulty, decreasing difficulty, and changing context while maintaining approximately the same difficulty. SETA, §3.2
EvoTerminal jointly adapts what to practice and how hard it should be: in addition to adjusting difficulty using success rates, it identifies specific capability bottlenecks from learner trajectories.
To provide a controlled comparison, we use SETA-Env and EvoTerminal data for SFT on the same Qwen3-8B base model. Both groups use the same train and evaluation settings and the Terminus 2.0 agent, with three repeated evaluations on the 89 tasks of Terminal-Bench 2.0.
| Base model | Training data | Avg@1 (%) | Gain |
|---|---|---|---|
| Qwen3-8B | SETA-Env | 11.2 ± 0.0 | — |
| Qwen3-8B | EvoTerminal | 14.6 ± 1.9 | +3.4 |
Under this shared base model, SFT training approach, and evaluation setting, EvoTerminal achieves a 3.4-percentage-point higher mean score. This supports the training effectiveness of its generated data in this setting.
2. W4: Sample standard deviations for Table 4b
Reply: Due to the computational and financial cost of SFT ablations, we conducted the Table 4b ablations only in harness adaptation mode; ablations in model adaptation mode would require constructing full training datasets at substantial cost. We report the standard deviations below.
| Variant | Avg@1 (%) | Eval 1 / 2 / 3 (%) | Gain |
|---|---|---|---|
| No curriculum | 41.5 ± 3.39 | 37.78 / 42.22 / 44.44 | +3.7 |
| Difficulty only | 42.2 ± 0.00 | 42.22 / 42.22 / 42.22 | +3.0 |
| Capability only | 43.0 ± 1.28 | 42.22 / 44.44 / 42.22 | +2.2 |
| Joint adaptation | 45.2 ± 1.28 | 46.67 / 44.44 / 44.44 | — |
Joint versus capability-only: paired task-level uncertainty. The following table summarizes average pass@1 (avg@1) and pass@3, with task-level 95% CIs in brackets. pass@3 is the fraction of tasks solved at least once in three evaluations; avg@1 averages each task's success rate across its three evaluations over the test tasks.
| Metric | Joint: score [95% CI] (%) | Capability-only: score [95% CI] (%) | Paired gain Δ [95% CI] (percentage points) |
|---|---|---|---|
| avg@1 | 45.19 [33.33, 57.04] | 42.96 [30.37, 55.56] | +2.22 [-2.96, 7.41] |
| pass@3 | 64.44 [51.11, 77.78] | 57.78 [42.22, 71.11] | +6.67 [-2.22, 15.56] |
For pass@3, the task-level changes relative to capability-only are:
| Task-level change (capability-only → Joint) | Definition | Number of tasks |
|---|---|---|
| Unsolved → solved | Failed in all three evaluations → succeeded at least once | 4 |
| Solved → unsolved | Succeeded at least once → failed in all three evaluations | 1 |
Reply to Q2 & Additional F1 – Response to Reviewer ntmU (3/4)
Q2 & Additional F1: Contribution relative to prior curriculum learning; why difficulty alone is insufficient; and whether the objective is the learning frontier, diversity, or something else
Reply: Thank you for your suggestions on our objectives and contribution. EvoTerminal addresses how to construct executable, verifiable Terminal environments at scale while continually adapting them to the current agent's training needs. We explain the motivation, objective, and contribution below.
1. Motivation: constructing Terminal training environments at scale that better match the current agent
-
Constructing usable Terminal environments at scale. Scaling Terminal training requires batch construction and validation of interactive environments with consistent instructions, runtimes, initial states, and verifiers—not just more questions. EvoTerminal turns training needs into runnable tasks through parameterized construction and execution-based validation.
-
Training value is not determined by difficulty alone. Equally difficult practice directions may deserve different priorities given their coverage and recent progress. Competence-progress-driven exploration and automatic curriculum learning allocate practice by learning progress [3,4]; CURIOUS extends this to multi-goal learning and relearning after forgetting [5], while IMGEP uses goal selection and cross-goal experience reuse [6]. These studies support deciding what and how much to practice through capability development and learning feedback.
-
Why distinguish difficulty and capability as two control targets? Tasks require different capabilities. If all tasks test fault localization in logs, adding logs or distractors adjusts difficulty but does not ensure practice in safe configuration changes or post-repair verification. Appropriate difficulty does not ensure adequate capability coverage. Capability/skeleton allocation assigns limited budget shares to behavioral targets; difficulty control adjusts challenge while preserving them.
-
Determining what to generate next from behavioral bottlenecks. Capability directions must become concrete, realizable practice targets. EvoTerminal converts trajectory bottlenecks, such as failing to locate the authoritative configuration source, into TaskIntents and compatible skeletons, linking curriculum allocation to the next environment batch's construction.
2. Curriculum objective: jointly deciding what, how much, and how hard to practice
- What and how much to practice: use coverage and learning feedback to allocate capability/skeleton budgets, translating trajectory-derived training needs into the next task distribution.
- How hard to practice: adapt each domain's difficulty distribution to the learner's state using training success rates.
Conclusion: Our objective is to allocate a fixed training budget to the current agent's capability needs at appropriate challenge levels: joint adaptation of capability and difficulty—What to Practice and How Hard.
3. Core contribution: turning joint adaptation into conditions for constructing environments from scratch
Dynamic curricula provide the general principle. EvoTerminal addresses how to extract reusable training needs from large numbers of trajectories, control curricula scalably, and turn these decisions into executable, verifiable new environments. We distinguish prior foundations from our specific challenges below.
| Prior foundation | Challenge addressed by EvoTerminal | Method |
|---|---|---|
| Analyze agent performance and failures from trajectories. | DISCOVER: How can reusable training needs be distilled from large numbers of trajectories? | Compare, cluster, and deduplicate trajectories by diagnosis type, execution stage, and capability into TaskIntents and compatible skeletons. |
| Adapt curricula and difficulty via learning feedback. | SHAPE: How can what, how much, and how hard to practice be controlled scalably? | Use coverage and learning feedback for hierarchical capability/skeleton budgets and per-domain difficulty adaptation. |
| Condition task generation and use agents for validation. | EVOLVE: How can current training needs become new environments that preserve targets, diversity, and quality? | Preserve construction targets; randomly sample other compatible parameters; filter via execution-based validation, repair, and judging. |
Conclusion: EvoTerminal turns trajectory diagnoses into reusable construction targets, sets generation proportions and challenge levels through curriculum allocation, and preserves training targets across varied environment instances.
Reply to W1/Q1 & W2 – Response to Reviewer ntmU (1/4)
Thank you for your detailed review and constructive suggestions on clarifying the method, its motivation, and the experimental evidence.
W1/Q1: Necessity of distribution updates, structural priors, and transfer costs
Reply: Thank you for asking about method complexity, design effort, and transfer dependencies. We distinguish the core distribution-update decisions, their supporting representations, and the development needed for transfer in two parts.
The update is straightforward: allocate what and how much to practice based on prior training exposure and recent learning feedback, adjust difficulty using domain-level success rates, and generate new tasks from selected skeletons and TaskIntents.
Shape makes two central decisions: what to practice and how hard it should be. Practice allocation combines learning utility with capability coverage; difficulty uses training-task success rates.
The aim is low-cost curriculum adaptation by reusing training trajectories, exposure records, and development evaluations, without test-set feedback. Supporting steps stabilize these proxies: sample-size shrinkage limits sparse observations' influence, smoothing moderates fluctuations, and exploration preserves opportunities for insufficiently observed directions.
Table 4b supports the two control blocks' complementarity under the same setting: no adaptation achieves 41.5%, difficulty only 42.2%, capability only 43.0%, and joint adaptation 45.2%.
We address three aspects:
Necessity: a shared interface for construction and curriculum control. Domains, families, capabilities, and difficulty levels let the generator construct task batches under explicit conditions and Shape adjust their sampling distributions. They specify what to generate and how to adapt the next batch, rather than requiring separately designed components for every task.
Design cost: a one-time, reusable investment. Initial design defines the schema, domain and capability vocabularies, compatible task families, and difficulty rules, linking Discover's diagnoses to Shape's allocation and Evolve's construction. These interfaces are reused across tasks and iterations; trajectory analysis produces specific TaskIntents and skeletons without redesigning the full structure each iteration. Unlike ongoing analysis, generation, and validation costs, this primarily one-time design cost is amortized as generation scales—a core contribution of our work.
Transfer cost: schema-based domain knowledge with LLM assistance. Adding domains within the Terminal setting mainly requires specifying domains, families, capabilities, difficulty rules, and construction constraints. An LLM can draft definitions from the existing schema and a new-domain description, followed by human review and execution-based task validation. The curriculum controller need not be rebuilt: the reusable schema and LLM assistance reduce definition effort, making transfer costs acceptable in our view.
W2: Clarifying the two uses of success rates
Reply: Thank you for the question. Both controls use success rates, but their feedback tasks, evaluation targets, and motivations differ:
These controls allocate practice and adjust difficulty, rather than duplicate success-rate control on the same task batch. Table 4b supports their complementarity in this setting: joint adaptation achieves 45.2%, versus 43.0% for capability-only and 42.2% for difficulty-only.
Goal GAN generates intermediate-difficulty goals for automatic curricula [1]. GenEnv centers its difficulty-alignment reward on a target success rate to keep tasks near the learner's zone of proximal development [2]. Both support difficulty calibration.
[1] Carlos Florensa, David Held, Xinyang Geng, Pieter Abbeel. Automatic Goal Generation for Reinforcement Learning Agents.
[2] Jiacheng Guo et al. GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators.