acceptodds
Under review as a conference paper at ICLR 2027

EvoTerminal: Co-Evolving Terminal Agents through Scalable Environment Evolution

Abstract

Sustained self-improvement requires not only updating the learner, but keeping its training experiences useful as capabilities change. Executable terminal environments provide a scalable and verifiable substrate for such practice, yet scaling their supply alone cannot keep the training distribution aligned with an evolving learner: informative tasks become redundant, previously unreachable requirements become learnable, and new capability bottlenecks emerge. Crucially, this mismatch is not reducible to difficulty, since a task can remain appropriately challenging while exercising behavior that is no longer limiting. We therefore argue that the executable training distribution should evolve with the learner, adapting both what to practice and how hard. We introduce EvoTerminal, a self-improvement framework that co-evolves terminal agents with their executable training distributions. Its Discover–Shape–Evolve loop identifies the learner's behavioral frontier from feedback, reallocates practice across capabilities while calibrating difficulty, and realizes these targets as diverse environments that preserve the learning objective. The updated learner reshapes the next distribution, making environment evolution a recurring part of training rather than a one-shot data-generation stage. The same loop supports both model and harness adaptation. On Terminal-Bench 2.0, EvoTerminal raises Qwen3-8B SFT by 12.0 points and Qwen3.6-35B-A3B skill optimization by 5.2 points over their bases, with gains transferring across benchmarks and backbones. Ablations further confirm the importance of feedback refresh and joint capability–difficulty adaptation, underscoring the value of co-evolving training distributions with the learner.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM Agent, Environment Evolution, Terminal Agent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ntmUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces EvoTerminal, a method for automatically adapting the training distribution of terminal agents as the agent capabilities evolve. It adjusts the distribution using feedback from past trajectories to decide both what capabilities to focus on, and how hard the problems targeting those capabilities should be. Their experiments on Terminal-Bench 2.0 show gains for both SFT model adaptation and harness adaptation. The paper also presents ablations on some of the method’s components, analyses of the resulting curriculum, and experiments that show SFT improvements transfer to SWE-Bench Verified and AIME 2025.

Critical strengths
  • The presentation is overall good.
  • The problem is interesting and well motivated.
  • The expanded description of the method in appendix D, the example in appendix E, and the Prompt templates used in appendix F are useful and give clarity on the method.
Critical weaknesses

Method

  • The method’s construction of the next iteration distribution has high complexity. It involves multiple steps, components, and hyperparameters. The evidence from the paper does not establish whether all this complexity is necessary to achieve the reported gains.
  • The prioritization based on difficulty is stated to be controlled separately from the prioritization based on ‘what to practice’, using wt+1dkw_{t+1}^{d_k} and πt+1\pi_{t+1} respectively. However, the utility of skeletons, which influences πt+1\pi_{t+1}, itself favors skeletons with an intermediate success rate. Should the roles be described differently? Is the overlap needed?

Experiments

  • The paper is on a curriculum method. I would like to see comparisons against other curriculum methods applied under all other conditions matched (same learner, budget, teacher LLM, etc). Most of the comparisons are against results from other papers where the differences extend beyond the methodology of the curriculum construction. This makes the benefits of the approach hard to disentangle from other potential causes of the seen differences.

  • I find table 4b as one of the most valuable results because it gives more direct evidence of the contributions of different method components. However, these ablations are constrained to the harness adaptation mode and to a single experimental setting, and do not show standard deviations.

Questions for authors
  • The method uses several different kinds of entities and predefined structures (e.g., families, domains, capabilities, L1-L5 difficulty levels, TaskIntent structure and Skeleton structure). Do you think all of them are necessary? How much work did it take to choose/develop these? Would you consider this dependence a weakness in transferring the approach to other settings?

  • Could you clarify to me what you see as the main novelty with respect to past work? I would also like a bit more clarity on the argument that difficulty based guidance is not enough. It is not fully clear to me the argument the paper makes for separating selection over families of tasks from favoring tasks at a reasonable level of difficulty within them. What underlying problem is it solving? Is it to more easily find tasks at the difficulty frontier, is it for diversity, or do you think it contributes in some other manner?

Additional (less important) feedback:

  • In the conclusion, the paper states ‘Our central insight is that the learning value of practice is learner-relative and non-stationary ‘. This seems to me like a very general statement and something recognized already by a lot of prior work in automatic curriculum learning. I would prefer to see something closer to what the paper uniquely contributes as the ‘Central insight’.

  • While I think the writing is good, I found the exact behavior of the algorithm (sections 3.2-3.4) and motivation for some design decisions could be clearer in the main text. For example, more clarity on the meaning of a TaskIntent, Skeleton, domain, and capability would have been useful to have without referring to the appendix.

  • The method makes the new distribution depend on insights extracted from the previous distribution. Can this lead to the distribution being too concentrated on certain aspects or directions of improvement and make it hard to uncover other unexplored regions of the task space where learning can also be fruitful?

6 comments
AuthorsOpenReview ↗

Reply to W1/Q1 & W2 – Response to Reviewer ntmU (1/4)

Thank you for your detailed review and constructive suggestions on clarifying the method, its motivation, and the experimental evidence.


W1/Q1: Necessity of distribution updates, structural priors, and transfer costs

Reply: Thank you for asking about method complexity, design effort, and transfer dependencies. We distinguish the core distribution-update decisions, their supporting representations, and the development needed for transfer in two parts.

1. The method’s construction of the next iteration distribution has high complexity

The update is straightforward: allocate what and how much to practice based on prior training exposure and recent learning feedback, adjust difficulty using domain-level success rates, and generate new tasks from selected skeletons and TaskIntents.

Shape makes two central decisions: what to practice and how hard it should be. Practice allocation combines learning utility with capability coverage; difficulty uses training-task success rates.

The aim is low-cost curriculum adaptation by reusing training trajectories, exposure records, and development evaluations, without test-set feedback. Supporting steps stabilize these proxies: sample-size shrinkage limits sparse observations' influence, smoothing moderates fluctuations, and exploration preserves opportunities for insufficiently observed directions.

Table 4b supports the two control blocks' complementarity under the same setting: no adaptation achieves 41.5%, difficulty only 42.2%, capability only 43.0%, and joint adaptation 45.2%.

2. The method uses several different kinds of entities and predefined structures

We address three aspects:

  • Necessity: a shared interface for construction and curriculum control. Domains, families, capabilities, and difficulty levels let the generator construct task batches under explicit conditions and Shape adjust their sampling distributions. They specify what to generate and how to adapt the next batch, rather than requiring separately designed components for every task.

  • Design cost: a one-time, reusable investment. Initial design defines the schema, domain and capability vocabularies, compatible task families, and difficulty rules, linking Discover's diagnoses to Shape's allocation and Evolve's construction. These interfaces are reused across tasks and iterations; trajectory analysis produces specific TaskIntents and skeletons without redesigning the full structure each iteration. Unlike ongoing analysis, generation, and validation costs, this primarily one-time design cost is amortized as generation scales—a core contribution of our work.

  • Transfer cost: schema-based domain knowledge with LLM assistance. Adding domains within the Terminal setting mainly requires specifying domains, families, capabilities, difficulty rules, and construction constraints. An LLM can draft definitions from the existing schema and a new-domain description, followed by human review and execution-based task validation. The curriculum controller need not be rebuilt: the reusable schema and LLM assistance reduce definition effort, making transfer costs acceptable in our view.

W2: Clarifying the two uses of success rates

Reply: Thank you for the question. Both controls use success rates, but their feedback tasks, evaluation targets, and motivations differ:

DimensionCapability/skeleton allocation πDifficulty allocation w
Source of success ratesDevelopment-task evaluationsTraining-task rollouts
What is assessedPractice directions' learning value: utility combines before/after progress with current successTraining-task difficulty for the current checkpoint
Control targetBudget per capability/skeleton: what and how much to practiceDifficulty-level sampling probabilities per domain: how hard to practice
Core motivationPrioritize directions worth further training, rather than rank by low success rates aloneKeep generated tasks appropriately challenging for the current learner, avoiding overly easy or hard tasks [1, 2]

These controls allocate practice and adjust difficulty, rather than duplicate success-rate control on the same task batch. Table 4b supports their complementarity in this setting: joint adaptation achieves 45.2%, versus 43.0% for capability-only and 42.2% for difficulty-only.

Goal GAN generates intermediate-difficulty goals for automatic curricula [1]. GenEnv centers its difficulty-alignment reward on a target success rate to keep tasks near the learner's zone of proximal development [2]. Both support difficulty calibration.


[1] Carlos Florensa, David Held, Xinyang Geng, Pieter Abbeel. Automatic Goal Generation for Reinforcement Learning Agents.

[2] Jiacheng Guo et al. GenEnv: Difficulty-Aligned Co-Evolution Between LLM Agents and Environment Simulators.

AuthorsOpenReview ↗

Reply to W3/W4 – Response to Reviewer ntmU (2/4)

W3/W4: Fair comparisons, per-run results, and standard deviations

1. W3: Fair comparisons under the same setting and against adaptive curriculum methods

Reply: Thank you for emphasizing fair comparisons. SETA-Evol is a difficulty-guided environment self-evolution method that uses task pass rates to choose between increasing difficulty, decreasing difficulty, and changing context while maintaining approximately the same difficulty. SETA, §3.2

EvoTerminal jointly adapts what to practice and how hard it should be: in addition to adjusting difficulty using success rates, it identifies specific capability bottlenecks from learner trajectories.

To provide a controlled comparison, we use SETA-Env and EvoTerminal data for SFT on the same Qwen3-8B base model. Both groups use the same train and evaluation settings and the Terminus 2.0 agent, with three repeated evaluations on the 89 tasks of Terminal-Bench 2.0.

Base modelTraining dataAvg@1 (%)Gain
Qwen3-8BSETA-Env11.2 ± 0.0—
Qwen3-8BEvoTerminal14.6 ± 1.9+3.4

Under this shared base model, SFT training approach, and evaluation setting, EvoTerminal achieves a 3.4-percentage-point higher mean score. This supports the training effectiveness of its generated data in this setting.

2. W4: Sample standard deviations for Table 4b

Reply: Due to the computational and financial cost of SFT ablations, we conducted the Table 4b ablations only in harness adaptation mode; ablations in model adaptation mode would require constructing full training datasets at substantial cost. We report the standard deviations below.

VariantAvg@1 (%)Eval 1 / 2 / 3 (%)Gain
No curriculum41.5 ± 3.3937.78 / 42.22 / 44.44+3.7
Difficulty only42.2 ± 0.0042.22 / 42.22 / 42.22+3.0
Capability only43.0 ± 1.2842.22 / 44.44 / 42.22+2.2
Joint adaptation45.2 ± 1.2846.67 / 44.44 / 44.44—

Joint versus capability-only: paired task-level uncertainty. The following table summarizes average pass@1 (avg@1) and pass@3, with task-level 95% CIs in brackets. pass@3 is the fraction of tasks solved at least once in three evaluations; avg@1 averages each task's success rate across its three evaluations over the test tasks.

MetricJoint: score [95% CI] (%)Capability-only: score [95% CI] (%)Paired gain Δ [95% CI] (percentage points)
avg@145.19 [33.33, 57.04]42.96 [30.37, 55.56]+2.22 [-2.96, 7.41]
pass@364.44 [51.11, 77.78]57.78 [42.22, 71.11]+6.67 [-2.22, 15.56]

For pass@3, the task-level changes relative to capability-only are:

Task-level change (capability-only → Joint)DefinitionNumber of tasks
Unsolved → solvedFailed in all three evaluations → succeeded at least once4
Solved → unsolvedSucceeded at least once → failed in all three evaluations1
AuthorsOpenReview ↗

Reply to Q2 & Additional F1 – Response to Reviewer ntmU (3/4)

Q2 & Additional F1: Contribution relative to prior curriculum learning; why difficulty alone is insufficient; and whether the objective is the learning frontier, diversity, or something else


Reply: Thank you for your suggestions on our objectives and contribution. EvoTerminal addresses how to construct executable, verifiable Terminal environments at scale while continually adapting them to the current agent's training needs. We explain the motivation, objective, and contribution below.

1. Motivation: constructing Terminal training environments at scale that better match the current agent

  1. Constructing usable Terminal environments at scale. Scaling Terminal training requires batch construction and validation of interactive environments with consistent instructions, runtimes, initial states, and verifiers—not just more questions. EvoTerminal turns training needs into runnable tasks through parameterized construction and execution-based validation.

  2. Training value is not determined by difficulty alone. Equally difficult practice directions may deserve different priorities given their coverage and recent progress. Competence-progress-driven exploration and automatic curriculum learning allocate practice by learning progress [3,4]; CURIOUS extends this to multi-goal learning and relearning after forgetting [5], while IMGEP uses goal selection and cross-goal experience reuse [6]. These studies support deciding what and how much to practice through capability development and learning feedback.

  3. Why distinguish difficulty and capability as two control targets? Tasks require different capabilities. If all tasks test fault localization in logs, adding logs or distractors adjusts difficulty but does not ensure practice in safe configuration changes or post-repair verification. Appropriate difficulty does not ensure adequate capability coverage. Capability/skeleton allocation assigns limited budget shares to behavioral targets; difficulty control adjusts challenge while preserving them.

  4. Determining what to generate next from behavioral bottlenecks. Capability directions must become concrete, realizable practice targets. EvoTerminal converts trajectory bottlenecks, such as failing to locate the authoritative configuration source, into TaskIntents and compatible skeletons, linking curriculum allocation to the next environment batch's construction.

2. Curriculum objective: jointly deciding what, how much, and how hard to practice

  • What and how much to practice: use coverage and learning feedback to allocate capability/skeleton budgets, translating trajectory-derived training needs into the next task distribution.
  • How hard to practice: adapt each domain's difficulty distribution to the learner's state using training success rates.

Conclusion: Our objective is to allocate a fixed training budget to the current agent's capability needs at appropriate challenge levels: joint adaptation of capability and difficulty—What to Practice and How Hard.

3. Core contribution: turning joint adaptation into conditions for constructing environments from scratch

Dynamic curricula provide the general principle. EvoTerminal addresses how to extract reusable training needs from large numbers of trajectories, control curricula scalably, and turn these decisions into executable, verifiable new environments. We distinguish prior foundations from our specific challenges below.

Prior foundationChallenge addressed by EvoTerminalMethod
Analyze agent performance and failures from trajectories.DISCOVER: How can reusable training needs be distilled from large numbers of trajectories?Compare, cluster, and deduplicate trajectories by diagnosis type, execution stage, and capability into TaskIntents and compatible skeletons.
Adapt curricula and difficulty via learning feedback.SHAPE: How can what, how much, and how hard to practice be controlled scalably?Use coverage and learning feedback for hierarchical capability/skeleton budgets and per-domain difficulty adaptation.
Condition task generation and use agents for validation.EVOLVE: How can current training needs become new environments that preserve targets, diversity, and quality?Preserve construction targets; randomly sample other compatible parameters; filter via execution-based validation, repair, and judging.

Conclusion: EvoTerminal turns trajectory diagnoses into reusable construction targets, sets generation proportions and challenge levels through curriculum allocation, and preserves training targets across varied environment instances.

Reviewer_nugjOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes EvoTerminal, a framework that co-evolves terminal agents and their training environments. The main idea is that training tasks should adapt to the learner's changing capabilities rather than relying on a fixed environment distribution. EvoTerminal follows a Discover–Shape–Evolve pipeline, where agent trajectories are analyzed to identify capability gaps, training tasks are allocated based on capability and difficulty, and new executable environments are generated accordingly.

The framework supports both model fine-tuning and skill optimization without updating model weights. Experiments on Terminal-Bench 2.0 show improvements over the corresponding base models, with additional results on SWE-bench Verified and AIME 2025. Ablation studies examine the effects of feedback refresh, capability allocation, and difficulty adaptation.

Critical strengths

S1. Meaningful problem and motivation. The paper addresses an important limitation of static training environments, where tasks may become less useful as an agent improves. Adapting the training distribution to the learner's current weaknesses is a reasonable direction for sustained improvement.

S2. Joint adaptation of capability and difficulty. Rather than focusing only on task difficulty, EvoTerminal also considers which capabilities need additional practice. This distinction makes sense, since agents with similar success rates may struggle with different aspects of a task.

S3. Evaluation through different learning mechanisms. The framework is tested with both SFT and frozen-weight skill optimization, showing that the proposed environment adaptation is not tied to a single training method. The cross-benchmark results also provide encouraging evidence of transfer.

S4. Useful ablation studies. The comparisons with static environments, frozen feedback, and individual curriculum components help establish the benefits of online adaptation. The environment-quality analysis and human assessment further strengthen the evaluation.

S5. The figures are very beautiful :))

Critical weaknesses

W1. Limited direct comparisons with closely related methods. The SFT results mainly compare against published results from models trained with different backbones, teachers, datasets, and optimization methods. More controlled comparisons with adaptive environment generation methods such as SETA or Agent-World would help establish the specific benefits of EvoTerminal.

W2. The improvements from environment evolution are relatively modest in the skill setting. In Table 3, EvoTerminal improves performance from 43.0% to 45.2% compared with training on the original environments. Although the result is encouraging, the gain is relatively small given the additional complexity of repeatedly analyzing trajectories, updating the curriculum, and generating new environments.

W3. Limited analysis of computational cost. EvoTerminal relies on multiple model calls for trajectory analysis, environment generation, validation, and teacher demonstrations. The experiments emphasize the number of training trajectories or task uses, but do not provide a complete comparison of the computational cost required to obtain these improvements.

Questions for authors

See Weaknesses

4 comments

Thank you for your thoughtful review and constructive suggestions; we especially appreciate your positive feedback on the figures!


W1: Fairness of the data/environment comparisons

Reply: For the SFT experiments, we acknowledge differences in teachers and evaluation harnesses. Within our computational constraints, we controlled the comparisons as closely as possible, including the base model and optimization method.

For Qwen3-8B, we compare against CLI-Universe and Terminal-World under SFT, and also include the RL-based Endless Terminals and SETA results.

For adaptive environment generation, we additionally compare against SETA (an adaptive environment generation methods) for SFT training and evaluation under identical evaluation settings, with the Terminus 2.0 agent and three repeated evaluations. The results are:

Base modelTraining dataAvg@1Gain
Qwen3-8BSETA-Env11.2 ± 0.0-
Qwen3-8BEvoTerminal14.6 ± 1.9+3.4

Although both approaches use self-evolving environments, EvoTerminal, which combines capability and difficulty adaptation, achieves better performance than SETA under the same setting.


W2. The improvements from environment evolution are relatively modest in the skill setting.

Reply: Thank you for examining the incremental gain of the full loop over original environments. We would like to clarify the provenance of this control:

  1. Original environments uses high-quality, manually curated non-test tasks from Terminal-Bench 2.0. These tasks share the benchmark provenance and construction context of the test set, making this a valuable control drawn from the same benchmark. Their distribution may also be closer to the test distribution.

  2. Table 3a provides additional comparisons with generated environment sources. Under identical settings, EvoTerminal achieves 45.2%, compared with 40.7%, 40.0%, and 42.2% for the other automatic environment construction methods, yielding larger observed mean gains over these generated sources.

In contrast, EvoTerminal starts from the generator's default prior and has the agent construct executable tasks automatically. The 45 held-out test tasks never participate in generation, selection, or training. The significance of this result is therefore not only the 2.2% over original environments: automatically generated practice also matches and exceeds the performance obtained with manually curated tasks from the same benchmark in this experiment.


W3. Limited analysis of computational cost.

Reply: For the skill-setting experiments, we measured the average token consumption at each stage. Gemini-3.6-Flash was used for all stages.

StageAverage input tokensAverage output tokensAverage cached input tokens (cache hit rate)
Trajectory analysis (per task group)47.8k3.9k-
Environment generation (per candidate)33.6k41.4k-
Agent Check (per candidate)481.9k12.1k442.1k (91.7%)
LLM Judge (per candidate)31.4k4.1k-

The cost of trajectory analysis is amortized across a batch of tasks, as its results can be reused for constructing multiple tasks. As shown in the table, most of EvoTerminal's input tokens are served from cache.

Dear Reviewer nugj,

Thank you again for your thoughtful review and encouraging feedback! As the discussion phase is approaching its end, we kindly request to let us know if the above clarifications and the previously added experiments have addressed the remaining questions. We would be happy to address any additional points the reviewer may have during the remaining time of the discussion phase. We thank the reviewer for engaging with us in the discussion.

Thank you for the detailed rebuttal and additional experiments. I appreciate the authors' efforts to address my concerns, particularly the new comparison with SETA and the computational cost analysis.

However, I remain concerned about the empirical results. The improvements in the skill setting are relatively modest, and the additional uncertainty analysis does not provide strong evidence that joint adaptation consistently outperforms simpler alternatives. Furthermore, the reported standard deviations are based on repeated evaluations rather than independent training runs, leaving the robustness of these gains unclear. The token consumption analysis also highlights the substantial computational overhead of the framework, particularly during environment generation and validation.

Overall, while the rebuttal provides useful clarifications, I do not find the additional evidence sufficient to substantially strengthen the empirical conclusions. I will therefore maintain my original rating of 3 (Weak Accept).

AuthorsOpenReview ↗

Gentle Follow-Up on Our Responses

Thank you for your careful follow-up review. We address your remaining concerns below.


1. The improvements in the skill setting are relatively modest, and the additional uncertainty analysis does not provide strong evidence that joint adaptation consistently outperforms simpler alternatives.

(1) Starting from the same human-curated environments, continued evolution improves performance by 3.7 percentage points. Original environments uses human-curated non-test tasks from Terminal-Bench 2.0, while our existing EvoTerminal setting automatically constructs environments from the generator's default prior. We compare the following three settings to examine environment evolution from different starting points:

SettingInitial poolEvolutionMean ± SD (%)
Original environmentsTB 2.0 curatedOff43.0 ± 2.6
EvoTerminalDefault priorOn45.2 ± 1.3
EvoTerminalTB 2.0 curatedOn46.7 ± 2.2

Starting from the same high-quality human-curated TB 2.0 environments, EvoTerminal increases the mean success rate from 43.0% to 46.7%. This 3.7-percentage-point gain exceeds the gain of Original environments over No skill, demonstrating that continued environment evolution adds value beyond training on high-quality human-curated tasks.

(2) Joint adaptation expands successful task coverage across three evaluations. In the Joint versus Capability-only comparison, pass@3 is the fraction of tasks solved at least once across three independent evaluations:

SettingPass@3 (%)
Capability-only57.78
Joint adaptation64.44
Δ (pp)+6.67

Joint improves pass@3 over Capability-only by 6.67 percentage points, further supporting the advantage of joint adaptation over capability-only adaptation.

Conclusion: The gain from environment evolution is practically meaningful: it exceeds the gain from skill training on the original environments itself. Joint adaptation also improves pass@3 by 6.67 percentage points. Together, these two comparisons directly support the benefits of continued environment evolution over Original environments and joint adaptation over capability-only adaptation.


2. The reported standard deviations are based on repeated evaluations rather than independent training runs

Three independent evaluations assess the trained system's performance across repeated executions. Terminal tasks involve multi-step tool use and environment interaction, so repeated evaluations capture how generation and execution randomness affect success rates.

Three independent training runs require separately generating and validating multiple rounds of environments, at substantial computational cost. We therefore conduct three independent evaluations and report per-run results, sample standard deviations, and paired task-level intervals to assess the execution stability of the trained policy.


3. The token consumption analysis also highlights the substantial computational overhead of the framework.

EvoTerminal's additional generation cost supports trajectory-guided task construction and validation, repair, and quality assessment before training. Its environment construction builds on TMAX's workflow, making TMAX a relevant reference. We report total input, uncached input, and output consumption separately.

TMAX data source: Repo2RLEnv experiment accounting. Both pipelines are priced using Gemini 3.6 Flash: official pricing.

PipelineInput/taskOutput/taskCache hit$/task
TMAX195.6k74.6k-$0.426
EvoTerminal1,017.8k (uncached: 209.7k)102.6k79.4%$0.603
Δ+7.2% (uncached)+37.5%—+$0.177

Of EvoTerminal's average input tokens per task, 79.4% are served from cache, leaving 209.7k uncached input tokens. The additional cost is just $0.177 per task.

The additional calls support validation, repair, and quality assessment before training. TMAX relies on soft filtering during RL, while EvoTerminal validates tasks upfront for SFT and skill optimization. Trajectory analysis results are reused across tasks to amortize costs.

This investment both aligns environments with current learning needs and provides quality assurance before training.

Conclusion: Under the pricing above, approximately $0.177 more per task enables trajectory analysis, validation, and repair, supporting environment construction and continued evolution with training gains. This additional cost is acceptable.


Thank you again for your constructive feedback. We hope these further responses address your concerns!

Reviewer_mzXBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvoTerminal adapts the executable training distribution of a terminal agent as its capabilities change. Its loop diagnoses behavioral bottlenecks from rollouts, allocates practice across capabilities and difficulty levels, and constructs validated environments. The framework supports both supervised model updates and external-skill optimization. Experiments report Terminal-Bench 2.0 gains, SFT transfer to SWE-Bench Verified and AIME 2025, and controls for feedback refresh, data ordering, and capability versus difficulty adaptation.

Critical strengths

Separating what to practice from how difficult the practice should be is useful, and the paper implements that distinction in an executable loop. Supporting both model-weight and frozen-model skill adaptation broadens its practical relevance. The contribution is the integration and validation of these components.

Table 4 provides informative controls: the SFT study fixes the generator, teacher, accepted trajectory budget, and update schedule, while the skill study separately switches capability and difficulty adaptation. These controls, cross-benchmark SFT transfer, explicit benchmark isolation, and a blind human audit of 200 accepted environments make the evidence stronger than a simple before/after benchmark comparison.

Critical weaknesses

The reliability of the component-level gains remains unclear. Appendix B.1 describes three repeated benchmark evaluations but does not establish independent reruns of the complete environment-evolution and learner-update pipeline. In the 45-task skill test, the 2.2-point gain over original environments and capability-only adaptation is roughly one additional solved task per evaluation. Table 4 reports means without variability. This does not establish that the gain is insignificant, but it prevents a confident assessment of how consistently joint adaptation improves on the stronger controls. The larger gains over the unadapted models are more persuasive than these smaller incremental claims.

The skill controller’s progress signal is partly entangled with the selection procedure it is intended to guide. Algorithm D.3 selects skill revisions on the same nine development tasks whose before/after scores are subsequently used to estimate learning utility in the outer loop. Measured progress can therefore include improvement selected on the gate itself, and many capability/skeleton estimates may have very few observations. The shrinkage and exploration rules help with noise but do not establish predictive validity on unseen tasks. This concerns the explanation of the adaptive loop.

Questions for authors

Do the three runs repeat only evaluation, or independently repeat environment generation and model/skill training? Could the authors provide paired task-level uncertainty and per-run results for the main controls in Table 4, especially joint versus capability-only adaptation?

Does the skill curriculum retain its advantage when progress is estimated on tasks distinct from those used to accept skill revisions? Alternatively, is there evidence that the estimated utility predicts subsequent held-out improvement, and how many active skeletons receive valid paired progress observations in a typical development batch?

4 comments
AuthorsOpenReview ↗

Response to Reviewer mzXB (1/2)

Thank you for your careful review and constructive questions about the reliability of our results and the curriculum feedback mechanism.


W1/Q1: Reliability of the gains, repeated evaluation, and per-run results

1. the 2.2-point gain over original environments and capability-only adaptation.

Reply: Thank you for examining the incremental gain of the full loop over original environments. We would like to clarify the provenance of this control. Original environments uses high-quality, manually curated non-test tasks from Terminal-Bench 2.0. These tasks share the benchmark provenance and construction context of the held-out test set, making this a valuable control drawn from the same benchmark. Their distribution may also be closer to the test distribution.

In contrast, EvoTerminal starts from the generator's default prior and has the agent construct executable tasks automatically. The 45 held-out test tasks never participate in generation, selection, or training. The significance of this result is therefore not only the 2.2-percentage-point improvement over original environments: automatically generated practice also matches and exceeds the performance obtained with manually curated tasks from the same benchmark in this experiment.

The comparisons with other generated environment sources in Table 3a further support this interpretation. Under identical settings, EvoTerminal achieves 45.2%, compared with 40.7%, 40.0%, and 42.2% for the other automatic environment construction methods, yielding larger observed mean gains over these generated sources.

Conclusion: EvoTerminal achieves the observed gain using environments generated from scratch, and its advantage is larger over the other generated environment sources than over the high-quality original environments.

2. Do the three runs repeat only evaluation, or independently repeat environment generation and model/skill training?

Reply: The three runs repeat only evaluation. Due to computational and cost constraints, we did not independently repeat environment generation and model/skill training.

3. per-run results for the main controls in Table 4

We provide the results of three independent evaluations for Table 4b below.

VariantAvg@1 (%)Evaluation 1 / 2 / 3 (%)Gain (percentage points)
No curriculum41.5 ± 3.3937.78 / 42.22 / 44.44+3.7
Difficulty only42.2 ± 0.0042.22 / 42.22 / 42.22+3.0
Capability only43.0 ± 1.2842.22 / 44.44 / 42.22+2.2
Joint adaptation45.2 ± 1.2846.67 / 44.44 / 44.44—

Joint versus capability-only: paired task-level uncertainty. The following table summarizes average pass@1 (avg@1) and pass@3, with task-level 95% CIs in brackets. pass@3 is the fraction of tasks solved at least once in three evaluations; avg@1 averages each task's success rate across its three evaluations over the test tasks.

MetricJoint: score [95% CI] (%)Capability-only: score [95% CI] (%)Paired gain Δ [95% CI] (percentage points)
avg@145.19 [33.33, 57.04]42.96 [30.37, 55.56]+2.22 [-2.96, 7.41]
pass@364.44 [51.11, 77.78]57.78 [42.22, 71.11]+6.67 [-2.22, 15.56]

For pass@3, the task-level changes relative to capability-only are:

Task-level change (capability-only → Joint)DefinitionNumber of tasks
Unsolved → solvedFailed in all three evaluations → succeeded at least once4
Solved → unsolvedSucceeded at least once → failed in all three evaluations1

Joint covers a net additional 3 tasks, corresponding to a 6.67-percentage-point increase in pass@3.

We also provide the SFT controls from Table 4a below.

VariantAvg@1 (%)Evaluation 1 / 2 / 3 (%)Gain (percentage points)
Static pool24.3 ± 0.6523.60 / 24.72 / 24.72+4.1
Frozen-feedback24.0 ± 1.7222.47 / 23.60 / 25.84+4.5
Evolved pool, shuffled25.5 ± 1.3024.72 / 24.72 / 26.97+3.0
EvoTerminal28.5 ± 1.7226.97 / 28.09 / 30.34—

Conclusion: Joint adaptation has a higher observed mean than capability-only adaptation and solves three additional tasks in net pass@3 coverage.

AuthorsOpenReview ↗

Response to Reviewer mzXB (2/2)

W2/Q2: Limited development feedback and coupling with skill selection in Shape

Thank you for examining Shape and recognizing our handling of sparse-feedback noise. We have also considered this trade-off. Your comments raise two related but distinct issues: (1) the valid feedback available from nine development tasks per skill-training iteration; and (2) potential selection effects from using the same tasks to accept skill revisions and estimate progress.

1. many capability/skeleton estimates may have very few observations.

Reply: Thank you for highlighting the potential sparsity of development feedback.

(1) Why are there only nine development tasks per iteration? In the skill setting, we follow SkillOpt's approximate 4:1:5 split: 35 training, 9 development, and 45 held-out test tasks. Development tasks support skill selection and learning feedback estimation each iteration; test tasks are excluded from curriculum adaptation. This budget choice does not imply abundant independent observations for every capability or skeleton.

(2) How can nine development tasks provide feedback for multiple skeletons? Dev tasks and skeletons are associated through domain × task capability, with 17 × 8 = 136 possible combinations. A synthesized task may involve 1–3 core capabilities, whereas trajectory analysis assigns each skeleton one primary bottleneck capability. Dev tasks and skeletons therefore need not have a one-to-one correspondence: feedback from the same development task can be associated with multiple matching skeletons.

(3) How many skeletons actually receive learning-utility feedback? The statistics for the skill setting are:

SettingTraining tasksDev tasksActive skeletonsSkeletons receiving learning-utility feedbackFeedback coverage
Skill: iteration 23591010100.0% (10/10)
Skill: iteration 3359131184.6% (11/13)

These results show that a small development set does not necessarily restrict feedback to only a few skeletons.

Conclusion: Nine development tasks provide feedback to 10/10 and 11/13 active skeletons through shared capability associations. This establishes broad feedback coverage.

2. Coupling between Gate and progress: Measured progress can therefore include improvement selected on the gate itself

Reply: We agree that skill revisions are selected using the current development batch, so gains on that same batch may reflect selection effects in addition to genuine improvement. We also clarify that learning utility serves as a low-cost curriculum heuristic rather than an unbiased estimate of capability improvement. We address its practical impact through the separated-feedback comparison and mechanism discussion below.

(1) Does curriculum performance persist when Gate and progress use separate development tasks? We compare sharing nine development tasks between Gate and progress with using two disjoint nine-task sets: Dev 1 for skill selection and Dev 2 for paired progress evaluation between the iteration's initial and final accepted skills, without participating in Gate. Each component therefore uses nine tasks in both settings. Cost and time constraints limit this comparison to the first three iterations.

SettingGate DevProgress DevIter 1 avg@1Iter 2 avg@1Iter 3 avg@1
Shared (original)Dev 1 (9)Dev 1 (9)42.2 ± 2.2243.7 ± 1.2843.7 ± 1.28
SeparateDev 1 (9)Dev 2 (9)41.5 ± 1.2843.0 ± 2.5744.4 ± 0.00
Δ (separate − shared)——-0.7-0.7+0.7

Similar learning trends and continued improvement with separate feedback suggest that predictive-validity concerns have little practical impact here, supporting the curriculum's effectiveness and robustness to the split.

(2) Possible explanations for these results.

  • Coverage-driven allocation. Default weights of 70% under-coverage and 30% utility reduce reliance on estimated development-set progress.
  • Fresh development tasks. Tasks are fixed within iterations for paired comparisons but refreshed across iterations, potentially reducing repeated adaptation to one batch.
  • Capability anchors. Feedback guides matching capabilities/skeletons to generate new tasks of the same type, rather than repeat development tasks. Its curriculum value may persist if learned strategies transfer.
  • Noise mitigation. Sample-size shrinkage limits sparse feedback's influence, temporal smoothing moderates fluctuations, and exploration preserves other practice directions. These may reduce optimistic feedback's allocation impact without eliminating systematic selection bias.

Conclusion: The curriculum retains comparable performance when Gate and progress use disjoint development sets across the first three iterations, showing that its observed effectiveness does not require reusing the Gate tasks for progress estimation.

Dear Reviewer mzXB,

Thank you again for your careful review. Now that we are past the midpoint of the discussion period, we would appreciate it if you could take a look at our responses, including the additional uncertainty analysis and the comparison using separate development sets for Gate and progress.

If you are satisfied, we kindly request you to consider updating the score to reflect the newly added results and discussion. We remain committed to addressing any remaining points you may have during the discussion phase.

Thank you for your response. After carefully reviewing the rebuttal, I find that my primary concerns remain insufficiently addressed. Therefore, I have decided to retain my original rating.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response and Summary

We thank the reviewers for their time and constructive feedback. We are glad that the reviewers found:

  1. Problem and motivation. As an agent’s capabilities change, previously valuable tasks can become repetitive, making it worthwhile to study how training environments can continually adapt to the agent’s current learning needs. nugj, ntmU
  2. Joint adaptation of capability and difficulty. Even when two tasks are similar in difficulty, they may test different capabilities. EvoTerminal turns “what to practice” and “how hard” into actionable curriculum decisions. mzXB, nugj
  3. Applicability to different learning mechanisms. The same environment evolution process supports both SFT and skill optimization with a frozen model, and the cross-benchmark SFT results further demonstrate the training value of the generated environments. mzXB, nugj
  4. Targeted experimental controls. Static environments, frozen feedback, and ablations of individual curriculum components help distinguish the roles of the different mechanisms. Reviewers also appreciated the isolation of the test set, the environment quality assessment, and the human audit. mzXB, nugj; ntmU Table 4b

We summarize the main points below:

  1. Core contribution and the two aspects of curriculum control. (ntmU W1/W2, Q2, Additional F1) Through capability allocation and difficulty control, EvoTerminal generates the next batch of executable, verifiable training environments from scratch to match the agent’s current capabilities. We verified EvoTerminal’s effectiveness under both learning mechanisms, SFT and skill optimization.

  2. Controlled comparison with an adaptive environment construction method. (nugj W1; ntmU W3) We conducted a controlled comparison with SETA under the same configuration. Across three repeated evaluations, EvoTerminal (14.6%) outperformed SETA-Env (11.2%) by 3.4pp, demonstrating the training value of the environments EvoTerminal constructs.

  3. Gains from environment evolution and their reliability. (mzXB W1/Q1; nugj W2; ntmU W4) Without skill train (40.0%) → with skill training on the TB 2.0 training set (43.0% / +3.0pp) → with evolution from the same pool (46.7% / +3.7pp). The generator’s default prior with evolution yielded 45.2%. This shows the benefit of environment evolution. In the curriculum ablation, joint adaptation raised pass@3 from 57.78% to 64.44% compared with capability-only adaptation, supporting the reliability of joint adaptation.

  4. Coverage and independence of development feedback. (mzXB W2/Q2) Through shared capabilities, 9 Dev tasks per iteration provided utility feedback to 10/10 active skeletons in iteration 2 and 11/13 in iteration 3. With disjoint development sets for skill selection (Gate) and progress estimation, curriculum performance remained comparable (within 0.7 pp). Together, these results demonstrate that feedback coverage was broad, and gate–progress coupling had little impact on performance in the first three iterations.

  5. Construction cost, transfer, and reuse. (nugj W3; ntmU Q1, Additional F3) EvoTerminal costs only $0.177 more per task on average than TMAX (Gemini 3.6 Flash). EvoTerminal’s predefined construction interfaces can be reused across iterations and across different terminal task domains, and parameter sampling can generate different environment instances for the same practice target, illustrating the method’s transferability.


We thank the reviewers and the area chair for their time and consideration. We look forward to hearing your thoughts and would be happy to clarify any remaining questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.