- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper constructs terminal-agent training environments by incrementally editing scenario novelty, skill rarity, or execution length, without requiring rollouts from the target learner during generation. A two-loop harness reviews plans and verifies executable environments, and an Evolution-Lineage scheduler advances the learner through the resulting generations. Experiments measure difficulty across three rollout models and report improved Terminal-Bench 2.1 performance after RL on two Qwen backbones.
Critical strengths
-
The approach offers a practical way to construct harder, executable tasks without continually querying the target learner. Oracle, no-op, rubric-based, and development-time human checks address the important risk that a lower pass rate merely reflects broken tasks. The experiments also examine effort levels and individual evolution directions, making the generation procedure more concrete.
-
Figure 5 shows sizable and sustained late-training gains over the implemented co-evolution and ensemble baselines on both backbones, from a shared post-RFT checkpoint. The cross-model difficulty measurements support the usefulness of the generated lineages beyond a single rollout model. Appendix D also usefully states the approximations behind the difficulty formulation, and Appendix B openly reports the lack of continued gains at 500 steps.
Critical weaknesses
-
The scheduler used for each arm of the main comparison should be made explicit before attributing its advantage specifically to environment generation. Figure 4 compares scheduling choices only through training reward and partially solved groups in the first 50 updates; there is no corresponding full-horizon held-out benchmark comparison. The three generation methods share the synthesis model and environment count, which is helpful, but equal counts and update steps do not ensure equal token exposure or compute as trajectories become longer. The early scheduler experiment does not isolate its contribution to final benchmark performance; if the comparison arms use different schedulers, generation-specific attribution is additionally confounded. These omissions leave the value of the complete pipeline clearer than the distinct sources of its advantage. A matched final-performance scheduler control and per-method rollout/token accounting would materially strengthen that attribution.
-
The experimental specification is not yet sufficient to reproduce and assess the main gains. The paper describes GRPO, RFT, staleness, and evaluation settings, but does not instantiate key training quantities such as the learning rate, effective update batch, and RFT data budget, or provide the exact plan-review rubrics, repair limits, and selected seed-environment identities. Appendix C reports aggregate API and GPU costs, rather than the per-method counts needed for the comparison. The five reported runs are not clearly identified as independent training runs or repeated evaluations, and Figure 5 emphasizes the best benchmark checkpoint from evaluations every ten steps without uncertainty. The persistent curve separation is encouraging, but a fixed selection rule and a clear account of run-level variability are needed to establish how robust the reported magnitude is.
Questions for authors
-
Which scheduler is used for each comparison arm? How much of the final Terminal-Bench gain remains with the same evolved environment pool but random scheduling, using a matched rollout and token budget? If full-run controls are unavailable, can the authors delimit the conclusion to the combined generation-and-scheduling pipeline and provide the existing per-method resource counts?
-
Can the authors supply the actual RFT/GRPO configuration, seed-pool identities, prompts/rubrics and repair limits, and clarify what the five runs repeat? Please also give results under a fixed checkpoint-selection rule, with per-run or paired uncertainty for the main comparisons. These are requests about the reported experiment, rather than suggestions for a new application domain.
Response to Reviewer oiEZ [1/n]
Thank you for your questions. We address them point by point below.
Weakness 1 & Q1
In the final experiments, all three methods follow the same scheduling principle: sampling environments within the model’s learnable range to ensure consistency in effective training samples. Specifically, Environment Evolution uses the EL Scheduler. For the other two methods, which lack a generation-by-generation structure and therefore cannot use the EL Scheduler, we prioritize environments within the model’s learnable range. This follows the same motivation as the EL Scheduler.
The results are as follows:
Under both sampling strategies, Environment Evolution outperforms the other two methods. Learnability-prioritized sampling improves performance for all three methods under the same rollout and training-step budgets.
We will separately describe the contribution of the overall pipeline and that of each component in the paper.
Weakness 2 & Q2
Thank you for these thorough and detailed questions.
RFT configuration: For each environment, we sample 16 trajectories and retain the successful ones. We then use embeddings to deduplicate the pool of successful trajectories.
GRPO configuration: Learning rate: 1e-6; maximum staleness: 5; batch size: 16; group size: 8.
Seed-pool identities and prompts/rubrics: We will release these materials upon acceptance of the paper.
Repair limits: Environments that fail the oracle or rubric checks are allowed up to 10 repair attempts. Those that still fail after 10 attempts are discarded.
Five evaluation runs: Following the official Terminal-Bench evaluation protocol, we evaluate each checkpoint five times and report the average. For each result reported in the paper and in our response to Weakness 1 and Q1, the same selected checkpoint is independently evaluated five times, since individual rollout results can be unstable on long-horizon tasks.