acceptodds
Under review as a conference paper at ICLR 2027

SkillGym: Training Skill-Use Agents with Automatic Verifiable Environment Generation

Abstract

Skills equip LLM agents with professional knowledge and guidance to complete long-horizon and complex tasks. Although skills have been widely adopted in recent agent paradigms and harnesses, how to synthesize reliable training data and how to train agents for skill use remain underexplored. In this work, we propose SkillGym, an automatic pipeline to build verifiable environments, collect trajectories, and train skill-use agents. SkillGym first crawls a large volume of skills from the internet, then keeps those whose workflows can run reproducibly offline. A builder-reviewer pipeline is used to construct difficulty-controlled tasks, spanning four task types, each with a reference solution and an executable verifier. With this pipeline, we build 6.8k environments and collect 19k verified successful trajectories for supervised finetuning. Finetuning on these trajectories improves LLMs of different families and sizes, from 2B to 27B parameters across four skill-use benchmarks; Our Qwen3.5-9B SFT model outperforms the 397B untrained model on two of them. Further analysis shows that training teaches agents to invoke skills, raising the rate of reading the relevant skill from 28% to 96%, and that the gains hold across reasoning structures, extending to task types that form a minority of the training data and to skills held out from training

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
agent, skill, environment

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8gYdOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SkillGym, an automatic pipeline that transforms community-written agent skills into verifiable training environments for skill-use agents, which comprises three stages: 1) Skill Collection and Curation: skills are crawled from public sources and filtered by completeness, offline executability, and quality, yielding a pool of about 11.9k skills; 2) Skill-Grounded Task Construction: a two-stage builder–reviewer agent system first builds Docker environments and then authors tasks under four reasoning-structure profiles (procedural, abductive, constraint satisfaction, and partial order), each with an instruction, initial workspace, reference solution, and executable verifier, filtered by validity and quality gates; and 3) Trajectory Collection and Training: three teacher models collect successful trajectories across four agent harnesses for supervised fine-tuning. With 6.8k tasks and 19k trajectories, experiments on five models from 2B to 27B show improvements on four skill-use benchmarks, including tasks with skills held out from training.

Critical strengths
  1. Training agents to use externally provided skills is a practical and timely problem, and turning public skills into executable training environments is a reasonable direction.
  2. The evaluation covers multiple model families and sizes and several external skill-use benchmarks, and the paper includes a held-out skill split.
Critical weaknesses
  1. Limited novelty. Synthesizing verifiable tasks from public skills and distilling teacher trajectories via SFT has already been done by SKT [1], which the paper cites, and profiling large skill corpora and synthesizing tasks for verifiable skills is also explored in [2]. More broadly, LLM-based environment building with build–test–repair loops and fail-before/pass-after validity checks is standard in recent environment-synthesis work [3, 4, 5]. The claimed differences, namely four reasoning-structure profiles and an LLM reviewer, are incremental engineering choices, and the training recipe is plain SFT distillation without any methodological contribution.

  2. The task construction process lacks essential details. The main text describes the core pipeline in only a few sentences, and even the appendix provides condensed rather than full prompts. Key information is missing: 1) which models serve as builder and reviewer; 2) the pass and skip rates at each gate, which would explain why only 3,494 of 11,897 curated skills yield tasks; 3) how the test split and held-out skills are selected, and whether tasks from the same skill can appear in both training and held-in test sets; 4) the dataset statistics are inconsistent: the 6,772 tasks include 1,581 reviewer-unresolved tasks, and partial-order tasks are stated to be excluded from these counts, so it is unclear what the final training set contains.

  3. Quality control is insufficient for a single cross-domain pipeline. 1) The same pipeline is applied to all 19 domains, although correctness criteria differ substantially across them, and the requirement of value-based verification excludes or reshapes skills with subjective outputs (e.g., writing skills are excluded entirely); 2) quality relies entirely on automatic checks and an LLM reviewer whose transcripts were not stored, and no human or expert audit of tasks or verifiers is reported; 3) skill dependency is not verified per task, unlike SKT [1], which keeps a task only if a run with the skill outperforms a run without it. As a result, Kimi-K3 solves 60.5% of test tasks without skills, with only 3–4 points of gain on abductive and partial-order tasks, suggesting that many tasks are skill-helpful rather than skill-critical.

  4. The experiments lack strong baselines. The main comparison is only between each base model and its SFT version. 1) The comparison with SKT copies numbers from its paper under a different harness, rather than training on SKT data under the same setting; 2) there is no comparison with other agentic training data of similar scale (e.g., [3, 4, 5]) or with a simple baseline such as SFT on the same number of teacher trajectories collected without skills, so it is unclear whether the gains come from skill-grounded tasks or simply from distilling stronger teachers; 3) no training-free skill-use methods or RL-based alternatives are compared.

  5. The evidence for improved skill use is weak. 1) The SkillGym test set comes from the same pipeline as the training data, while on the human-curated SkillsBench the gains are much smaller, the 2B model degrades, and improvements appear only for skills that provide reference documentation; 2) the largest gain on Skill-Use-Bench comes from the Trigger score (28 to 96), which may mainly reflect learning to open the skill file rather than applying it, as acknowledged by the authors; 3) the SkillGym test set and Skill-Use-Bench are evaluated with a single run, and all data-ablation models fall below the base model on SkillEval, making these conclusions unreliable.

[1] SKT: Skill-Use Training at Scale via Verified Synthetic Data Generation. arXiv:2608.02287, 2026.

[2] SkillsVote: Lifecycle Governance of Agent Skills from Collection, Recommendation to Evolution. arXiv:2605.18401, 2026.

[3] Endless Terminals: Scaling RL Environments for Terminal Agents. arXiv, 2026.

[4] SETA: Scaling Environments for Terminal Agents. arXiv, 2026.

[5] LiteCoder-Terminal: Scaling Long-Horizon Terminal Environments for Learning Language Agents. arXiv:2605.29559, 2026.

Questions for authors
  1. SkillEval, SkillsBench, and Skill-Use-Bench also draw on publicly available skills, some from the same sources (e.g., skills.sh) as SkillGym. Did you check for overlap between the training skills or tasks and the skills used in these external benchmarks, and were overlapping skills removed before training?

  2. On the SkillGym test set, success on held-out skills is higher than on held-in skills for both the base and the SFT model. How are the two splits constructed and matched in difficulty and domain? Does this indicate that the held-out split is easier, and if so, how should the claimed generalization to unseen skills be interpreted?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 8gYd # 1

Thank you for your excellent review and we try our best to address them as follows:

W1

Limited novelty. Synthesizing verifiable tasks from public skills and distilling teacher trajectories via SFT has already been done by SKT [1] … The claimed differences, namely four reasoning-structure profiles and an LLM reviewer, are incremental engineering choices, and the training recipe is plain SFT distillation without any methodological contribution.

We agree that some of the building blocks, such as iterative building and SFT on teacher trajectories, are standard, and we do not claim them as our main contributions. SKT is a close work that appeared around one month before our work, and we regard it as concurrent work. At the time of submission SKT had not released its training data or a checkpoint, which is why the paper could only quote its reported numbers.

Our main contribution is a large open resource and an empirical study of what training on it changes. SkillGym covers 3,494 skills and 6,772 tasks across 19 domains, collects trajectories in three agent harnesses, and is used to train six models from 2B to 122B parameters. Every task must fail on an empty attempt as well as pass on its reference solution, and a reviewer model examines each verifier for defects, the effect of which we measure. The paper also studies questions that prior work on skill-use training leaves open, including what a model learns to do with a skill, whether prompting alone achieves the same, and whether general agent ability is retained. The full corpus will be released. SkillsVote [2] profiles skill libraries and synthesizes tasks to record skill behavior, but does not train models on them.

SKT has since released part of its trajectories, so we can now compare under the same setting. We fine-tuned the same base model with the same recipe on SKT's released data and on a SkillGym subset with the same number of tokens (36M), and evaluated both on our test set in three agent harnesses. SKT's released trajectories were collected in OpenCode, and the SkillGym subset in mini-swe-agent, Terminus-2 and AgentFly.

Training datamini-swe-agentTerminus-2OpenCode
None (base model)42.637.834.4
SKT (released data)37.035.541.3
SkillGym subset44.245.244.4

In OpenCode the SkillGym subset is slightly better than SKT although it contains no OpenCode trajectories, and in the other two harnesses it is ahead by 7 to 10 points. The revised paper adds this comparison in Appendix F.

W2

The task construction process lacks essential details. … Key information is missing: 1) which models serve as builder and reviewer; 2) the pass and skip rates at each gate, which would explain why only 3,494 of 11,897 curated skills yield tasks; 3) how the test split and held-out skills are selected, and whether tasks from the same skill can appear in both training and held-in test sets; 4) the dataset statistics are inconsistent …

We answer your four points in order.

  1. Environments are built by DeepSeek-V4-Pro. Tasks are built by DeepSeek-V4-Pro, GLM-5.2 and MiniMax-M3. The reviewer is DeepSeek-V4-Pro for 96.5% of tasks and GLM-5.2 for the rest. Trajectories are produced by Kimi-K3, DeepSeek-V4-Flash and GLM-5.2.

  2. The gap between 11,897 and 3,494 is mostly a sampling decision. Most curated skills are software-engineering skills, so we capped that domain to keep the domains balanced, and 6,395 skills were not attempted. Of the 5,502 skills we attempted, 3,494 produced at least one accepted task. Among the remaining 2,008, 751 were rejected by the reviewer, 714 failed the validity checks, 475 were skipped by the builder, and 68 failed for infrastructure reasons.

  3. Skills are assigned to the two sides at random by skill family. The held-out skills never appear in training. Each side of the test set has 50 tasks for each of the four task types. Tasks from the same skill can appear in both the training set and the held-in test set, and this is by design. Held-in measures new tasks on seen skills, and held-out measures unseen skills. No test task appears in training.

  4. The 6,772 tasks are 5,191 that the reviewer approved and 1,581 that passed every execution check but that the reviewer still considered too strict. A verifier that is too strict can reject a correct solution but cannot accept an incorrect one, so trajectories that pass it are safe to train on. Partial-order tasks were built in a separate run and were omitted from that count. They are part of the training set. The final training set contains 19,070 trajectories from 4,934 tasks, namely the tasks that at least one teacher solved, excluding those of the held-out skills.

The revised paper adds these details in Sections 3.2 and 4.1, replaces Tables 8 and 9 with a single table that follows the tasks from construction to the training set (Appendix K), and gives the full prompts in the supplementary material.

AuthorsOpenReview ↗

Response to Reviewer 8gYd # 2

W3

Quality control is insufficient for a single cross-domain pipeline. 1) The same pipeline is applied to all 19 domains … writing skills are excluded entirely; 2) quality relies entirely on automatic checks and an LLM reviewer whose transcripts were not stored, and no human or expert audit of tasks or verifiers is reported; 3) skill dependency is not verified per task, unlike SKT … many tasks are skill-helpful rather than skill-critical.

  1. The process is shared across domains, but the verifier is not. Each verifier is written for its own task from the content of the skill, so the correctness criteria are specific to the task. The checks also behave consistently across the 19 domains. Outside software engineering, between 84% and 92% of built tasks pass the validity checks in every domain, and between 41% and 57% pass review. Writing skills are excluded since their outputs are subjective and would need an LLM judge rather than an executable check, so we left them out to keep every task verifiable by execution. Together with skills about agents themselves, this excludes 9% of the skill pool.

  2. In the submitted version, quality is verified mainly by execution and the reviewer model. Every task must pass on its reference solution and fail on an empty attempt, and a reviewer model then examines the verifier. To examine the verifiers directly, we audited the graders of 50 randomly sampled training tasks for problems that make the verdict on the task's outcome unreliable, such as grading that is too strict and checks requirements the task does not explicitly state. For comparison, we applied the same procedure to other datasets. Because of the time limit of the rebuttal, we checked 20 samples from each of them.

DatasetTasks checkedTasks with a grader problem
SkillGym (ours)507 (14%)
Skill-Use-Bench203 (15%)
SkillsBench206 (30%)
Endless Terminals206 (30%)
SkillEval209 (45%)

The rate for SkillGym is at the low end of these datasets, including the human-written SkillsBench. Given the sample sizes, we only read this comparison as indicative.

  1. Making tasks skill-helpful rather than skill-critical is a deliberate choice. Whether a skill is needed depends on the model that attempts the task. Of the 42 test tasks that the 9B base model solves only with the skill, Kimi-K3 solves 35 without it, so a filter run with one model does not hold for another. SKT's filter is an alternative, and we chose instead to keep tasks that a strong model can solve unaided, since they still require a weaker model to consult and apply the skill. The tasks do reward skill use for the models we train. On our test set the skill raises the pass rate of the 9B model by 7.5 points before training and by 16.5 points after it. The revised paper reports the audit in Appendix N.

W4

The experiments lack strong baselines. … 1) The comparison with SKT copies numbers from its paper under a different harness …; 2) there is no comparison with other agentic training data of similar scale (e.g., [3, 4, 5]) or with a simple baseline such as SFT on the same number of teacher trajectories collected without skills …; 3) no training-free skill-use methods or RL-based alternatives are compared.

  1. We now compare with SKT under the same setting, as reported under W1.

  2. Among the datasets you cite, LiteCoder-Terminal-SFT [5] provides the largest released set of trajectories, so we trained the same base model on it with the same number of tokens (36M) and the same recipe. In Terminus-2, the harness it was collected in, it reaches 25.2 on our test set, against 45.2 for the SkillGym subset and 37.8 for the base model.

We also ran the baseline you describe. For 1010 training tasks, we collected trajectories from the same teacher in the same harness with and without the skill, and trained one model on each set.

Trained onTest setReads the skillSkill-Use-Bench SU
Trajectories with skills47.361.7%57.0
Trajectories without skills46.819.1%13.4
None (base model)42.623.0%18.8

Both models improve the pass rate over the base model by a similar amount, so at this scale that gain comes from the tasks and the teacher. Skill use, however, is learned only from trajectories collected with skills. The model trained without them reads and follows skills no more than the base model.

  1. As a training-free baseline, we instructed the base model explicitly to read the relevant skill before acting. As reported under W5, this does not improve it. Training-free methods act at inference time and can be combined with a trained model. RL-based alternatives are time-consuming and outside the scope of this paper, and we leave them to future work.

The revised paper reports the comparison with public data in Appendix F, the baseline without skills in Appendix O, and the prompt-only control in Appendix H.

AuthorsOpenReview ↗

Response to Reviewer 8gYd # 3

W5

The evidence for improved skill use is weak. 1) The SkillGym test set comes from the same pipeline as the training data, while on the human-curated SkillsBench the gains are much smaller, the 2B model degrades, and improvements appear only for skills that provide reference documentation; 2) the largest gain on Skill-Use-Bench comes from the Trigger score (28 to 96) …; 3) the SkillGym test set and Skill-Use-Bench are evaluated with a single run, and all data-ablation models fall below the base model on SkillEval …

  1. Three of our four benchmarks are from other work, and the trained model improves on all of them. On our own test set, half of the tasks use skills that never appear in training, and the gain there (+19.5) is as large as on seen skills. On SkillsBench the gain grows with model size, from +7.6 at 9B to +14.8 at 27B and +23.5 at 122B. The 2B model is the only one of the six that does not improve on it.

  2. Consulting a provided skill is itself a capability, and one that prompting does not produce. When we instruct the base model explicitly to read the relevant skill first, it reads the skill no more often (21.9% of test tasks against 23.0%) and does not score higher. The gain is also not limited to consultation. On the tasks where both the base and the trained model read the skill, the trained model still solves more of them.

Tasks where both models read the skillBaseSFT
Our test set, pass rate33.762.0
SkillEval, strict26.636.7
SkillsBench38.236.8

On SkillsBench the two models are level on these tasks, and the gain there comes from the trained model consulting skills more often. On Skill-Use-Bench we agree that the gain comes mostly from the Trigger score.

  1. For the two benchmarks evaluated with a single run, we computed paired comparisons over tasks. On our test set the trained model gains 18.2 points (95% CI 14.0 to 22.5), with 81 tasks solved only by the trained model and 8 only by the base. On Skill-Use-Bench the SU score gains 34.8 points (95% CI 28.4 to 41.1). The data-ablation models are trained on small token-matched subsets and are meant to be compared within each pair. They fall below the base model only on SkillEval's partial-credit pass rate. On its strict metric, where every check must pass, all four are at or above the base (18.3 to 26.3 against 17.3).

Q1

Did you check for overlap between the training skills or tasks and the skills used in these external benchmarks, and were overlapping skills removed before training?

Yes. No benchmark task appears in our training data. Every training task is synthesized by our pipeline from the content of a skill, and the benchmark tasks were written independently by their authors, so the two can share a skill but not a task. We compared our tasks with the benchmark tasks on shared skills, and they differ in instruction, environment and verifier.

At the level of skills, we matched by source, content and name. A small number of benchmark skills also occur in our training set, and these remain in it.

BenchmarkSkillsSame content as a training skillSame name onlyTasks affectedTasks with no overlap
SkillsBench202101028 of 8759
SkillEval18616923 of 10077
Skill-Use-Bench782213 of 175162

To confirm that these shared skills do not drive our results, we recomputed the gain of the trained model over the base model on the tasks with no overlapping skill.

MetricAll tasksTasks with no overlap
SkillEval pass rate+9.0+10.3
SkillEval strict+16.0+16.9
Skill-Use-Bench SU+34.8+35.0
SkillsBench+7.8+5.5

The gain is unchanged on SkillEval and Skill-Use-Bench, and smaller but still positive on SkillsBench. On our own test set, the 150 held-out skills never appear in training. The revised paper includes both tables in Appendix G.

AuthorsOpenReview ↗

Response to Reviewer 8gYd # 4

Q2

How are the two splits constructed and matched in difficulty and domain? Does this indicate that the held-out split is easier, and if so, how should the claimed generalization to unseen skills be interpreted?

Skills are assigned to the two sides at random by skill family, with 150 skills on each side. The two sides are matched in task type, with 50 tasks of each of the four types, and in domain, where the counts of the 19 domains differ by at most 3.

To measure whether one side is easier, we used 16 models that were never trained on our data.

Pass rateHeld-inHeld-outDifference
Average of 16 models never trained on our data53.057.0+4.0
Qwen3.5-9B40.042.5+2.5
Qwen3.5-9B with SkillGym SFT57.062.0+5.0

The held-out side is indeed slightly easier, by about 4 points, and this comes almost entirely from the partial-order tasks. The higher held-out score of the trained model therefore reflects the split rather than an advantage on unseen skills. Generalization should be read from the gain within each side, which is +17.0 on seen skills and +19.5 on unseen skills. Training transfers to unseen skills as well as it does to seen ones. The revised paper states the claim in this form in Section 4.2.

We hope our response could resolve your concerns. And if you still have questions or unresolved concerns, we are happy to answer further.

Reviewer_ewfKOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

SkillGym converts public skill packages into executable training environments using a builder-reviewer pipeline. Tasks include reference solutions and outcome verifiers, and successful trajectories from several teachers are used for SFT. The paper reports broad improvements across five backbones and four benchmarks, including held-out skills. Its analyses find a large increase in skill consultation, benefits from quality-reviewed training tasks, and uneven transfer to tasks requiring domain methods or bundled tools.

Critical strengths
  1. Constructing training environments from public skills is practically useful. The pipeline explicitly checks initial failure, reference success, and potential verifier defects rather than relying solely on generated task descriptions.
  2. The evaluation spans multiple model families and includes human-written tasks, skill-access ablations, held-out skills, and token-matched data comparisons. The authors also report negative results for some models and task categories.
Critical weaknesses
  1. The training appears to improve skill consultation more than skill application. Figure 5 shows a large gain on reference-documentation tasks, but little improvement on tasks involving domain methods, bundled tools, or modifications to existing systems. A base model explicitly prompted to consult skills would be a useful control for assessing what SFT adds beyond this behavior.

  2. The evaluation does not establish broader agent capability gains or retention. The benchmarks are largely skill-oriented, whereas the generated trajectories involve tool calls, multi-step execution, and diagnosis. I would expect training on these trajectories to help with practical agent tasks beyond the skill-use setting. Evaluation on independent benchmark such as Terminal-Bench would clarify whether these abilities transfer and whether existing capabilities are preserved.

  3. The final training corpus and verifier reliability need clarification. Table 8 includes 1,581 unresolved tasks while excluding separately constructed partial-order tasks; Table 9 accounts for all successful trajectories using three harnesses despite the four-harness description. It is unclear how these counts map to the actual training data.

  4. The contribution beyond SKT is not sufficiently isolated. The main differences include task structures and quality review, but the four-profile mixture does not outperform procedural-only training in Table 2. The review-approved comparison is encouraging, although token matching does not control differences in skill coverage or task difficulty.

Questions for authors
  1. Failed trajectories may also contain useful information about mistakes and recovery. Have the authors considered using the collected failed trajectories for training, rather than relying only on successful ones?

  2. How does the base model perform when explicitly prompted to read the relevant skill before acting? How much of the SFT gain remains over this prompt-only baseline?

4 comments
AuthorsOpenReview ↗

Response to Reviewer ewfK # 1

Thank you for your helpful review and we really appreciate it! To your concerns, we address them as follows.

W1

The training appears to improve skill consultation more than skill application. Figure 5 shows a large gain on reference-documentation tasks, but little improvement on tasks involving domain methods, bundled tools, or modifications to existing systems. A base model explicitly prompted to consult skills would be a useful control for assessing what SFT adds beyond this behavior.

Thanks for your suggestions. We ran the control you suggested. We added an explicit instruction to read the relevant skill before acting to the system prompt of the base model and evaluated it on our test set, SkillEval and Skill-Use-Bench.

BaseBase with explicit promptSFT
Reads the skill, our test set23.0%21.9%80.0%
Pass rate, our test set42.640.859.5
Reads the skill, SkillEval33.1%28.7%96.3%
SkillEval strict17.714.033.3
Skill-Use-Bench SU18.811.349.6

The explicit prompt changes neither how often the base model reads the skill nor its score, probably because of the limited long-context and instruction-following capability of a 9B model. So the full gain of SFT remains over this prompt-only baseline. Consulting a provided skill is therefore a behavior that training produces and prompting does not.

To measure application separately, we compared the two models only on tasks where both of them read the skill.

Tasks where both models read the skillBaseSFTTasks
Our test set, pass rate33.762.092
SkillEval, strict26.636.7109
SkillsBench38.236.868

On our test set and on SkillEval the trained model also applies a skill better once it has read it, and on our test set these tasks account for 36% of the total gain. On SkillsBench it does not, and the gain there comes from tasks where only the trained model read the skill. This is consistent with your reading of Figure 5. The revised paper reports this control and the full breakdown of the gain in Appendix H.

W2

The evaluation does not establish broader agent capability gains or retention. … Evaluation on independent benchmark such as Terminal-Bench would clarify whether these abilities transfer and whether existing capabilities are preserved.

We ran the evaluation you suggested. On Terminal-Bench 2.0, with no skills provided, we compared Qwen3.5-27B before and after SkillGym training over three runs of 87 tasks each.

Terminal-Bench 2.0, no skillsRun 1Run 2Run 3Mean
Qwen3.5-27B29343035.6%
Qwen3.5-27B with SkillGym SFT40334545.2%

The trained model does not regress on this independent benchmark and is better on average, although the runs vary and we do not claim more than that. We ran the 27B model because the 9B model is at the floor on Terminal-Bench, with 2 tasks solved before training and 4 after.

For the 9B model we instead removed the skills from the skill-oriented benchmarks, which measures general task ability on the same tasks.

Qwen3.5-9B, skills withheldBaseSkillGym SFT
Our test set37.143.0
SkillsBench5.8 ± 3.39.4 ± 3.7

The trained model remains ahead when no skill is available, so the gains are not limited to skill use. The revised paper adds these results in Appendix I.

AuthorsOpenReview ↗

Response to Reviewer ewfK # 2

W3

The final training corpus and verifier reliability need clarification. Table 8 includes 1,581 unresolved tasks while excluding separately constructed partial-order tasks; Table 9 accounts for all successful trajectories using three harnesses despite the four-harness description. It is unclear how these counts map to the actual training data.

The counts in the paper are correct, but they were spread over two tables and did not show how one leads to the other. The last rows of the full accounting are shown here.

ProceduralAbductiveConstraint satisfactionPartial orderTotal
Tasks solved at least once by a teacher1,0672,1241,5577355,483
Tasks in the training set9441,9251,3617044,934
Training trajectories4,8908,3464,4901,34419,070

Trajectories are collected on all accepted tasks. The training set keeps the tasks that at least one teacher solved, and excludes the tasks of the 150 held-out skills.

On your three specific points. The 1,581 unresolved tasks passed every execution check, and the only remaining objection of the reviewer model was that their tests are too strict. A verifier that is too strict can reject a correct solution but cannot accept an incorrect one, so trajectories that pass it are safe to train on. Of these tasks, 777 are in the training set. The partial-order tasks were built in a separate run and Table 8 omitted them. They are part of the training set, with 704 tasks and 1,344 trajectories. Finally, trajectories were collected in four harnesses and three are used for training, mini-swe-agent (6,433), Terminus-2 (6,701) and AgentFly (5,936), which are the counts in Table 9. The OpenCode trajectories were collected later and are not part of the training set. We are sorry to cause any confusion and the revised paper replaces Tables 8 and 9 with a single table that follows the tasks from construction to the training set (Appendix K).

W4

The contribution beyond SKT is not sufficiently isolated. The main differences include task structures and quality review, but the four-profile mixture does not outperform procedural-only training in Table 2. The review-approved comparison is encouraging, although token matching does not control differences in skill coverage or task difficulty.

SKT released part of its training data after our submission, so we can now compare with it directly instead of quoting its reported numbers. We fine-tuned the same base model with the same recipe on SKT's released trajectories and on a SkillGym subset with the same number of tokens (36M), and evaluated both on our test set in three agent harnesses. SKT's data was collected in OpenCode, and the SkillGym subset in mini-swe-agent, Terminus-2 and AgentFly.

Training datamini-swe-agentTerminus-2OpenCode
None (base model)42.637.834.4
SKT37.035.541.3
SkillGym subset44.245.244.4

In OpenCode, SKT's own harness, the SkillGym subset is slightly better although it contains no OpenCode trajectories, and in the other two harnesses it is ahead by 7 to 10 points. SKT's data improves the base model only in the harness it was collected in.

You are right that the four-profile mixture does not outperform procedural-only training at equal tokens, and we do not claim that it does. The purpose of the additional profiles is coverage. The procedural-only row of Table 2 already uses every procedural trajectory in the corpus (4,890 trajectories, 132M tokens), so procedural-only training cannot be scaled beyond it, and it reaches 42.5 on our test set. The other three profiles supply 74% of the training trajectories, and of the 3,425 skills for which we built tasks, 1,670 (48.8%) have only non-procedural tasks. With them, the full corpus reaches 59.5.

The revised paper states this in Section 5.3 and reports the comparison with SKT in Appendix F.

AuthorsOpenReview ↗

Response to Reviewer ewfK # 3

Q1

Failed trajectories may also contain useful information about mistakes and recovery. Have the authors considered using the collected failed trajectories for training, rather than relying only on successful ones?

Yes. We trained the same base model with the same recipe on the successful trajectories together with 4,163 failed ones from mini-swe-agent, used as ordinary supervised targets.

Training dataOur test setSkillsBenchSkillEval pass rateSkillEval strictSkill-Use-Bench SU
Successful trajectories only59.522.4 ± 4.274.7 ± 1.233.3 ± 3.249.6
Successful and failed trajectories60.521.0 ± 3.974.2 ± 0.534.0 ± 2.049.9

Adding the failed trajectories makes no measurable difference on any benchmark. Used as plain supervised targets they neither help nor hurt, which is why the corpus keeps the successful ones. We agree that they may carry useful signal for methods designed to learn from failure, such as preference optimization or reinforcement learning. We keep all failed trajectories and will release them together with the corpus. The revised paper includes this experiment in Appendix J.

Q2

How does the base model perform when explicitly prompted to read the relevant skill before acting? How much of the SFT gain remains over this prompt-only baseline?

Please see our response to W1, where we report this control. The explicit prompt does not improve the base model, and the full gain of SFT remains.

We hope our response could address your concern. And if you still have anything unclear or unresolved concerns, we are happy to respond further. Thank you again for your review.

Reviewer_ewfKOpenReview ↗

Response to Rebuttal

Thank you for the reply and the additional experiments. Some of my concerns have been addressed, but the following still remain.

For W1, The model did not follow the explicit instruction to read the skill, so there is only so much we can conclude from this experiment. On tasks where both models read the skill, the results are mixed and do not fully address my concern. One follow-up question: how many tasks did both models read the skill on, and what percentage of each benchmark was that?

For W2, The 9B model’s poor performance on Terminal-Bench may suggest that the proposed training mainly teaches the model to use skills, rather than improving its broader agentic abilities, such as tool calls, multi-step execution, and diagnosis.

For Q1, using failed trajectories directly for SFT does not seem very reasonable to me. As you mentioned, they may be better suited for preference optimization or reinforcement learning.

Reviewer_rJ5yOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes to help agents learn to use skills by synthesizing environments with difficulty control, and verifiers. This helps to collect successful skill-use trajectories for these agents, and further improves these agent's capabilities with using these skills after training on these trajectories.

Critical strengths

The paper is written clearly to demonstrate the performance gain after synthesizing the environment and use the validated trajectories, and also demonstrated that the skills the models learn to use are important.

Critical weaknesses

The environment synthesizing and data collection process is kind of like a black box, since the method is already using multiple models for this synthesizing process, a few problems occur:

(1) how to guarantee that there is not any skill / test set leakage during the synthesize process: the environment generated could actually be similar to the test set due to the perception of knowledge of these models during pre-training stage; it would look like a distillation if these knowledge are from stronger models.

(2) Necessary comparisons is needed with a baseline, where the agents generating the synthetic environments are directly used to produce skill using trajectories and empower the base model, in a way of offline distillation or OPD.

(3) What is the token cost of synthesizing the environments ? is it worth to synthesize even the environment to synthesize training data ?

Questions for authors
  1. Does the authors have human evaluation and verification for the environments, verifiers, and generated trajectories ?

  2. Is this learning paradigm generalizable to any skills ? when skills are complex themselves, the environments could be exponentially becoming more complex, and seems that this way if learning is not scalable.

3 comments
AuthorsOpenReview ↗

Response to Reviewer rJ5y #1

Thank you for your insightful review! We now address each of your concerns as below:

W1

how to guarantee that there is not any skill / test set leakage during the synthesize process: the environment generated could actually be similar to the test set due to the perception of knowledge of these models during pre-training stage

Thank you for raising this. Before we answer, we would like to clarify that we don't aim to have a zero overlap for skills in benchmarks. And our synthesizing pipeline has no context of these benchmarks, therefore, the generated tasks are different. To your concern, we measured the overlap between our training skills and the skills of each benchmark, and show the gains do not depend on it.

  • On our own test set, half of the tasks use held-out skills that never appear in training. The gain on these skills is +19.5 points, compared with a similar level of +17.0 on seen skills.

  • For the external benchmarks, we matched skills by source, content and name. A small number overlap with our training set, as shown below.

BenchmarkSkillsSame content as a training skillSame name onlyTasks affectedTasks with no overlap
SkillsBench202101028 of 8759
SkillEval18616923 of 10077
Skill-Use-Bench782213 of 175162
  • We then recomputed the gain of the trained model over the base model on the tasks with no overlapping skill. The gain is unchanged on SkillEval and Skill-Use-Bench. On SkillsBench it is smaller but remains positive. Where a benchmark shares a skill with our training set, the tasks built on it differ.
MetricAll tasksTasks with no overlap
SkillEval pass rate+9.0+10.3
SkillEval strict+16.0+16.9
Skill-Use-Bench SU+34.8+35.0
SkillsBench+7.8+5.5

Regarding knowledge acquired by the builder models during pre-training, a builder is given only the skill's own files and never sees a benchmark task. The results above also bound the effect of any such similarity, since the gains hold on the benchmark tasks whose skills never appear in our training set.

We address your related point on distillation from stronger models under W2. The revised paper includes both tables in Appendix G.

W2

Necessary comparisons is needed with a baseline, where the agents generating the synthetic environments are directly used to produce skill using trajectories and empower the base model, in a way of offline distillation or OPD.

Thank you for your baseline suggestion. Our training is itself offline distillation from strong models. The base model is fine-tuned on trajectories produced by Kimi-K3, DeepSeek-V4-Flash, and GLM-5.2. What the synthesized environments contribute is the task for these models to act on and a verifier for the outcome, since a skill-use trajectory cannot be produced without an executable task. The models that build the tasks and the models that solve them are largely different. Only 2.4% of training trajectories come from a model solving a task it built.

To test whether distilling strong models is sufficient without our environments, we fine-tuned the same base model on two public sets of strong-model agent trajectories and on a SkillGym subset, with the same number of tokens (36M) and the same recipe. SKT's released data was collected in OpenCode, LiteCoder-Terminal-SFT in Terminus-2, and the SkillGym subset in mini-swe-agent, Terminus-2 and AgentFly. The table reports the pass rate on our test set in three agent harnesses.

Training datamini-swe-agentTerminus-2OpenCode
None (base model)42.637.834.4
SKT (OpenCode)37.035.541.3
LiteCoder-Terminal-SFT (Terminus-2)4.525.20.0
SkillGym subset (three harnesses, no OpenCode)44.245.244.4
AuthorsOpenReview ↗

Response to Reviewer rJ5y # 2

In OpenCode, the harness SKT's data was collected in, the SkillGym subset is slightly better than SKT (44.4 against 41.3) although it contains no OpenCode trajectories. In the other two harnesses it is ahead by 7 to 10 points, and in Terminus-2 it reaches 45.2 against 25.2 for LiteCoder, which was collected there. Data from a single harness does not carry over to the others. SKT improves the base model only in OpenCode, and the model trained on LiteCoder keeps answering in Terminus-2's command format, which the other harnesses do not accept. The SkillGym subset stays at 44 to 45 in all three. This paper studies the offline setting. On-policy distillation would use the same environments and is complementary to it, which we would leave to future work.

If you had a different baseline in mind, for example one that uses no executable environment at all, we would be glad to clarify and add it.

W3

What is the token cost of synthesizing the environments ? is it worth to synthesize even the environment to synthesize training data ?

Task construction, review and trajectory collection cost between 5.7k USD and 8.4k USD in tokens, which is about 1 USD per released task (appendix cost table). Building the environments adds about 0.2k USD. We believe this cost is justified for following reasons.

First, it is a one-time cost and the result is reused. Each environment is built once per skill, and each task is then solved by three teacher models in several agent harnesses, so a single task yields multiple verified trajectories. The same corpus trained all six models in the paper, from 2B to 122B parameters, without any additional construction. The tasks also serve as our evaluation set, and they can be used directly as reinforcement learning tasks. The return is large relative to the cost. Training on this corpus improves the base model by 18.2 points on our test set and transfers to three external benchmarks written by other groups, for example +16.0 on SkillEval strict and +34.8 on Skill-Use-Bench. Human-written skill tasks of comparable scale and complexity would cost far more than 1 USD each.

Q1

Does the authors have human evaluation and verification for the environments, verifiers, and generated trajectories ?

In the submitted version, quality is mainly verified by our automatic pipeline. Every task must pass checks in the sandbox, and is then examined by an independent reviewer model. Every training trajectory passed its task's verifier by execution.

During the rebuttal phase, we audited the graders of 50 randomly sampled training tasks. We then checked for grader problems that make the verdict on the task's main outcome unreliable, such as grading that is too strict and checks requirements the task does not explicitly state. For comparison, we applied the same procedure to other datasets. Because of the time limit of the rebuttal, we checked 20 samples from each of them.

DatasetTasks checkedTasks with a grader problem
SkillGym (ours)507 (14%)
Skill-Use-Bench203 (15%)
SkillsBench206 (30%)
Endless Terminals206 (30%)
SkillEval209 (45%)

The rate for SkillGym is at the low end of these datasets, including the human-written SkillsBench. However, given the sample sizes, we only read this comparison as indicative. The revised paper reports this audit in Appendix N.

Q2

Is this learning paradigm generalizable to any skills ? when skills are complex themselves, the environments could be exponentially becoming more complex, and seems that this way if learning is not scalable.

The complexity of an environment does not grow with the complexity of the skill. An environment does not have to cover the whole skill. Each one is built around a single use of the skill, and its size is set by the task type rather than by the size of the skill. A complex skill therefore yields tasks of the same scale as a simple one, and the cost grows linearly with the number of skills. The table groups the skills in our pool by size.

SkillsTasks accepted per attempt
Shortest quarter of SKILL.md files39.2%
Longest quarter of SKILL.md files37.0%
1 to 2 files38.4%
21 or more files32.8%

The rate at which accepted tasks are produced changes little with the size of the skill. The trained model also generalizes to skills it has never seen, which we have shown in the paper.

We hope the response could address your concerns, and we would like to hear from you if there are further questions and suggestions.

Thanks for the comprehensive response for the paper. I think my concerns are partly resolved. I appreciate the efforts that the authors have made to substantially improve the validity of the environment, the validity of learned skills. I think this work could be valuable for industrial applications instead of academic predominantly though, and I'm not fully convinced of whether this work would help the line of research in RSI and AI4AI.

I would like to raise my ratings to borderline in light of these explanations. Given ICLR's current rating policy, I have adjusted my score to 3 while lower my confidence to 2, I also think this paper might be more suitable or valuable or a demo track.

Good Luck.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.