- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces Skill Transfer, a training-free framework that adapts skills written for strong agents to weaker agents. It abstracts paired strong and weak trajectories into typed execution structures, identifies procedural gaps between them and converts them into structured patches to the skill. Experiments show that the adapted skills consistently improve weak-agent performance over the original skills and generally outperform skill-evolution baselines.
Critical strengths
- The paper studies an important and practically relevant problem: how to make skills developed for strong agents usable by weaker and cheaper models without retraining them.
- The empirical results are strong and consistent across multiple benchmarks and both GPT and Qwen model families, with clear improvements over the original skills and the skill-evolution baseline.
- The core idea is intuitive and well motivated: explicitly identifying the procedural steps that strong agents implicitly supply, and then incorporating those missing steps into the skill, is a reasonable way to close the strong–weak agent gap.
Critical weaknesses
This paper seems to provide a nice solution to the important problem of adapting skills developed for strong agents to weaker agents, but I have several concerns about the current method and evaluation:
(1) The role of GED in gap localization is unclear. Although GED is presented as a central component for comparing strong and weak execution structures, it appears to provide only a global scalar measure of trajectory divergence. The actual localization of missing_node, weak_node, order_difference, etc. is performed through separate label matching and LLM-based comparison of node details. As a result, it is unclear whether GED itself contributes to localization at all, or is mainly an auxiliary similarity metric. The paper also does not provide an ablation that isolates GED from the rest of the framework.
(2) The skill coverage is limited. The evaluation only considers three benchmark-level skill configurations, built from a small number of spreadsheet, finance/retrieval, and data-analysis skills. The paper does not demonstrate whether the method generalizes to a broader range of skills with substantially different structures, domains, lengths, or levels of procedural complexity, which limits the strength of the claim that Skill Transfer is broadly applicable to agent skills.
Questions for authors
(1) The current evaluation covers only a small number of skill configurations. Could the authors provide more characterization of the evaluated skills, and ideally evaluate on a broader and more diverse set of skills?
(2) Can all trajectory steps and strong–weak differences be reliably captured by the six fixed Tier-1 categories and the five predefined gap types, and how does the framework handle ambiguous, multi-role, or out-of-taxonomy behaviors?
4 comments
Response to Reviewer 96HC 2/2
Q2. Coverage of the Tier-1 categories and gap types
Tier-1 categories. Table R12 gives the Tier-1 distribution of all nodes abstracted during training on the four benchmarks, including the new SeqQA2 domain. Only 6 nodes (under 0.1%) fall into other. Each of them is the final node of a run and contains no tool call. It is a note that the abstractor adds to describe how the run ended, for example that the agent stopped before writing the result file, so none of them is a step the agent performed. A step that serves several purposes is split into one node per purpose, and domain-specific detail goes into the free-text Tier-2 label, so the fixed Tier-1 set only has to capture the procedural role.
Table R12. Share of nodes (%) in each Tier-1 category.
| Benchmark | setup | execute | validate | recover | output | other |
|---|---|---|---|---|---|---|
| SpreadsheetBench | 40.5 | 19.4 | 13.2 | 9.1 | 17.8 | 0.1 |
| DABench | 45.8 | 25.3 | 1.3 | 5.2 | 22.3 | 0.0 |
| OfficeQA | 38.8 | 32.7 | 2.9 | 4.8 | 20.6 | 0.1 |
| SeqQA2 | 43.3 | 22.5 | 3.1 | 9.1 | 21.9 | 0.1 |
Gap types. An execution structure is a labelled path graph, and the four operations of graph edit distance [3, 4] suffice to transform any such graph into any other. With the strong run as the reference, each operation is one gap type. A deleted node is a missing_node, a substituted node (matched, but executed to a lower bar) is a weak_node, an inserted node is an extra_wrong_node, and an edited edge (two matched steps in opposite order) is an order_difference. No structural difference therefore falls outside these four. The fifth type, wrong_skill_routing, is designed for skill use. When the weak agent does not activate a skill the strong agent used, that skill's description is edited so that the weak agent selects it. These are in §3.3 and Appendix J of the revised paper.
References
[1] Laurent et al. LABBench2: An Improved Benchmark for AI Systems Performing Biology Research. arXiv:2604.09554, 2026.
[2] GPTomics. bioSkills: a library of bioinformatics skills for AI agents. Open-source skill library, 2026.
[3] Sanfeliu and Fu. A distance measure between attributed relational graphs for pattern recognition. IEEE Transactions on Systems, Man, and Cybernetics, 1983.
[4] Gao et al. A survey of graph edit distance. Pattern Analysis and Applications, 2010.
Thank you for the clarification. My concerns have been addressed, and I will raise my rating accordingly. I encourage the authors to incorporate these additional experiments into the final version of the paper.
We sincerely thank the reviewer for the positive feedback and for raising the rating. The additional experiments are already included in the revised paper. The ablation of GED is reported in Appendix A. The SeqQA2 experiment and the characterization of the evaluated skills are in Appendix I. The coverage of the Tier-1 categories is in Appendix J, and the correspondence between the gap types and the graph edit operations is described in §3.3. We will keep all of these additions in the final version.
Response to Reviewer 96HC 1/2
We appreciate the reviewer's constructive feedback and respond to each weakness and question below. The revised paper highlights all changes in teal.
W1. The role of GED and its ablation
Thank you for raising this important point. GED is a global scalar and does not localize gaps. Gaps are localized and typed by the label matching and the node details (§3.3). GED summarizes the structural differences in one number and has two roles. During training, the diagnoser sees each task's normalized GED per iteration as an auxiliary progress signal. Its prompt states that a GED decreasing across iterations means the structural gap is closing, and that a flat or increasing GED means the current approach is not working. At evaluation time, GED measures how far the weak agent's procedure on held-out tasks is from the strong agent's (§4.3).
We added the ablation that isolates GED (w/o GED). It removes GED from the diagnoser's case listing, case history and prompt, and keeps abstraction, validator reports, gap taxonomy, cross-iteration statuses and patching rules unchanged. We also report the raw-trajectory arm of Appendix A (w/o structure), which removes the whole structured pipeline.
Table R9. Test pass rate (%). Each ablation is one training run. The original skill and Skill Transfer are means over 5 runs.
Removing GED lowers the pass rate on SpreadsheetBench (44.4 → 42.5) and OfficeQA (28.6 → 24.3), so GED improves training to some extent, although its main role is that of an evaluation metric. Removing the whole structured pipeline costs considerably more on SpreadsheetBench and DABench (13.4 and 14.4 points). We have revised §3.3 and Appendix A accordingly.
W2 & Q1. Broader skill coverage
We appreciate this suggestion. Following it, we added a fourth benchmark, whose skill library is larger and comes from a different domain. SeqQA2, from LABBench2 [1], consists of molecular-biology sequence tasks of 20 types, such as primer design, restriction digests and codon optimization, graded by the benchmark's official validators. The initial skills are six bioinformatics skills from the third-party bioSkills library [2] (sequence reading, primer design, primer validation, restriction-enzyme selection, restriction sites, and transcription and translation). With these skills, the strong agent (gpt-5.4) solves 71.7% of the test tasks, against 67.5% without skills. We use 20 training, 20 validation and 120 test tasks.
Table R10 characterizes the four skill configurations. They cover four domains, contain one to six skills, and range from a 1.7k-character skill to skills of over 30k characters, and they differ in whether they bundle scripts and reference files. With several skills, the agent must also choose which ones to activate.
Table R10. The evaluated skill configurations.
Table R11. Test pass rate (%) on SeqQA2.
Skill Transfer improves both weak models over the original library, by 9.2 and 21.7 points. With gpt-4.1-mini the original library brings no gain over using no skill (40.8 vs. 42.5), while the adapted library reaches 62.5%. In both runs, Skill Transfer edited five of the six skills, demonstrating Skill Transfer is effective on broader coverage of skills. These results are in Appendix I of the revised paper.