acceptodds
Under review as a conference paper at ICLR 2027

SkillCraft: Can LLM Agents Learn to Use Tools Skillfully?

Abstract

Real-world tool-using agents operate over long-horizon workflows with recurring structure and diverse demands, where effective behavior requires not only invoking atomic tools but also abstracting and reusing higher-level tool compositions. However, existing benchmarks mainly measure instance-level success under static tool sets, offering limited insight into agents’ ability to acquire such reusable skills. We address this gap by introducing SkillCraft, a benchmark explicitly designed to stress-test agents' ability to form and reuse higher-level tool compositions, which we call Skills. SkillCraft features realistic, highly compositional tool-use scenarios with difficulty scaled along both quantitative and structural dimensions, designed to elicit skill abstraction and cross-task reuse. We further introduce a lightweight evaluation protocol that lets agents auto-compose atomic tools into executable Skills, cache and reuse them within and across tasks, thereby turning compositional skill acquisition into a directly observable, measurable behavior. Evaluating state-of-the-art agents on SkillCraft, we observe a clear capability gradient: stronger models exploit the same composition affordance to reduce token usage by up to 80%, while weaker ones fail to do so. Moreover, success rate positively correlates with skill execution ability, establishing compositional skill acquisition as a core, separately measurable capability of tool-using agents.

Primary area:
datasets and benchmarks
Keywords:
LLM, agents, tool use, tool composition, skill acquisition, skill reuse, agent evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_tLKBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SkillCraft, a benchmark of 126 tasks across 21 task families. It studies whether agents can create, reuse, and combine executable skills. Experiments with eight models show that skills often improve success and reduce token use, while deeper skill hierarchys can be less reliable.

Critical strengths
  1. The benchmark makes skill creation and reuse directly observable. Its controlled task difficulty helps study when reusable code benefits agents.
  2. The experiments cover multiple models, skill hierarchies, and transfer settings. They also show that successful skill execution does not always lead to task success.
Critical weaknesses
  1. Transfer mainly occurs within existing task families, and Hard-to-Hard transfer repeats the same tasks. This provides limited evidence of generalization to new workflows.
  2. The DirectExec baseline is useful but covers only 48 tasks. Token comparisons use jointly solved subsets, making overall efficiency and the cost of creating skills harder to assess.
Questions for authors
  1. How well do skills transfer to held-out workflow structures rather than repeated tasks or task families?
  2. Can you extend DirectExec to the full benchmark and report total costs across all tasks, including skill creation?
1 comment

We thank the reviewer tLKB for the constructive suggestions on transfer and baselines. We are glad that the reviewer finds that the benchmark makes skill creation and reuse directly observable, that the controlled difficulty helps study when reusable code benefits agents, and that the experiments cover multiple models, hierarchies and transfer settings. Here we answer all the questions and hope they can address the concerns.

Q1: How well do skills transfer to held-out workflow structures?

Thank you for your suggestion! We add a leave-families-out evaluation using skills from 16 families and 30 tasks from five held-out families with disjoint APIs. We compare Claude-4.5-Sonnet with an empty library against a foreign skill library. Both conditions allow new skill creation and share the same tools and base instructions; the foreign-library condition additionally exposes skill descriptions, input/output schemas, and invocation examples. The results are presented below:

Initial libraryTask successAvg. tokensTasks creating new skillsTasks executing foreign skills
Empty30/30865K10/300/30
Foreign families30/30433K24/300/30

Exposure to skills from other families helps the agent construct skills for held-out workflows: skill creation increases from 10/30 to 24/30 tasks, with 50% fewer tokens and unchanged task success. This benefit occurs through the provided skill information, without directly executing foreign skills. We follow your advice and add in the paper in line 1201-1212.

Q2: Can you extend DirectExec to the full benchmark and report total costs across all tasks, including skill creation?

Thank you for your suggestion! We extend the baseline evaluation and report full-trajectory costs below.

Expanded evaluation. We rerun all three modes on 30 tasks covering all 21 families: each family's hard task plus nine tasks with the largest prior disagreement across models and modes. The rebuttal budget limits coverage to this subset rather than all 126 tasks.

ModelModeTask successAvg. tokensTotal cost ($)
Claude-4.5-SonnetNormal28/301.09M27.76
Claude-4.5-SonnetDirect Exec30/30241K7.31
Claude-4.5-SonnetSkill29/30548K12.54
GPT-5.2Normal19/271.47M21.21
GPT-5.2Direct Exec25/27263K5.07
GPT-5.2Skill25/27951K10.55

GPT uses the same 27 tasks across modes after excluding three API-interrupted tasks. Costs include successful and unsuccessful trajectories. These results are consistent with Appendix D.1: constructing parameterized, reusable skills introduces overhead in single-task execution. The value of the resulting procedures also depends on whether they remain useful beyond the task in which they were created. Our cross-difficulty and cross-model analyses test this reuse without modifying the skills, while the hierarchy analysis examines the reliability of composing them into larger workflows. Together, these analyses characterize both the cost of skill construction and the transferability and composition reliability of the acquired skills. We follow your advice and add in the paper in line 877-883.

Full-benchmark costs. We aggregate all available trajectories, including failed tasks, skill creation, repairs, and fallback:

ModelTasks (Normal / Skill)Total tokens (Normal / Skill)Token reductionTotal cost $ (Normal / Skill)
Claude-4.5-Sonnet126 / 126170.0M / 51.0M70%133.08 / 35.04
DeepSeek-V3.2126 / 126224.1M / 71.4M68%41.07 / 39.67
GPT-5.2-Codex126 / 12652.1M / 17.1M67%68.67 / 25.85
GLM-4.7126 / 125144.8M / 68.0M53%40.30 / 19.66
Gemini-3-Pro126 / 12484.5M / 43.4M49%76.10 / 50.40
DeepSeek-R1126 / 12695.5M / 52.3M45%32.05 / 24.32
Kimi-K2-Thinking126 / 12629.5M / 17.1M42%11.96 / 7.24

GLM and Gemini lack one and two Skill records, respectively. Skill Mode reduces total tokens by 42–70% relative to Normal, including the above overheads. We follow your advice and add in the paper in line 1217-1221.

Reviewer_hjmDOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces SkillCraft, a benchmark with 21 task families and 126 instances built to contain repeated entities and repeated tool-call patterns. Difficulty is varied along the number of entities and the length of the per-entity tool chain. The proposed Skill Mode exposes four operations for saving, retrieving, listing, and executing parameterized Python skills, together with a lightweight coding verifier. The authors evaluate eight models and report task success, skill execution, reuse, token use, cost, turns, and tool calls. They also study hierarchy depth, cross-difficulty reuse, and cross-model reuse. The paper claims that skill mode reduces cost and improves success, that stronger models exploit skills more effectively, that deep automatically generated hierarchies are error-prone, and that well-formed skills can transfer across models.

Critical strengths
  1. The benchmark directly operationalizes the target capability by constructing repeated entities and tool chains. This is consequential because instance-only success benchmarks do not reveal whether an agent can form or reuse higher-level procedures. Measuring execution, reuse, tokens, cost, and turns supports a more informative evaluation than success alone.

  2. The hierarchy analysis identifies a concrete failure mechanism. The combination of null values, latent bugs, dependency accumulation, and debugging overhead explains why deeper generated skills can reduce end-to-end success despite high per-skill execution rates. This negative result is useful for designing future skill libraries.

Critical weaknesses
  1. The main result is confounded by what Skill Mode adds and what the prompt reveals. Skill Mode changes persistent storage, parameterized Python, batching, verification, and prompting at the same time. The Direct Exec control is especially damaging to the claimed mechanism: on the 48-task subset, Claude reaches 0.16M tokens under Direct Exec versus 0.34M under Skill, and GPT-5.2 reaches 0.06M versus 0.26M. In addition, the system prompt tells the model to create skills early and identifies N-at-least-three as the useful condition, while every task satisfies that condition. The experiment therefore does not isolate persistent reusable skills from code-based batching and does not test autonomous discovery of when abstraction is beneficial. This directly undermines the paper's central causal claim.

  2. The transfer and reliability evidence is too narrow for the claims. The 126 instances derive from 21 families that share APIs, output schemas, and templates. The reported cross-task transfer varies entity counts within the same family rather than testing new structures. Cross-model reuse uses only eight hard tasks, and the paper reports no repeated-run variance or paired uncertainty for differences that often involve one to five tasks. Efficiency is also conditioned on tasks where both modes succeed, omitting failed skill creation, repairs, and fallback. As a result, the evidence does not establish broad transfer or the strongest claims about model-independent skill quality.

Questions for authors
  1. Can you report leave-one-family-out transfer involving new APIs, schemas, task structures, and error modes rather than new entity counts within the same family?

  2. What are the end-to-end token and cost figures over all tasks, including failed tasks, skill-creation failures, repeated repairs, verifier rejection, and fallback to atomic tools?

  3. Can you provide repeated runs and task-level paired uncertainty for the success comparisons, and explain how eight hard tasks support conclusions about universal cross-model success?

  4. What sandbox, timeout, filesystem, network, and resource controls apply to generated Python, and are external API responses snapshotted for reproducibility?

4 comments

Thanks for your detailed feedback! We appreciate the recognition of SkillCraft's evaluation capability and the hierarchy analysis's insights into skill-library failures. We address the questions below.

W1: Is the main result confounded by what Skill Mode adds, and does Direct Exec undermine the central claim?

SkillCraft makes skill creation, reuse, and composition measurable. Skill Mode provides the evaluation protocol (Abstract and Section 3).

Direct Exec Mode isolates code execution from skill management (Appendix D.1). It removes skill storage and parameterized invocation but retains code writing, revision, execution, loops, batching, and filtering. On 48 tasks, Direct Exec uses fewer tokens, while Skill Mode achieves 97–99% execution success versus 68%, revealing an efficiency–reliability trade-off.

We isolate the prompt hint by changing only the guidance. Without "Create pattern early" and "When to use", GPT-5.2 creates skills in 26/27 tasks (27/27 with the standard prompt), while Claude creates skills in 14/30 (30/30). Tools and other instructions remain unchanged. The hint affects Claude substantially less than GPT-5.2's skill creation behavior. We add these results in lines 1178–1187.

W2: Is the transfer and reliability evidence too narrow for the claims?

Our transfer experiments evaluate agent-produced skills without modification. Section 5.2 tests Easy→Hard and Hard→Easy across all 21 families, achieving 97–100% execution success across Claude and Gemini.

We expand cross-model evaluation from eight tasks to all 21 families' hard tasks. Skills transfer without modification:

Creator↓/Executor→Claude-4.5-SonnetGemini-3-ProGLM-4.7MiniMax-M2.1
Claude-4.5-Sonnet1.000.810.520.90
Gemini-3-Pro0.950.860.330.90
GLM-4.70.710.900.480.81
GPT-5.20.950.900.520.95
MiniMax-M2.11.000.900.430.86

With Claude as executor, success ranges from 0.71 to 1.00 across creators, compared with 0.33–0.52 with GLM as executor. We add these results in lines 1230–1237.

Q1: Can you report leave-one-family-out transfer with new APIs, schemas, task structures and error modes?

We add leave-families-out evaluation using five held-out families whose APIs are absent from the other 16. Claude-4.5-Sonnet evaluates 30 tasks with either an empty or foreign-family skill library. Both conditions allow new skill creation and share tools and instructions; the foreign library also exposes skill descriptions, schemas, and invocation examples.

Initial libraryTask successAvg. tokensTasks creating new skillsTasks executing foreign skills
Empty30/30865K10/300/30
Foreign families30/30433K24/300/30

Foreign skill exposure increases new skill creation from 10/30 to 24/30, with 50% fewer tokens and unchanged success. Although foreign skills are not executed, their interface examples aid skill construction on unseen APIs. We add these results in lines 1201–1212.

Q2: What are the end-to-end token and cost figures over all tasks, including failures?

We recompute efficiency over all 126 tasks without conditioning on success, counting failed skill creation, repairs, and fallback calls. Skill Mode reduces tokens by 67–70% in two representative models:

Model#Tasks(Normal/Skill)Tokens(Normal)Tokens(Skill)ReductionCost$(Normal/Skill)
Claude-4.5-Sonnet126/126170.0M51.0M−70%133.08/35.04
GPT-5.2-Codex126/12652.1M17.1M−67%68.67/25.85

Full results for all seven models are in the revised paper (lines 1221–1225).

Q3: Can you report repeated runs and paired uncertainty?

Paired comparisons. We use McNemar's exact test and paired bootstrap (5,000 resamples) for token-reduction 95% confidence intervals.

Model#TasksSuccess(Normal)Success(Skill)Solved only by Skill/only by NormalMcNemar pToken reduction[95% CI]
GPT-5.2-Codex1260.3170.3738/10.03971.4%[56.6%,82.4%]
GLM-4.71250.7440.90421/1<0.00155.8%[35.6%,70.6%]

Full paired results are in the revised paper (lines 1222–1227).

Repeated runs. We independently rerun Claude-4.5-Sonnet on the same 30 tasks:

MetricOriginal runRerunMean±stdTask outcome agreement
Normal success1.000.930.97±0.0328/30
Skill success1.000.970.98±0.0229/30
Token reduction(Skill vs Normal)76.5%49.8%63.2%±13.4%–

Cross-model results are reported in W2, covering all 21 families.

Q4: What sandboxing, resource limits, and API controls are applied?

Generated Python runs without OS-level sandboxing or separate CPU/memory quotas. Filesystem and network access are unrestricted for generated code, and external API responses are live, not snapshotted. We provide the full execution, timeout, and resource-limit details in lines 1493–1502.

Thank you for the response, but my main concern remains, Direct Exec still uses fewer tokens than Skill Mode, so the cost savings is unlcear. I will keep my rating.

Thanks for your reply! We appreciate your concern. First, compared with Direct Exec, Skill Mode achieves higher skill-execution success in Table 6: 97–99% versus 68%, while retaining parameterized skills for subsequent reuse. We agree that this comes with higher token consumption. However, our main motivation is to evaluate whether agents can autonomously create, reuse, and transfer tool skills. The advantage we observe is therefore at the skill-execution level, rather than lower single-pass cost. Second, we hope our transfer experiments help clarify the benefits of learned skills. Without modification, skills transferred across difficulty levels achieve 97–100% execution success and reduce target-stage monetary cost by 24–46% relative to the no-transfer baseline (Table 4). On unseen API families, exposure to foreign skill interfaces also reduces tokens from 865K to 433K, with 30/30 success in both conditions, even though no foreign skill is executed (Table 9). We believe these results provide complementary evidence that learned skill information can improve subsequent problem solving and reduce token consumption in subsequent tasks.

Thank you for the follow-up! We respectfully disagree that the existence of an efficient Direct Exec baseline undermines the purpose of Skill Mode, for the following reasons.

First, SkillCraft is a benchmark for skill acquisition, reuse, and transfer, and Skill Mode is its evaluation protocol rather than a proposed method. Our goal is not to show that Skill Mode is the cheapest way to solve a single task, but to measure whether agents can create, reuse, and transfer skills, together with the accuracy and cost trade-offs involved. Direct Exec's per-task efficiency is therefore an informative benchmark result rather than evidence against our claims: a one-off script can batch API calls without the overhead of designing a reusable interface.

Second, skills generalize beyond the task in which they are created, which a one-off Direct Exec script cannot. As shown in our earlier responses, skills transfer across difficulty levels with 97–100% execution success, across models with 0.71–1.00 success for strong executors, and across task families, where exposure to a skill library raises skill creation on unseen APIs from 10/30 to 24/30 tasks with 50% fewer tokens.

In addition, even in terms of cost, the advantage of skills grows as the same type of task recurs, which is common in real-world workflows (e.g., periodic reports, monitoring, or processing new records with a fixed pipeline). We conduct an additional experiment with GPT-6 Luna on three task families: after acquiring a skill, the agent solves new inputs from the same family, either writing a new script (Direct Exec) or invoking the saved skill (Skill Mode). All tasks succeed in both modes:

Task familyPairsTokens (Direct / Skill)Cost $ (Direct / Skill)
TV series9508,148 / 337,6960.0225 / 0.0108
Vocabulary6197,129 / 162,3560.0073 / 0.0048
Universities5251,329 / 183,9760.0107 / 0.0050
Total20956,606 / 684,0280.0405 / 0.0206

Each recurrence with Skill Mode costs 28.5% fewer tokens and 49.0% less money at identical task success. The one-time acquisition cost is thus amortized over recurrences: after only three, Skill Mode is already cheaper overall in four of five streams, and the gap widens with every further recurrence.

Together, we view Skill Mode as the instrument through which SkillCraft studies how agents acquire, reuse, and transfer tool skills. Our core goal is to measure these capabilities and their trade-offs across models, rather than to propose Skill Mode as the most efficient way to solve an individual task.

Reviewer_vfbzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces SkillCraft, a benchmark for 126 tool-use tasks that evaluates whether LLM agents can compose atomic tools into reusable executable skills. It contains 21 task families, with difficulty increased through more entities and tool calls. The paper introduces “Skill Mode,” an MCP-style protocol through which agents can save Python-based workflows (scripts that chain atomic tools) and reuse them on new inputs reducing repeated reasoning and intermediate-output processing. Experiments across eight models show substantial token savings and generally improved task success. Additional experiments investigate hierarchical composition finding that deeper hierarchical composition hurts success, and skill potentially transfers across difficulty levels and models.

Critical strengths
  1. Important problem: Acquiring and reusing skills is key for agents in repetitive, long-horizon workflows, but existing benchmarks mainly measure task success with a fixed toolset. SkillCraft’s repeated-structure tasks and four-primitive protocol make skill creation and reuse directly observable, yielding concrete metrics (Exec rate, reuse factor) and enabling comparisons of skill quality across models.

  2. Strong empirical gains and insightful analysis: Skill Mode cuts token use by up to 79% across 8 models while maintaining or improving success. The analysis experiments are informative too: deeper hierarchies lower success because errors propagate up the hierarchy, and the cross-model study shows that skill quality depends on which model created it.

Critical weaknesses
  1. The experiments do not isolate the benefit of reusable skills. Skill Mode also introduces executable code, loops, intermediate-result filtering. These mechanisms could explain much of the improvement; the appendix’s competitive Direct Exec results reinforce the need for a matched code-execution baseline.

  2. Generalization evidence is limited. Transfer primarily occurs within task families, Hard->Hard reuses the same tasks, and cross-model evaluation covers only eight tasks.

Questions for authors

Please see weaknesses.

3 comments

We thank the reviewer for the constructive suggestions on baselines and generalization. We are glad that the reviewer finds the problem important, the benchmark effective at making skill creation and reuse observable, and the empirical gains and analysis insightful. We address all questions below.

Q1: Do the experiments isolate the benefit of reusable skills from code execution, loops and filtering?

We would like to clarify the matched code-execution baseline, following Appendix D.1.

Skill Mode is an evaluation protocol, not a proposed method. As stated in the Abstract and Section 3, SkillCraft provides recurring task structures and an interface for constructing, saving, and invoking skills, making artifacts and reuse directly observable. Our contribution is the benchmark and capability evaluation framework.

Direct Exec Mode ablates the explicit skill-construction and reuse workflow. Agents can write, revise, and execute task-specific Python scripts through exec_script, with input values embedded in the code. Loops, batching, tool chaining, and intermediate-result filtering remain available; operations for saving, retrieving, and invoking named, parameterized skills are removed. Thus, the agent can solve tasks through executable code without constructing and managing a reusable skill library.

Results on 48 tasks show that Direct Exec provides a strong code-execution baseline, while Skill Mode achieves higher execution success for both models:

ModelModeTask successExecution success (Exec)Avg. tokens
Claude 4.5 SonnetSkill43/4899%0.34M
Claude 4.5 SonnetDirect Exec46/4868%0.16M
GPT-5.2Skill43/4897%0.26M
GPT-5.2Direct Exec41/4868%0.06M

Task success covers all 48 tasks; token averages use tasks jointly solved by Base and the respective mode. Skill Mode achieves 97–99% execution success versus 68% for Direct Exec. Task success favors Skill for GPT-5.2 and Direct Exec for Claude. Direct Exec uses fewer tokens, reflecting the absence of parameter abstraction and the save–execute workflow. This comparison captures the efficiency–reliability trade-off.

We also expanded the experiments on 30 tasks covering all 21 families: each family's hard task plus nine tasks with the largest prior disagreement across models and modes. Due to the rebuttal budget, we evaluate this subset rather than all 126 tasks.

ModelModeTask successAvg. tokensTotal cost ($)
Claude-4.5-SonnetDirect Exec30/30241K7.31
Claude-4.5-SonnetSkill29/30548K12.54
GPT-5.2Direct Exec25/27263K5.07
GPT-5.2Skill25/27951K10.55

GPT uses the same 27 tasks across modes, excluding three API-interrupted tasks. Costs include successful and unsuccessful trajectories. These results are consistent with above conclusion. Constructing parameterized, reusable skills introduces overhead in single-task execution. We follow your advice and revised the paper in line 877-883.

Q2: Is the generalization evidence limited?

Thank you for your constructive advice! Our generalization experiments are controlled analyses of agent-produced skills, testing whether saved compositions remain usable when task difficulty or the executor model changes.

  1. Cross-difficulty transfer across all 21 task families.In Section 5.2, source-task skills are transferred without further modification. Across Claude and Gemini, Easy→Hard and Hard→Easy achieve 97–100% skill execution success and 19–41% token reductions relative to Base. Gemini's task success improves from 16/21 to 19/21 for Easy→Hard and from 55/63 to 60/63 for Hard→Easy. Shared APIs provide a controlled setting for testing reusable structure across changes in task scale and complexity. These two directions provide our cross-difficulty transfer evidence; Hard→Hard separately measures re-execution on the same tasks.

  2. Expanded cross-model evaluation. We extend the original eight-task analysis to the hard (h1) task of all 21 families. Skills from each creator (row) are used by each executor (column) without modification. The results, presented below, show task success rates:

Creator ↓ / Executor →Claude-4.5-SonnetGemini-3-ProGLM-4.7MiniMax-M2.1
Claude-4.5-Sonnet1.000.810.520.90
Gemini-3-Pro0.950.860.330.90
GLM-4.70.710.900.480.81
GPT-5.20.950.900.520.95
MiniMax-M2.11.000.900.430.86

Results show that transfer depends on both creator and executor. Claude, Gemini, and MiniMax achieve 0.71–1.00 success across creators, while GLM achieves 0.33–0.52. With Claude as executor, success ranges from 0.71 with GLM-created skills to 1.00 with Claude- or MiniMax-created skills. This demonstrates the benchmark's diagnostic value in revealing successful transfer and capability limitations. We follow your advice and add in the paper in line 1230-1237.

Thanks for the rebuttal and the additional experiments. As is evident from the experiments that on task success Skill does not outperforms Direct Exec. and is also costly on tokens mostly. So it is not clear to me the advantage of reusable skills with respect to this Direct Exec. - it conflates whether the benefit is coming from code exec. or actual reusable skills. I appreciate the authors positioning that Skill as an evaluation protocol rather than proposed method, I think this requires a bit of repositioning of later parts of the paper (especially experimental section) to make it fully aligned and properly scoped. My concerns regarding generalization evidence are mostly addressed. Therefore, I will raise my score to 3.

Thank you for following up! We are glad that our responses have addressed your concerns about generalization, and we appreciate your recognition of Skill Mode as an evaluation protocol. Following your suggestion, we revised the paper to align the experimental discussion with this positioning:

  1. Section 4 (lines 298–303, 313–320, 350–351). We attribute single-task savings over atomic tool calls to executable composition and present Table 2 as a measurement of how well each model creates and executes skills. This view highlights what SkillCraft reveals beyond task success: skill execution rate correlates with task success (r = 0.65), and the value of reuse appears beyond the creating task, where transferred skills reach 97–100% execution success and a skill library raises skill creation on unseen APIs from 10/30 to 24/30 tasks with 50% fewer tokens (§5.3, Appendix D.4).

  2. Appendix D.1 (lines 855–863). We clarify that Skill Mode adds overhead on a single task because agents define and save a parameter interface before invoking it, while achieving higher execution success (97–99% vs. 68%), which is measured separately from end-to-end task success. We also add the expanded three-mode comparison on 30 tasks covering all 21 families, which confirms the Table 6 pattern at larger scale: both code-based modes substantially outperform atomic tool calls (e.g., GPT-5.2 success rises from 19/27 to 25/27, and Claude’s token use drops by about half or more).

We believe these revisions make the paper fully aligned with the role of Skill Mode as an evaluation protocol for studying how agents construct and reuse skills. Thank you again for your constructive feedback!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.