acceptodds
Withdrawn from ICLR 2027

From Loading Skills to Knowing Them: Scaffolding-Removed Fine-Tuning for Agent Skill Internalization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Loading skills at run time has become a common paradigm for strengthening an agent's problem solving: the agent pulls a full instruction document into context and follows its methodology. However, this benefit reaches only a minority of users, as most do not invoke skills in practice. We argue that generalizable skills, which encode procedural knowledge for recurring tasks such as code review and debugging, should not remain optional external add-ons but become part of the model's own capability. Internalizing this knowledge into the model makes the corresponding capabilities directly available to all users, without requiring explicit skill loading. To this end, we present SkillCore, a data pipeline and supervised fine-tuning recipe that internalizes such skills. Specifically, we transform real agent trajectories by removing the skill-loading scaffolding while preserving the task context and subsequent actions, thereby supervising direct execution without external skill content. Because this objective can also suppress appropriate calls to task-specific skills that provide genuinely external information, the recipe additionally mixes in a small proportion of skill-native trajectories to keep that calling behavior intact. On a held-out skill-free evaluation, SkillCore successfully internalizes generalizable skills in both Qwen3-8B and Qwen2.5-72B, improving performance over base models by 54.4% and 24.8%, respectively. Besides, it preserves skill invocation behavior on a separate task-specific-skill probe, without degrading other aspects of the model's performance, as measured by SWE-bench Verified.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, agent skills, skill internalization, supervised fine-tuning, post-training, tool use

Related papers

Discussion (0)

Sign in to comment.