SkillDyG: Dynamic Grouping for Structured Skill Retrieval

Skill-augmented agents reuse procedural knowledge from previously solved tasks to improve task execution. For complex tasks, however, ranking skills by semantic similarity alone is insufficient: the usefulness of a skill depends on both its capability and its relations to other skills. Existing methods use structured skill libraries to encode skill roles and relations for retrieval, but often construct the organization only once from skill descriptions, without sufficiently distinguishing when skills apply or how they relate. Errors in that organization can then persist, causing retrieval to confuse candidates that appear relevant to the same task but differ in applicable conditions or overlook redundancy and incompatibility among them. We propose SkillDyG, a framework that dynamically constructs skill groups through validated updates and uses the resulting organization to guide retrieval. In each update round, SkillDyG uses local metadata describing candidate skills’ capabilities and execution conditions to propose their placement relative to the current skill groups. It then jointly revises affected groups and reviews the candidate organization before committing any changes. The validated organization provides global metadata that distinguishes each skill’s role within its group and captures how that group relates to others. A dual-head retriever combines both views to distinguish similar candidates and retrieve skills that are individually relevant and collectively compatible. Across five backbone models on SkillsBench and ALFWorld, SkillDyG consistently achieves the highest task reward, with an average relative improvement of 11.3% over the strongest baseline on SkillsBench. All related resources and analysis are collected for the community at https://anonymous.4open.science/r/SkilIDyG.