Skill-Forest: Randomized Ensembles of Textual Skill Optimizers

Agent skills adapt frozen large language model agents by encoding reusable procedural knowledge outside model parameters. Under limited data, noisy textual updates and repeated selection on small validation sets can make single-path optimization sensitive to early decisions and evaluation noise. We introduce Skill-Forest, an optimizer-level ensemble inspired by the randomize-and-aggregate principle of Random Forests. Each tree uses an independently sampled training–validation split and independently searches for its tree-local best Skill. Gain-prioritized expansion under a shared budget, validation-guided pruning, and early rejection enable efficient forest construction. The resulting tree-local best Skills are then aggregated into one deployable Skill through direct selection (Select) or hierarchical merge (Merge). This design diversifies both optimization paths and validation views to reduce the high variance associated with textual-gradient optimization. We evaluate Skill-Forest across five benchmarks spanning direct-chat tasks, tool use, and embodied tasks. Skill-Forest Select and Merge improve the mean held-out Hard score over No-skill by 12.82 and 12.87 percentage points (pp), respectively. Under the same reflection strategy, they outperform SkillOpt by 2.59 and 2.63 pp, respectively. Skill-Forest reduces training and validation rollouts during forest construction by 73.5% across the five benchmarks relative to complete tree construction, supporting efficient multi-path ensemble Skill exploration.