Beyond Initial Skills: Contrast-Guided Skill Evolution for Robust Learning in LLM Agents

Large language model (LLM) agents can learn continually by accumulating and updating external procedural skills while keeping their model parameters frozen. Existing approaches typically evolve these skills by revising prior skills based on interaction experiences collected from new tasks. However, we find that their learning gains vary substantially with initial skills, even under the same sequence of subsequent training tasks, resulting in unstable or even negative skill evolution. Further analysis shows that initial skills influence both the experiences collected through agent interactions and the subsequent skill updates, causing their effects to persist throughout the evolution process. To enable robust skill evolution beyond initial skills, we propose Skill-CoE, a Contrast-guided skill Evolution framework that explores alternative skill update directions and leverages their complementary strengths for effective updates. Specifically, given the same interaction experience, Skill-CoE constructs two evolution candidates by revising the current skill and reconstructing a new skill, respectively. It then compares their executions on the same tasks using a pairwise preference objective, with the current skill serving as a fixed reference to identify complementary strengths. Guided by these comparisons, Skill-CoE recombines the candidates to incorporate useful new procedures while preserving effective existing experience. Experiments on ALFWorld, AppWorld, and ScienceWorld show that Skill-CoE consistently achieves stable learning gains across all evaluated initial skills and outperforms existing methods in final test performance. Code is available at https://anonymous.4open.science/r/Skill-CoE-2737.