Revise, Verify, Distill: Self-Evolving Skills for Language Agent Learning

Textual skills let language agents reuse past experience, but a skill that looks reasonable may still fail when executed. We study how agents can turn these failures into experience that transfers to new tasks and improves the model itself. To achieve this, our framework operates in three stages. First, Skill Evolution repairs a skill on the task that produced it: after a failed rollout it localizes where progress stopped, changes one rule using evidence from the trajectory, and tests the result in the environment; the first successful version is certified, and its failed parent is kept to record what changed. Second, Skill Transfer asks whether repaired skills also help on tasks they were never written for: certified skills are generalized into a frozen bank for held-out tasks, while later repairs stay local. Third, Skill Internalization moves the corrections into the model itself. We introduce EvoOPD: a frozen copy of the base model scores the skill-free student’s own actions under both skill versions, the certified one supplying the training target and their score difference weighting the actions affected by the repair. Together, these stages repair a skill where it failed, reuse it where it was never tested, and fold the verified edits into the parameters; we call the overall process Evolution Learning. Revision raises source-task success from 13.62% to 46.27% on ALFWorld and from 24.54% to 28.96% on WebShop. On held-out ALFWorld tasks, the frozen bank raises success from 6.43% to 37.86% (seen) and from 5.22% to 35.07% (unseen). With all skills removed at test time, EvoOPD reaches 54.52% and 64.18% over three seeds, compared with 7.86% and 4.48% for the base model. The same verified edits therefore support both skill reuse and model learning.