Self-Improving Skills from Failure: Learning to Recover, Remember, and Evolve

Agent frameworks increasingly package procedural knowledge as reusable skills that are selected from a skill library and executed on demand. Yet even a correctly selected skill may fail in execution, and such failures are typically treated as terminal outcomes rather than reused for improvement. We propose a skill-augmented agent system that learns from failures through reflective recovery, reusable case memory, and persistent skill revision, enabling the model and skill library to co-evolve through interaction. Optimizing this system is challenging because outcome-level reinforcement learning cannot distinguish credit across skill selection, execution, and reflection. We therefore introduce FIRE (Failure-Informed Reinforcement and Evolution), which performs stage-aware credit assignment for these components. Across four in-distribution task families and three out-of-distribution benchmarks, FIRE achieves strong skill-routing and task performance across Qwen3-4B and Llama-3.1-8B. With Qwen3-4B, FIRE reaches $95.1\%$ Skill Hit, $92.2\%$ Task Success, and $88.0\%$ Joint Success in-distribution, together with $35.4\%$ average OOD Task Success.