SkillCertainty: Uncertainty Quantification for Self-evolving Agent Skills

Large Language Model (LLM) agents increasingly rely on reusable skills to solve complex tasks. However, existing skill-evolution methods typically use task-level rewards or uncalibrated self-reflection, providing limited guidance on whether a skill should be revised and which execution steps caused failure. We introduce SkillCertainty, an uncertainty-aware framework that uses step-level conformal prediction to identify behavior deviations during skill evolution. The framework constructs behavioral references from successful trajectories and applies position-stratified conformal calibration to accommodate heterogeneous, variable-length executions. Action-importance weights are learned to emphasize behaviorally informative actions and are fixed during online skill evolution. A trajectory-level score combines task performance with behavioral adherence: conforming executions support direct skill distillation, whereas nonconforming executions trigger targeted revision. Step-level diagnostics then localize deviations from successful behavior and condition the skill-generation prompt, grounding each revision in explicit behavioral evidence. When the conforming branch is reached, the distilled skill is evaluated in a separate fresh validation run. Experiments across diverse tasks and LLM backbones show competitive task performance, stronger behavioral adherence, and improved evolution efficiency over existing skill-evolution methods. These results establish conformal uncertainty as an actionable signal for self-evolving agent skills rather than merely a confidence estimate.