acceptodds
Under review as a conference paper at ICLR 2027

SkillCertainty: Uncertainty Quantification for Self-evolving Agent Skills

Abstract

Large Language Model (LLM) agents increasingly rely on reusable skills to solve complex tasks. However, existing skill-evolution methods typically use task-level rewards or uncalibrated self-reflection, providing limited guidance on whether a skill should be revised and which execution steps caused failure. We introduce SkillCertainty, an uncertainty-aware framework that uses step-level conformal prediction to identify behavior deviations during skill evolution. The framework constructs behavioral references from successful trajectories and applies position-stratified conformal calibration to accommodate heterogeneous, variable-length executions. Action-importance weights are learned to emphasize behaviorally informative actions and are fixed during online skill evolution. A trajectory-level score combines task performance with behavioral adherence: conforming executions support direct skill distillation, whereas nonconforming executions trigger targeted revision. Step-level diagnostics then localize deviations from successful behavior and condition the skill-generation prompt, grounding each revision in explicit behavioral evidence. When the conforming branch is reached, the distilled skill is evaluated in a separate fresh validation run. Experiments across diverse tasks and LLM backbones show competitive task performance, stronger behavioral adherence, and improved evolution efficiency over existing skill-evolution methods. These results establish conformal uncertainty as an actionable signal for self-evolving agent skills rather than merely a confidence estimate.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Conformal Prediction, Uncertainty Quantification, Large Language Model Agents, Skill Generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vQ1COpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SkillCertainty, a framework that combines trajectory-level routing with step-level conformal diagnostics to guide revision of an LLM agent's skill document. In ATUQ, ten executions with the existing skill and ten without are collected per task. Successful with-skill executions are used to build a position-stratified StepLibrary of per-bucket conformal thresholds and a Skill Reference Vector, and failed without-skill executions serve as contrastive negatives for learning action-importance weights. In CSASO, each new trajectory receives a composite score of task outcome and adherence to the Skill Reference Vector, and a fixed threshold on this score routes it. Conforming trajectories are distilled into a skill that is validated in one fresh run. Nonconforming trajectories go through Step-CP diagnostics, which flag steps that exceed their bucket threshold and insert them into the skill-generation prompt. On SkillsBench, across six LLM backends and four skill-evolution baselines, the method reports competitive final scores, the lowest adherence nonconformity, and fewer evolution iterations than EvoSkill.

Critical strengths
  1. Step-level diagnostics give the skill generator targeted feedback. Step-CP diagnostics highlight specific failed or unusual steps and their outputs alongside the task outcome and raw trajectory. Table 3 shows that adding these diagnostics raises mean ΔGS from +0.206 to +0.384 on three tasks compared with using raw trajectories alone. The per-task examples in the appendix illustrate how flagged steps inform specific skill revisions.

  2. Fresh validation of distilled skills. Table 4 shows that the score of the trajectory triggering distillation can overstate the score obtained when the resulting skill is executed: mean ΔGS falls from 0.293 at detection to 0.094 on a fresh run, and Energy Market Pricing falls from 1.000 to 0.000. This is a useful finding for skill-evolution methods that judge a skill by the trajectory it was distilled from.

Critical weaknesses
  1. The benefit of conformal calibration is unclear. Table 3 shows that adding Step-CP diagnostics improves results compared with raw trajectories alone. However, it does not compare them with simpler feedback, such as listing failed steps and their outputs or highlighting rare actions without calibrated thresholds. Algorithm 1 already prioritizes failed-status steps, and Appendix B acknowledges that the calibration does not establish predictive coverage for arbitrary future steps. The experiments therefore show a benefit from the diagnostic feedback, but do not establish whether conformal calibration is needed for that benefit.

  2. The adherence and efficiency comparisons have limitations. SkillCertainty uses s_adh both to evaluate adherence and to decide when to stop revising the skill. The baselines do not explicitly target this score, so lower s_adh alone does not demonstrate better task performance or transfer. The efficiency comparison reports evolution iterations but excludes the 20 ATUQ runs per task and the fresh validation run. The main-comparison update budgets are unspecified, and it is unclear whether the baselines receive the same ATUQ runs. The method uses fewer updates than EvoSkill, but the paper does not show whether it uses fewer task executions or tokens overall.

  3. Skills are not tested on new tasks. The paper motivates skills as reusable, but evaluates them on the same tasks used during skill evolution. No new tasks or task variants are tested, so the results do not show whether the evolved skills can be reused beyond those tasks.

Questions for authors
  1. How does Step-CP compare with simply highlighting failed steps and their outputs, using the same skill generator and budget?
  2. Why do Tables 6 (δ = 0.25) and 7 (α = 0.10) imply different starting scores for GPT-5.1? Why do four EvoSkill composite scores in Table 1 differ from Eq. 11 applied to the reported GSf and s_adh?
Reviewer_da4tOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes SkillCertainty, which casts skill evolution as calibrated behavioral inference, using calibrated deviations from successful reference behaviors to determine when a skill should be revised and to guide its evolution. It combines position-aware calibration and learned action geometry to capture local action conformity and global skill adherence in heterogeneous, variable-length trajectories. Experiments across diverse tasks and LLM agents demonstrate the gains in task performance.

Critical strengths
  • Framing the problem as "should the skill change, and which steps justify it" is better posed than "rewrite after every failure". The distinction between a task-outcome signal and a behavioral signal is a real gap in the existing skill-evolution literature.
  • The calibration design is a sensible, lightweight engineering answer to variable-length trajectories.
Critical weaknesses
  • My main concern is that the experimental scale is too small to support the paper's conclusions. According to Sec. 5.1, the multi-model comparison covers only 10 tasks. These tasks are never listed, and Tab. 1 gives neither variance nor the number of seeds. Two of the three ablations use only 3 tasks each (Tab. 2–3), and the third uses 7 (Tab. 4). None of the ablations name the backend model or report variance. Both sensitivity analyses (Tab. 6–7) use a single task.

  • Adherence and completeness are scored against references that SkillCertainty fits itself, so these metrics favor it by construction. The conformal layer holds the main formal claim, yet its coverage is never tested. Because flags are raised only below a frequency threshold and failed steps are ranked first, the method may just report the failing steps. Transfer of skills to new tasks is also not addressed.

  • Tab. 6 shows that evolution hurts whenever it runs. The default δ=0.25 works only because it triggers 0 iterations.

Questions for authors

Please see the weaknesses.

Reviewer_GBBPOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SkillCertainty, an uncertainty-aware framework for self-evolving agent skills. The authors observe that existing skill evolution methods rely on task-level rewards or uncalibrated self-reflection, which can indicate that something failed but not where the agent deviated or whether the deviation is unusual relative to successful runs. SkillCertainty formulates skill evolution as calibrated behavioral inference. An offline phase builds a behavioral reference from certified successful trajectories, consisting of position-stratified conformal thresholds, a learned action importance weighting, and a skill reference vector. An online phase scores each new trajectory with a composite nonconformity measure that combines task outcome and behavioral adherence, routes conforming trajectories to skill distillation followed by a fresh validation run, and routes nonconforming trajectories through step-level diagnostics that localize anomalous steps and inject them into the skill generation prompt. Experiments on SkillsBench across six LLM backends show competitive final task performance, consistently lower behavioral nonconformity than the compared skill-evolution baselines, and strong evolution-update efficiency, supported by ablations and per-task case studies.

Critical strengths
  1. The formulation is novel and well motivated. Casting skill revision as calibrated behavioral inference rather than terminal-reward-driven rewriting is a new angle for the agent skill literature, and turning uncertainty into an operational signal for routing and targeted revision is a substantive contribution beyond passive confidence estimation.

  2. The treatment of statistical assumptions is careful. The paper distinguishes trajectory-level exchangeability from step-level dependence, acknowledges that its calibration property is not a standard finite-sample coverage guarantee, uses leave-one-out construction to avoid self-inclusion, and introduces fresh validation runs that honestly expose cases where the distilled skill fails to reproduce the performance of the triggering trajectory.

  3. The empirical analysis is fairly comprehensive. Evaluation spans six diverse LLM backends, reports behavioral and efficiency metrics in addition to task scores, includes component-level ablations for diagnostic injection and learned action weights, sensitivity analysis over the routing threshold and calibration level, and concrete per-task case studies of the underlying mechanism.

Critical weaknesses
  1. The conformal machinery is weaker than the framing suggests. Steps within a trajectory are dependent, the nonconformity score is a frequency-based rarity heuristic augmented with a failure penalty and a hand-set threshold floor, and sparse-reference tasks fall back to in-sample scoring or neutral diagnostics. The theoretical results largely restate empirical quantile properties and deterministic consequences of the routing rule, so the principled guarantee that motivates the method is mostly absent in the form used in practice.

  2. The framework contains an internal tension that is exposed but not resolved. The validation ablation shows that the trajectory which triggers the conforming branch can be an optimistic proxy for the distilled skill, with some candidates failing completely on the fresh run, yet the method terminates immediately after this single validation run regardless of its outcome. The strong dependence of results on the routing threshold further suggests that the reported efficiency and convergence behavior hinge on a single hyperparameter whose favorable default is demonstrated only on the evaluated benchmark tasks.

Questions for authors
  1. How is exchangeability maintained at test time, given that the evolving skill changes the trajectory distribution relative to the offline calibration references?

  2. Are the adherence and composite nonconformity metrics for the baselines computed using your own reference artifacts, and if so, how do you ensure a fair comparison on those metrics?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.