acceptodds
Under review as a conference paper at ICLR 2027

SkillCred: Online Value Estimation for Evolving Agent Skills

Abstract

Agent skills encapsulate reusable strategies and procedural knowledge to improve agent task performance. Recent work has begun to evolve such skills continually from historical interaction trajectories. However, skill evolution need not yield monotonic improvements. Newly generated versions may encode erroneous strategies, and blindly replacing an existing version with the latest one can degrade agent performance. To address this problem, we present SkillCred, an online framework that maintains competing versions within each skill cluster and estimates their values from trajectories generated during normal task execution. For each skill version invoked in an episode, a local progress assessor evaluates the agent behavior observed across all of its invocation intervals and produces one ternary local progress signal. SkillCred uses these signals to update a Dirichlet posterior over each version's local progress probabilities and applies Thompson sampling to route among competing versions while accounting for estimated value and uncertainty. It further uses posterior-based lifecycle control to promote, retain, or archive challengers according to posterior evidence and evaluation budgets. Because all value estimates are learned from ongoing task trajectories, SkillCred requires neither additional evaluation rollouts nor a held-out validation set. Experiments with three executor models on ALFWorld and WebShop show that SkillCred consistently achieves the highest mean success rate among the compared methods. Averaged across the three executors, its relative success-rate gains over the strongest baseline are 14.5% on ALFWorld and 38.2% on WebShop. The code is available at https://anonymous.4open.science/r/SkillCred.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Large Language Model Agents, Agent Skills, Online Skill Value Estimation, Skill Evolution

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_arjUOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper introduces SkillCred, an online framework for assessing and managing agent skills as they evolve. The framework updates Bayesian value estimates using local progress signals collected during task execution, selects among skill versions through Thompson sampling, and governs version retention with a predictive criterion. These components support skill management without requiring separate evaluation rollouts.

Critical strengths
  1. Reliable skill assessment is crucial for continual skill evolution, as newly generated versions can introduce errors and degrade agent performance.

  2. Learning from normal execution trajectories avoids dedicated validation rollouts and reduces the overhead of evaluating competing skill versions.

  3. Experiments include diverse models on different benchmarks exhibit consistent improvements, supported by component ablations.

Critical weaknesses
  1. Local progress labels are crucial to value estimation, but their accuracy and attribution reliability are not directly validated. The LLM-only and synthetic-noise experiments provide useful evidence, yet fixed-checkpoint replays do not capture downstream effects, and symmetric label corruption does not address systematic errors such as misattributing upstream failures to downstream skills.

  2. The paper characterizes skill versions as arms and local progress to rewards, but does not clearly clarify what is the difference between the bandit formulation and simple version-selection strategies. Comparisons with greedy selection or uniform evaluation would help demonstrate the value of explicitly balancing exploration and exploitation.

  3. In this work, skill-version routing is formulated as a multi-armed bandit problem, but the choice of Thompson sampling is not supported by comparisons against alternatives such as UCB. The experiments compare retention criteria rather than routing algorithms.

Questions for authors

Why do the experiments use Qwen3.5-9B and Qwen3.6-35B-A3B rather than models from the same version? Could differences in model generation and architecture confound the interpretation of results across model scales?

2 comments

Thanks for the rebuttal. However, the following questions remain unsolved.

  1. The methodological novelty remains unclear. CRV appears to estimate retention value by sampling possible futures and averaging their returns, a standard Monte Carlo approach. What is new beyond applying this established multi-arm bandit technique to skill lifecycle management?

  2. The model selection rationale remains incomplete. Qwen3.5 already includes a 35B-A3B MoE model, so covering both a 9B dense configuration and a 35B-A3B MoE configuration does not sufficiently explain the cross-generation pairing of Qwen3.5-9B and Qwen3.6-35B-A3B. This raise serious concerns on the soundness of the proposed method.

  1. The clarification does not resolve my concern, as none of the four listed components appears novel. Local progress assessment offers limited performance improvement over directly using the DeepSeek Harness. Bayesian value estimation and posterior-sampling routing are simply components of a standard multi-armed bandit algorithm, and comparisons with alternatives such as UCB are still missing. Predictive lifecycle control likewise appears to apply existing bandit evaluation and elimination ideas. Combining these components into a framework does not sufficiently establish methodological novelty.

  2. I know your comparison is fair when using the same model, but my question is: “Could differences in model generation and architecture confound the interpretation of results across model scales?” It is unusual to use two models with different versions for experimental evaluation. So my main concern is that the proposed method is not applicable to other model sizes in Qwen 3.5 or 3.6, which raises severe concerns on the generalization of this method, and adding the experimental results of other sizes will address this.

Reviewer_q9fDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SkillCred, an online framework for estimating the value of evolving skill versions from trajectories generated during normal task execution, without additional validation rollouts or a held-out validation set, which comprises three components: 1) Fine-Grained Online Skill Value Estimation: a local progress assessor assigns a ternary signal to each invoked skill version based on its invocation intervals, and these signals update a Dirichlet posterior over each version's value; 2) Posterior-Sampling Skill Routing: competing versions within a skill cluster, together with a no-skill arm, are treated as bandit arms and selected via Thompson sampling; and 3) Predictive Skill Lifecycle Control: challengers are promoted by posterior comparison with the incumbent, and early archived according to a Competitive Retention Value (CRV) estimated by posterior-predictive simulations. Experiments on ALFWorld and WebShop with three executor models show that SkillCred outperforms baselines such as Trace2Skill and SkillOpt.

Critical strengths
  1. The paper identifies a practical and underexplored problem: skill evolution is not monotonic, so selecting among evolved versions matters, and the motivation is well supported by empirical evidence.
  2. Estimating skill value from ongoing task trajectories avoids dedicated validation rollouts, resulting in substantially lower evaluation overhead than validation-based methods such as SkillOpt.
Critical weaknesses
  1. The claim of label-free and validation-free estimation is overstated. The paper motivates SkillCred by the difficulty of obtaining validation tasks or outcome labels in open-ended environments, yet the main experiments rely on benchmark-specific assessors: the WebShop assessor is a script that parses product attributes and checks exact requirement matching, which is essentially a hand-crafted dense reward, and the ALFWorld LLM judge receives the terminal task outcome in its prompt. Moreover, the evolution anchors require agreement between the terminal outcome and the local signal. Designing such assessors for every new environment is itself a form of supervision whose cost is not acknowledged.

  2. The gains mainly come from the assessor rather than the proposed bandit framework. When local progress is replaced by terminal outcomes (w/o LPA), ALFWorld success drops to 56.7, below SkillOpt (57.9), and with an LLM-only assessor the WebShop success rate is 19.5 versus 18.4 for SkillOpt, within one standard deviation. This suggests that the improvement is largely driven by the hand-designed assessors, while the contribution of posterior routing and CRV remains unclear.

  3. The methodological novelty is limited, and key assumptions are not discussed. Dirichlet–Thompson sampling and promotion by posterior comparison are standard bandit techniques, and CRV is a simulation-based value-of-information criterion closely related to knowledge-gradient and best-arm identification methods (e.g., top-two Thompson sampling), which are neither discussed nor compared. In addition, each arm is modeled as stationary and independent, although its value depends on the concurrently evolving versions in other clusters within the same episode, making the problem non-stationary.

  4. The experimental comparison does not isolate the contribution of version selection. Since the contribution lies in selecting versions rather than generating them, the baselines should share the same evolver and differ only in the selection rule (e.g., greedy by empirical mean, UCB, outcome-based Thompson sampling, a SkillOpt-style validation gate, and an EvoSkill-style elite pool [1]). Instead, Trace2Skill and SkillOpt differ in both generation and selection, so the comparison is confounded. EvoSkill [1] is not cited, and SkillAudit [2], which also evaluates skills without ground-truth labels, is cited but not compared.

  5. The evaluation scope and the analyses are limited. 1) Only ALFWorld (valid seen split only) and WebShop are used, with six manually defined clusters and three seeds, and ALFWorld is nearly saturated for two of the three executors; 2) all ablations and analyses are conducted only with Qwen3.5-9B; 3) the agreement and noise-robustness analyses rely on the same 30 completed decisions, which is a small and selected sample, and the injected noise is symmetric and independent rather than reflecting systematic assessor biases; 4) the cost comparison ignores the online exploration cost of routing real tasks to inferior versions, and no hyperparameter sensitivity is reported for κ\kappa, the promotion threshold, RpromR_{\text{prom}}, or Rmax⁡R_{\max}.

  6. Several details are unclear. 1) Whether the frozen system at test time uses the final incumbents or continues Thompson sampling; 2) how many versions are generated, promoted, and archived per cluster within the 500 adaptation episodes, given that promotion requires at least 64 rounds; 3) how the Human Skill baseline is constructed and whether all baselines start from the same initial skills and cluster structure; 4) some results look anomalous and need explanation, e.g., DeepSeek-V4-Flash obtains only 9.9 SR on WebShop without skills, and Trace2Skill falls below No Skill in this setting.

[1] EvoSkill: Automated Skill Discovery for Multi-Agent Systems. arXiv:2603.02766, 2026.

[2] SkillAudit: Ground-Truth-Free Skill Evolution via Paired Trajectory Auditing. arXiv:2606.14239, 2026.

Questions for authors
  1. Have you measured the accuracy of the local progress signals directly, e.g., by comparing the assessor outputs with human annotations on a sample of invocation intervals? How often does the assessor abstain, and how are the errors distributed across clusters and outcome types?

  2. Since all versions in a cluster share a fixed capability description and the cluster set is defined manually, how would SkillCred handle cases where evolution should change a skill's applicability scope, split or merge clusters, or create new ones? How often is the no-skill arm selected or promoted to incumbent in practice?

Reviewer_EvhROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SkillCred studies online valuation and lifecycle management of evolving skill versions. It collects ternary local-progress signals from normal task trajectories, maintains Dirichlet value posteriors, and uses cluster-first Thompson sampling among skill versions and a no-skill arm. Competitive Retention Value compares predicted returns with and without a candidate to guide early archiving. Frozen evaluation with three executors on ALFWorld and WebShop yields average per-executor relative gains of 14.5% and 38.2%, with local-signal, retention-rule, assessor, and noise ablations.

Critical strengths
  1. The experiments are well aligned with the central claims. Methods share a 500-task adaptation stream, adaptation and final test instances are disjoint, and systems are frozen before evaluation across three executors and two environments. This is a reasonable test of the resulting skill libraries.
Critical weaknesses
  1. Shared cluster descriptions do not ensure equal difficulty, task types, or execution stages across versions, and adaptive routing changes the observation distribution. Posterior rankings may partly reflect these context differences rather than version utility. Assessment also matters: scripted and LLM-only assessors yield WebShop success rates of 23.6 and 19.5. Overall system results do not establish reliable version valuation under matched contexts and reduced scripted support, limiting the estimator's scope.
  2. Removing lifecycle components or replacing retention criteria also changes subsequent generation parents and candidate libraries. The results support the complete online system but do not fully isolate CRV's increment on an identical candidate stream. The 28/30 held-out decision agreement covers only completed decisions. A fixed-candidate comparison and information on unresolved cases are needed for a stronger assessment of the retention policy itself.
Questions for authors
  1. Are tasks and invocation stages comparable across versions in a cluster? Is randomized-assignment or matched-context evidence available linking local posterior rankings to actual task gains from a version?
  2. Under the LLM-only assessor, are the main losses due to routing, promotion, or archiving errors? How often do its version decisions agree with the scripted assessor

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.