acceptodds
Under review as a conference paper at ICLR 2027

CtrlSkill: A Control-Theoretic View of Controllable Skill Self-Evolution

Abstract

Large language model agents can distill reusable skills from execution trajectories, yet accumulating experience does not necessarily yield reliable performance gains. Success on local validation is insufficient to justify persistent skill adoption, failures with mixed causes can lead to spurious generalization, and tool execution errors may be misattributed to reasoning deficiencies. We introduce CtrlSkill, a Control-Theoretic View of Controllable Skill Self-Evolution that organizes updates to an external skill policy as an evidence-driven feedback loop. CtrlSkill first constructs structured evidence to distinguish intended actions, tool execution, and observed outcomes. It then converts shared evidence gaps into candidate skills with explicit applicability conditions and verification requirements. Finally, it compares the active and candidate policies on subsequent matching tasks, controlling whether updates take effect through acceptance, deferral, or rejection. By decoupling candidate generation from policy commitment, CtrlSkill subjects skill updates to explicit constraints on supporting evidence, edit scope, and validation feedback. Across these settings, CtrlSkill raises the overall average score over baselines by up to 23%.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Self-Evolution;Skill Self-Evolution;Controllable

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_C9HFOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CtrlSkill improves external agent skills without updating model weights. It organizes execution traces into structured evidence, proposes scoped skill edits, and admits candidates only after paired comparisons with the active policy on subsequent matching tasks. Experiments report improvements on SpreadsheetBench and WikiTableQuestions with two skill-author models, alongside component ablations, transfer to Kimi K3, and a DocVQA evaluation.

Critical strengths
  1. A coherent approach to controlling persistent updates. Separating candidate generation from admission addresses a practical failure mode: plausible lessons can harm later execution. Grounding edits in tool behavior and saved artifacts makes the proposed feedback more actionable and auditable.

  2. Substantial reported improvements. On held-out SpreadsheetBench, CtrlSkill improves over Trace2Skill by 26.5 and 12.5 percentage points in GPT-5.5 and DeepSeek Deepening, respectively. Removing individual components reduces GPT-5.5 Deepening accuracy by 17–19.5 points. These results provide useful initial evidence for the design.

Critical weaknesses
  1. The incremental contribution over closely related methods is not established experimentally. SkillCAT [1] already filters candidate patches through replay, while SkillOpt [2] accepts bounded edits only after held-out validation improves. Direct comparisons with these methods are needed to establish CtrlSkill's incremental benefits. Subsequent-task admission and structured evidence may offer additional value, but comparisons only against Trace2Skill do not establish this advantage. Missing compute accounting also prevents assessing the benefit of additional validation relative to its cost.
  2. The core admission mechanism is insufficiently specified and validated. Equation (9) defines clustering similarity, but the signature extraction procedure and task–candidate matcher m(z,s) are not operationally specified. These determine which examples support admission. The detailed analysis covers only six candidates and 18 matched-task validations; ablations cover one configuration, with no reported variability across runs or task orders. The reliability of admission from such limited evidence remains uncertain.

References:

[1] SkillCAT: Contrastive, Assessment-Augmented and Topology-AwareSkill Self-Evolution for LLM Agents

[2] SkillOpt: Executive Strategy for Self-Evolving Agent Skills

Questions for authors
  1. Comparison and cost — targeted validation. Can you compare against SkillCAT and a held-out-validation method such as SkillOpt under matched models, splits, and computation budgets? Please report total evolution tokens/calls or time, including evidence construction, editing, and paired validation.

  2. Matching — clarification. How are structural signatures and m(z,s) computed in practice? Please provide the extraction/routing algorithm or prompts and clarify which inputs are available before versus after baseline execution.

  3. Metric consistency — clarification. Table 1 reports Soft = 23.8% and Hard = 28.6% for DeepSeek Human-Written. The official SpreadsheetBench definitions imply Hard ≤ Soft on the same evaluated tasks. Is this a reporting error or a difference in evaluation protocol? Please reconcile the scores and any affected averages.

  4. DocVQA — clarification. Section 4.1 specifies 50 DocVQA examples for evolution, whereas Section 5 describes transfer beyond spreadsheet tasks used during evolution. Does Table 4 evaluate frozen spreadsheet skills or skills adapted on DocVQA? The distinction changes the generalization claim.

  5. Admission robustness — targeted validation. Do the promotion decisions and ablation gains persist across task orders or repeated evolution runs? C1 and C2 are admitted after repairing only one matching failure each, so variability could materially affect the reliability conclusion.

Reviewer_UPWsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper casts skill self-evolution in LLM agents as a feedback-control loop. Each trajectory is summarized in an evidence card that records the task goal, missing evidence, tool outputs and artifact checks. Weak trajectories are clustered by structural signature, and a proposer drafts candidate skills with explicit applicability conditions and verification requirements. The distinguishing component is delayed admission: a candidate is held in a pending queue and becomes active only after paired comparisons on later matching tasks show a gain over the current policy, and it is otherwise eventually rejected. Implemented on the Trace2Skill framework, the method is evaluated on SpreadsheetBench, WikiTQ and DocVQA with GPT-5.5 and DeepSeek-V4-Flash, together with a transfer study on Kimi K3. The reported gains are substantial. In the GPT-5.5 Deepening setting, accuracy on the verified split rises from 33.0 with human-written skills to 64.5, and removing admission control, evidence accumulation or tool constraints each costs between 17 and 19.5 points.

Critical strengths
  1. Decoupling skill generation from commitment to the active policy is a sensible design choice, since most existing methods adopt a skill as soon as it is written. The paired evaluation on subsequent tasks is easy to implement, and the ablation without admission, which promotes eight skills yet falls to 45.0, makes a convincing case that a larger skill library is not automatically a better one.
  2. The workbook example in Appendix A.2 illustrates the value of execution-aware observation well. The agent writes correct formulas, but the saved file carries empty cached values, so the tool reports success while the evaluator sees nothing. Detecting failures of this kind, which leave no trace in the language of the trajectory, is probably the most transferable idea in the paper.
Critical weaknesses
  1. It is unclear whether the gains stem from the control framework or from one or two benchmark-specific fixes. Training yields only six candidates, three of which are promoted, and these three skills alone lift GPT-5.5 by more than thirty points. The worked example and all three evidence-chain types in Appendix A.5 describe the same failure, namely missing cached values after saving, which looks like a property of the evaluation pipeline. A single instruction to write literal values, or to recalculate and read the file back, could plausibly account for much of the improvement, in which case the results say little about the generality of the evidence-state and admission machinery. The admission gate is also statistically weak. Candidate C1 is accepted after repairing one of three matched tasks, each evaluated with a single stochastic rollout, so one favorable sample is enough to pass. Finally, the control-theoretic framing remains largely terminological. The paper introduces a plant, an observer and a controller, but never analyzes stability, convergence or any other property that the analogy would suggest.
  2. Several reported numbers are difficult to reconcile, and the empirical comparison is thin. In Table 1, the human-written baseline for DeepSeek-V4-Flash scores 23.8 under the soft restriction and 28.6 under the hard restriction, although the hard metric requires passing every test case and cannot exceed the soft one. The GPT-5.5 baseline reaches only 33.0, compared with 69.5 for DeepSeek-V4-Flash, which raises the possibility that the GPT-5.5 harness has a defect that the evolved skills merely repair. The soft and hard metrics are computed on the full SpreadsheetBench, from which the 200 training tasks were presumably drawn, so evaluation and training may overlap. All results come from single runs without variance estimates. Trace2Skill is the only learned baseline, SkillCAT is described as using the same setting but is not compared, and the additional cost of running both policies during validation is not reported. The manuscript would also benefit from careful editing, as the abstract, for instance, refers to these settings before any have been introduced.
Questions for authors
  1. What proportion of the repaired failures is attributable to the missing cached values? If that category is excluded, or if the harness is corrected to recalculate formulas before scoring, how much of the improvement remains? The answer would substantially change my view of the paper.
  2. Could the authors explain why the hard score exceeds the soft score for the DeepSeek-V4-Flash human-written baseline, and state whether the training tasks overlap with the set used for the soft and hard metrics?
  3. How stable is the admitted skill set across random seeds? With a threshold of three matches and single rollouts, I would expect considerable variation.
  4. Why is SkillCAT omitted from the comparison, and what is the rollout overhead of paired validation relative to Trace2Skill?
Reviewer_sLk2OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

CtrlSkill separates skill evolution into evidence collection, scoped candidate generation, and delayed admission. On SpreadsheetBench and transfer benchmarks, it outperforms the reported baselines; ablations suggest that admission control, evidence accumulation, and tool-grounded verification each contribute to performance.

Critical strengths
  1. The separation of candidate generation from policy adoption is a clear way to reduce harmful updates from isolated successes or ambiguous failures.
  2. The ablations test key components, and the artifact-level examples help explain how execution evidence informs skill updates.
Critical weaknesses
  1. The paper does not report the runtime or token cost of paired candidate validation, making the practical overhead difficult to assess.
  2. It provides limited analysis of how robust results are to the admission thresholds or how those thresholds transfer across tasks.
Questions for authors
  1. What are CtrlSkill’s runtime and token costs relative to the baselines, especially for paired validation?
  2. How sensitive are results to the admission thresholds, and were the thresholds retuned for DocVQA?

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Responses

Summary of Responses

Dear Chairs and Reviewers,

We thank you and the three reviewers for the careful evaluation of our work and the valuable suggestions. These comments have helped us clarify the method, the experimental evidence, and the scope of our conclusions. Since several concerns overlap across the reviews, we summarize our main responses and the corresponding manuscript locations below.

◆ Whether CtrlSkill’s contribution comes only from cached-value or formula repairs

We explain how CtrlSkill updates skills through structured execution evidence, scoped candidate edits, and delayed admission. The failures it addresses include not only missing cached values but also non-empty yet semantically incorrect outputs, such as errors in source-data and target-range mapping, auxiliary-list usage, and sorting rules.

Corresponding locations: Sections 3.2–3.4, Section 5, and Appendices A.2 and A.5.

◆ Whether a candidate is accepted after a single success, and how task matching works

We clarify that a single success does not directly trigger admission. Candidates must accumulate paired-validation evidence on subsequent matched tasks and jointly satisfy the requirements for sample size, baseline-failure count, repair rate, and mean gain. The matching set is determined by baseline routing before candidate execution, and candidate results do not modify this set. We also provide pseudocode covering structural-signature extraction, candidate editing, and admission.

Corresponding locations: Sections 3.2–3.4, Equations (8)–(26), Section 4.2, and Appendices A.1 and A.6.

◆ Admission-threshold sensitivity and whether thresholds were retuned for DocVQA

We analyze the admission decision boundaries using existing paired-validation records and explicitly distinguish candidate decisions under fixed evidence from guarantees about final accuracy after rerunning the entire evolution process. For DocVQA, the minimum validation sample size remains 3 and the mean ANLS gain threshold remains 0.02, while the additional binary repair-rate conditions used for SpreadsheetBench are not imposed.

Corresponding locations: Section 4.2 and Appendix A.7.

◆ Stability under different training-task orders

We conducted supplementary experiments with three random seeds under the DeepSeek-V4-Flash Deepening setting. CtrlSkill achieves 93.7% ± 0.29% accuracy, compared with 82.7% ± 1.00% for Trace2Skill + Comb. The responses also describe the common operational themes of the experiences retained across runs, while distinguishing this observation from a guarantee that the skill texts are identical.

Corresponding locations: Table 5 reports accuracy across repeated runs. Table 2 and Appendices A.1 and A.4 provide ablation results and process analyses for admission control and evidence accumulation. Observations about experience-content consistency are provided in the reviewer responses.

◆ Runtime, token costs, and comparisons with related methods

We report accuracy, evolution and testing token costs, and runtime for Trace2Skill and CtrlSkill, with paired-validation costs listed separately. We additionally compare with SkillOpt to address the request for a method using held-out validation. For SkillCAT, the responses explain why a direct reproduction was not conducted.

Corresponding locations: Section 5 and Table 5 provide the overall comparison. Appendix A.8 and Table 9 provide stage-level costs and the SkillOpt configuration. SkillOpt’s accuracy and runtime are reported in the reviewer responses. Appendix A.3 compares the mechanisms of related methods.

◆ Evaluation metrics, data splits, and the interpretation of the DocVQA experiment

We corrected the swapped values in Table 1 to Soft = 28.6% and Hard = 23.8%. The responses clarify that the Vrf evolution and test sets are disjoint, while the full Soft/Hard evaluation scope includes some evolution tasks.

For DocVQA, we clarify that 50 samples are used for task-specific Creation and 100 independent samples are used for evaluation. This experiment demonstrates applying the framework to a new task, rather than directly transferring frozen SpreadsheetBench skills across domains.

Corresponding locations: Table 1, Section 4.1, Section 5, and Table 4. The overlap between the Soft/Hard evaluation scope and evolution tasks is explained in the reviewer responses.

◆ Positioning of the control-theoretic framework

We clarify that control theory describes the functional relationships among observation, candidate editing, feedback, and delayed updates. The paper does not claim formal stability or convergence guarantees.

Corresponding location: Section 3.1.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.