acceptodds
Withdrawn from ICLR 2027

SkillHelix: Coevolving Agent Skills via Decision and Reasoning Synergy

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Agent skills are natural-language documents that guide a model through a class of tasks. The model can improve these skills by examining its failures and rewriting the instructions. Such loops assign credit to entire rewrites. They reveal whether a rewrite helped, but not which edits contributed. Harmful edits are therefore accepted alongside helpful ones, and the optimizer skill guiding the rewrites cannot learn which kinds of edits work. Assigning credit to individual edits requires counterfactual evaluations, with up to runs for edits. Yet accuracy on the small batch used in each round rarely distinguishes one edit combination from another. We obtain this credit by pairing the black-box reasoning model with a small open-weight decision model. The reasoning model's outcomes ground the signal in its own behavior, identifying the questions an edit should fix and those it must preserve. The decision model provides a dense signal through forward passes that measure how each edit combination changes the likelihood of the gold answers on these questions. SkillHelix uses this signal in two ways. It selects edits for the task skill and refines the wording of their most influential span, retaining the change only if validation accuracy improves. It also assigns a Shapley value to every proposed edit and uses these credits to revise the optimizer skill, allowing the two skills to coevolve. With GPT-5.5 as the reasoning model, SkillHelix outperforms all baselines on six benchmarks, improving accuracy by an average of 3.8 points over the strongest baseline and by 12.9 points on LiveMath. Ablations show that edit-level decisions and coevolution both contribute to these gains.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Reasoning, Language agents, Skill co-evolution, Recursive self improvement.

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_PKfaOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

SkillHellix proposes iterative optimization of skills for black-box LLMs using an open-weight model as a proxy evaluator of the combinations of proposed edits. The decision model computes the teacher-forced likelihood on datasets and select the edit subset that most increases likelihood on failed questions, without lowering on solved questions. The same subset scores are also converted into Shapley values, which the optimizer model uses to optimize its own skill. Experiments cover six benchmarks and five reasoning backbones.

Critical strengths
  1. Evaluating every combination of edits with the black-box model would be expensive, so leveraging token probabilities from a open-weight model is a simple and practical way to get feedback at the level of individual edits.

  2. The method is evaluated on six benchmarks, with five different reasoning backbones. It outperforms strong baselines such as GEPA and SkillOpt, so the gains do not seem limited to a particular task or model.

Critical weaknesses
  1. (Prensentation) The title is not intuitive, since it does not say what the method actually does. In Figure 1 (right), it is unclear what each axis exactly measures. In Figure 2, M\mathcal{M} appears in the caption but not in the figure, and the text uses “reasoning model” and “decision model” while the figure uses “Reasoner” and “Decider.”

  2. The paper mostly reports final scores and does not explain much about why the method works. Without this kind of analysis, it is hard to understand where the gains come from. For example, The method relies on the assumption that raising the decision model’s answer likelihood on failed questions also raises the reasoning model’s accuracy. The paper states this assumption but does not show it experimentally.

  3. With n=4n=4, there are only 16 subsets, so selecting among them directly by the reasoning model’s accuracy seems feasible. This baseline is not included, so it is not clear whether the decision model actually helps compared with direct black-box selection.

  4. The paper does not clearly explain how spans are defined in span evolution. There is also no analysis of which spans were selected or how they were changed.

  5. Shapley values are used to credit each edit, but the paper does not compare them with other ways of assigning credit. So it is hard to tell whether Shapley values are actually important here.

  6. The decision model computes the likelihood by teacher-forcing the answer after the question, without any reasoning. The reasoning models (ex. GPT 5.5) answer after long reasoning, so it is not clear that the proxy score would change in the same direction.

Questions for authors

Refer to the weaknesses.

Reviewer_Z9ipOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper looks at reflective skill optimization, where a black-box LLM improves a natural-language "skill" document by reading its own failures, and argues that current loops give credit to a whole rewrite instead of to the individual edits in it. SkillHelix proposes n edits per round (n = 4) using an optimizer model (GPT-5.5) guided by an "optimizer skill". A small open-weight decision model (Qwen3.5-9B) then scores all 2^n edit combinations by the length-normalized teacher-forced log-likelihood of the gold answer. Scoring runs on the training-batch questions that the black-box reasoner failed (the objective) and on those it solved (a constraint that this likelihood must not drop). The best admissible combination is picked. The span with the largest one-step integrated-gradients attribution is rewritten k = 8 times, and the best rewrite is kept if it improves accuracy on a held-out gate set. Exact Shapley values of each edit under the same likelihood surrogate go into a feedback record that the optimizer model uses to revise the optimizer skill, so the two skills "coevolve". Experiments use the six-benchmark suite and splits of SkillOpt (SearchQA, OfficeQA, DocVQA, LiveMathematicianBench, SpreadsheetBench, ALFWorld) with five reasoning backbones. The paper reports that SkillHelix beats the strongest alternative in 28 of 30 settings, with a GPT-5.5 average gain of 3.8 points and a 12.9-point gain on LiveMath. It also includes component and edit-pool ablations (GPT-5.5 only) and an optimizer/reasoner pairing grid.

Critical strengths
  1. The problem framing is useful and the mechanism is simple. Separating "where to look" (the black-box model's solved/failed split) from "how much an edit moves things" (a cheap white-box forward pass) is a clean idea, and it is the main reason I take the paper seriously.

    • Because n is small (2^4 = 16 coalitions), the Shapley values are exact under the surrogate rather than sampled.
    • The black-box validation gate is a reasonable safeguard against the surrogate being wrong.
  2. The empirical sweep is broad, and one ablation gives at least directional support for the selection step. The evaluation covers six benchmarks, five backbones including an open-weight one, and six comparison methods (Table 1).

    • The all-edits-applied ablation (Table 2) is lower than Full on all six benchmarks.
    • For GPT-5.5: 80.6 vs 86.1 on SearchQA, 58.1 vs 67.7 on LiveMath, 89.6 vs 95.5 on ALFWorld.
Critical weaknesses
  1. The proxy that carries the method is never validated, and the ablations do not isolate it. Every selection decision, span-evolution decision and Shapley credit comes from Qwen3.5-9B likelihoods, and the paper gives no evidence that these predict the reasoning model's behaviour.

    • No validation. There is no correlation between r(C) and the reasoner's accuracy change, no check that highly ranked coalitions do better for GPT-5.5, and no comparison of Shapley credit with leave-one-out accuracy. Figure 1 is "a real round", but the claim that dropping e1 is better is measured only under the proxy.
    • The logic is odd. The solved/failed split is defined by M, but the optimized quantity is how likely a different 9B model finds the gold answer, read at the answer position with no reasoning (Eq. 3), for a skill written in "the prompt format of M" for a black-box GPT-5.5. Why should raising p_theta's likelihood of an answer GPT-5.5 reaches through multi-step reasoning mean GPT-5.5 will now reach it?
    • Targets are unclear or off-policy. For ALFWorld the targets are successful GPT-5.5 trajectories (A.3), off-policy for the other four reasoners. I could not tell what the "gold answer" is for SpreadsheetBench.
    • Alternative explanations aren't ruled out. The gains could come from the extra k = 8 span rewrites per round, from proposing fewer and more conservative changes, or from the gate filtering out bad candidates. "All edits applied" is the weakest possible alternative to selection, and it's unclear whether span evolution and the gate stay on in that variant.
    • Missing controls. A non-likelihood selector at matched cost (a random admissible subset, or selection by the reasoner's own batch accuracy) is needed; the claim that batch accuracy "rarely distinguishes one edit combination from another" is never shown. The frozen-optimizer ablation shows that updating the optimizer skill helps on five of six benchmarks, not that Shapley credit is what helps. An update using only the gate outcome, or uniform credit, would test this.
  2. The statistical support is thin compared with the size of most margins, and the comparison conditions are not specified. Every number comes from one run with no seeds, intervals or significance tests, even though LLM optimization loops are high-variance.

    • Many wins are about one test item (Table 3 sizes). Qwen DocVQA 91.7 vs 91.4 (about 1 of 374), Qwen Spreadsheet 47.9 vs 47.5 (about 1 of 280), Qwen OfficeQA 47.7 vs 47.1 (about 1 of 172), GPT-5.5 OfficeQA 73.3 vs 72.1 (about 2 of 172). SearchQA, with 1,400 items, is the exception. So "28 of 30" mostly counts margins that may not survive a second seed.
    • The LiveMath gain (67.7 vs 54.8) is about 16 of 124 questions. That is meaningful, but it is one run, and on the 18-item gate one question is worth 5.6 points.
    • Figure 5 margins have the same issue: 4.17 points on a 24-item gate is one question, and 7.50 on a 40-item gate is three tasks. The final margins are also oddly identical for GPT-5.4 and GPT-5.5 on both benchmarks.
    • The abstract's "3.8 points over the strongest baseline" holds only against SkillOpt as a single method. Against the best baseline per benchmark (GEPA on SearchQA), the GPT-5.5 average is (1.3+1.2+1.6+12.9+1.5+0)/6 ≈ 3.1, and five of six gains are at most 1.6 points.
    • Conditions are unspecified. I could not find whether baselines were re-run or copied from SkillOpt, whether they use GPT-5.5 as optimizer, or whether they get the same rounds and black-box calls. SkillHelix gets k = 8 extra candidates per round plus white-box compute, and |B_t|, wall-clock time, API cost and GPU cost are not reported. Trace2Skill appears in Table 1 with no citation, and the source of the "Human skill" documents is not given.
  3. The method section is compact, but I had to read the first three pages several times to work out who does what. The introduction uses "rewrite-level" and "edit-level" credit before defining them, and "judges the rewrite by its effect on accuracy" (p. 1) leaves it unclear whether the model judges its own rewrite or the skill. For a while I also could not tell which model's likelihoods are used. Figure 1 plots "change on failed/solved" without saying it is the decision model, and the decision model p_theta is introduced in the text before Figure 2 shows it. It only became clear in Sec. 2.2 that the black-box optimizer proposes edits and the open-weight model only scores them, and that no likelihood threshold is involved beyond the "does not lower l+" admissibility test. The paper also never gives the intuition for why a separate 9B model's gold-answer likelihood should say anything about edits for GPT-5.5 (see Weakness 1). A short paragraph early on that names the three models and their roles, and a figure that labels the likelihood axes by model, would help a lot. Separately, the paper says little about the experiments: the appendix is about one page, and several details needed to interpret the results are missing (listed under Questions).

Questions for authors

Clarifications that could change my assessment:

  1. Can you report direct proxy validation? For example: across rounds, the rank correlation between r(C) and the reasoner's accuracy change on gate or test questions for all 16 coalitions, or at least for C* against the full rewrite. Also, how often does the gate reject the selected skill?
  2. Can you add a matched-cost selection control: a random admissible subset, or selection by black-box accuracy on B_t, with span evolution and the gate unchanged? In the "All edits applied" variant, are span evolution and the gate still active?
  3. For the coevolution claim, can you ablate Shapley credit specifically, e.g. optimizer-skill updates using only the gate outcome, or using leave-one-out credit?
  4. How many seeds were run? Please report mean and standard deviation, at least for the GPT-5.5 rows of Table 1 and for Table 2.
  5. Were the baseline numbers re-run under your setup or taken from SkillOpt? Do all iterative baselines use GPT-5.5 as optimizer, the same four-epoch schedule, and the same budget of black-box calls? What is the total cost of SkillHelix compared with SkillOpt and GEPA, in API calls/tokens and GPU hours?
  6. What exactly is teacher-forced as y* for each benchmark, in particular SpreadsheetBench and the multi-step ALFWorld trajectories? What does "prompt format of M" mean for a closed GPT model? For ALFWorld with non-GPT-5.5 reasoners, are the GPT-5.5 reference trajectories still used?
  7. What is |B_t|, and how many questions are in B_t^- in a typical round? With a 35-question LiveMath train split, the admissibility test seems to rest on very few questions.
  8. Why are the final gate margins in Figure 5 identical for GPT-5.4 and GPT-5.5 on both Spreadsheet (7.50) and OfficeQA (4.17)? Why does Figure 5 show eight evolution steps under a "four-epoch" schedule?
  9. How does the method behave when the decision model is much weaker than, or from a different family than, the reasoner? Qwen3.5-9B and Qwen3.6-35B-A3B are the same family, yet that is the backbone with the smallest gains over SkillOpt. A different or larger decision model in an ablation would help separate proxy quality from method design.
  10. Algorithm 1, line 1 splits B_t into solved and failed questions. What decides correctness for each benchmark (exact match, a checker, an LLM judge)? For tasks without a crisp verifier, how reliable is this split, and does an error in it propagate into the admissibility test?
  11. The OfficeQA/GPT-5.4-nano result (25.6 vs 46.5 for human skills) is a large failure. Do you have a diagnosis, e.g. the proxy disagreeing with a very weak reasoner?

Suggestions (not required for my assessment): 12. Releasing the final task skills, optimizer skills, and the fixed instructions used for the optimizer-skill update would help reproducibility. So would a qualitative look at how the optimizer skill changes over rounds. 13. A short discussion of how prompts and skills transfer across models would be useful, since the method depends on it. Sclar et al. is cited only for the gate.

Reviewer_74v4OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes SkillHelix, a framework that coevolves agent skills through decision and reasoning synergy, addressing the limitations of existing skill optimization methods in attributing contributions to individual edits. SkillHelix uses a white-box Decision Model to evaluate combinations of skill edits and applies Shapley values for edit-level credit assignment. It further integrates span-level skill evolution and Optimizer Skill updates to jointly improve task skills and the optimization process. Experiments across six benchmarks and five reasoning models show that SkillHelix generally outperforms existing baselines, while ablation studies demonstrate the contributions of edit selection, span evolution, and Optimizer Skill updates.

Critical strengths
  1. Fine-grained credit assignment and coevolution. SkillHelix uses a frozen white-box Decision Model to evaluate skill-edit combinations on failed cases and applies exact Shapley values to quantify individual edit contributions, moving beyond conventional rewrite-level evaluation. Its joint optimization of task skills and Optimizer Skills also provides a valuable framework for continuous skill improvement.
  2. Broad and convincing experimental validation. The paper evaluates SkillHelix across six benchmarks and five reasoning models. It outperforms the strongest baseline in 28 out of 30 model-benchmark settings and achieves an improvement of up to 12.9 percentage points on LiveMath. Ablation studies further validate the effectiveness of its key components.
Critical weaknesses
  1. Insufficient direct validation of the core proxy mechanism. SkillHelix uses the answer likelihood from a white-box Decision Model as a proxy for the effectiveness of skill edits. However, the paper does not adequately establish its alignment with the actual task performance of the black-box Reasoning Model or validate the accuracy of edit-level credit assignment. This leaves the central assumption underlying the method insufficiently supported.
  2. Limited statistical reliability and reproducibility. The paper does not report results across multiple random seeds, confidence intervals, or statistical significance tests, while some benchmarks have relatively small validation sets. Consequently, it is difficult to rule out random variation or validation overfitting, limiting confidence in the stability of the reported improvements.
Questions for authors
  1. Proxy alignment: Have the authors evaluated the correlation between changes in the white-box Decision Model's teacher-forced gold-answer likelihood and changes in the black-box Reasoning Model's actual accuracy? In particular, do Shapley values reliably reflect the true contributions of individual edits to black-box performance?
  2. Computational cost and scalability: The current method uses an edit pool of (n=4) and requires evaluating (2^n) edit combinations, computing Integrated Gradients, and generating and reranking (k=8) candidates. Can the authors provide runtime and resource consumption measurements for these steps, and clarify how the method scales with larger edit pools or longer contexts?
Reviewer_dFWfOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The reflective skill optimizer accepts or rejects edits as a whole bundle, so it can't tell which specific edit actually did the work. SkillHelix's approach is to bring in a small open-source decision model (Qwen3.5-9B) to evaluate edit combinations: it tries every subset of n=4 edits and measures how much each subset raises the likelihood of the gold answer on questions the black-box reasoning model got wrong, while not lowering the likelihood on questions it got right. After the best subset is refined, it's only kept if the gate accuracy actually improves. The same set of scores can also be used to compute Shapley values, which are then used to revise the optimizer's own instructions, letting task skills and optimizer skills evolve together. Across 6 benchmarks × 5 reasoning models, the paper reports that SkillHelix wins 28 out of 30 settings, with the largest gain being +12.9 for GPT-5.5 on LiveMath.

Critical strengths

(1) Clear problem identification and a clever, testable solution. The paper identifies a real limitation of optimizers that assign credit to entire rewrites: harmful edits may be accepted along with useful ones. It proposes using a small model to score all edit combinations on problems the reasoning model gets right or wrong. This is inexpensive, and the same scores guide both edit selection and optimizer feedback. Final acceptance is still controlled by the black-box model on a validation set, so inaccurate scores from the small model will not reduce accuracy on the validation gate. The idea is valuable, and its effectiveness depends on the reliability of the small model's scores, which can be tested experimentally.

(2) Broad evaluation. The paper compares against strong recent baselines, including GEPA and SkillOpt, across 6 benchmarks and 5 reasoning models. According to the reported results, removing edit selection lowers GPT-5.5's scores on all six benchmarks. Even when using the numbers from SkillOpt's original paper, SkillHelix still leads in 25 of 30 settings. Gains are especially clear on LiveMath with weaker reasoning models (e.g., +7.6 for GPT-5.4 and +9.1 for GPT-5.4-mini). However, most other gains are small.

Critical weaknesses

(1) Key baseline results are inconsistent with the cited original paper, without explanation. Seven SkillOpt scores in Table 1 are lower than those originally reported (e.g., GPT-5.5 LiveMath: 54.8 vs. 66.9; GPT-5.4-nano OfficeQA: 12.2 vs. 50.0). Nearly all other baseline scores match the original paper exactly, except for one likely typo. The submission does not explain where these numbers come from. Using the original scores, SkillHelix's advantage on GPT-5.5 largely disappears (2 losses, 1 tie, 3 small wins), undermining the paper's main conclusion.

(2) The core assumption, fairness of comparison, and remaining gains are unverified. No experiments show that changes in teacher-forcing likelihood from a 9B model predict accuracy changes in black-box reasoning models. Thus, it is unclear whether the gains come from the credit signal or simply additional search. The comparison is also not compute-matched: SkillHelix adds a small model, 8 rewrite candidates per round, and optimizer skill updates on top of SkillOpt, without reporting costs. Some gains may simply result from extra compute. Moreover, most improvements are only 1–2 points (a few test questions), potentially within noise. Without repeated runs or variance estimates, even the remaining gains are not established.

Questions for authors
  1. Why are the SkillOpt baseline results inconsistent? In Table 1, seven results are lower than those in the original paper (GPT-5.5: SearchQA 80.4/87.3, LiveMath 54.8/66.9, Spreadsheet 76.4/80.7; GPT-5.4: SearchQA 78.5/83.1, DocVQA 90.4/91.2, Spreadsheet 61.4/62.5; GPT-5.4-nano: OfficeQA 12.2/50.0). All other results match, except for one possible typo. Please explain these discrepancies and compare against the original results. Otherwise, the main conclusions are questionable.

  2. Are the small-model scores reliable? We suggest sampling several rounds and using GPT-5.5 to evaluate all 16 combinations of four edits per round. Report the Spearman rank correlation between small-model scores r(C)r(C) and actual accuracy gains, as well as how often the selected combination C∗C^* matches the true best combination.

  3. Are the improvements stable and significant? Gains over the strongest baseline are usually only 1–2 points, while five of the six benchmarks have only 18–53 validation examples. Please run at least three independent trials with GPT-5.5, varying training order and data splits. Report the mean, standard deviation, and paired significance tests against SkillOpt under identical conditions.

  4. Is the budget comparison fair? SkillHelix uses an additional small model, eight rewrites per round, and one optimizer update, beyond SkillOpt. Please compare API tokens, validation evaluations, and GPU time across SkillHelix, SkillOpt, and GEPA, and rerun comparisons under equal budgets.

  5. How does the small model evaluate different tasks? Appendix A.3 states that successful GPT-5.5 trajectories are used as reference targets for ALFWorld. Please clarify how these trajectories are serialized for Equation 3 and which tokens are scored. Please also specify the reference targets and scoring methods for free-text QA (SearchQA, DocVQA, OfficeQA) and spreadsheet operations (SpreadsheetBench), whether reasoning traces are included, the batch size ∣Bt∣|B_t| per round, and the ratio of correct to incorrect examples.

2 comments
AuthorsOpenReview ↗

Clarifying the SkillOpt Baselines: Repeated Runs and Independent Evidence

We sincerely thank the reviewer for the careful and detailed comparison with the original SkillOpt results. Your scrutiny highlights an important issue in baseline reporting, and we appreciate the opportunity to clarify the evidence and improve the transparency of our comparisons. Independent studies have also reported reproduction difficulties or obtained SkillOpt baseline scores different from those originally published [1–3]. These discrepancies are therefore not unique to our experiments.

We conducted three full GPT-5.5 SkillOpt runs using the released implementation and four-epoch configurations. The mean and sample standard deviation are:

BenchmarkSkillOpt: three-run mean ± SD
SearchQA80.5 ± 0.6
SpreadsheetBench75.6 ± 0.9
OfficeQA71.9 ± 0.9
DocVQA91.0 ± 0.5
LiveMath55.1 ± 0.5
ALFWorld95.8 ± 0.5

The repeated results remain close to the lower scores reported in our table. In particular, SearchQA, SpreadsheetBench, and LiveMath remain below the published scores of 87.3, 80.7, and 66.9, respectively. The gaps persist across repeated runs and substantially exceed the observed run-to-run standard deviations, supporting a persistent reproduction discrepancy within our setting.

The external evidence is consistent with this observation. COBRA-Skills and SoftSkill document reproduction difficulties or evaluation discrepancies [1,2]. SkillOpt-Lite also reports different SkillOpt baseline scores, including 78.8 on GPT-5.5 SearchQA and 76.2 on SpreadsheetBench [3]. These studies use their respective evaluation settings and provide context for the discrepancies.

We agree with the reviewer that readers should be able to distinguish published results from our reproduced measurements. We will make the provenance of the baseline entries explicit and present published and reproduced results separately. As your review notes, SkillHelix still leads in 25 of 30 settings when the original SkillOpt scores are substituted, although the GPT-5.5 improvement margins become smaller.

We appreciate the time and care invested in this assessment. We respectfully ask that the repeated-run evidence, independent reports, and clarified evaluation settings be considered when reassessing the baseline concern and the associated ethics flag. Your feedback will help us make the experimental reporting clearer and more useful to readers.

[1] COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization. arXiv:2609.11682, 2026.

[2] SoftSkill: Behavioral Compression for Contextual Adaptation. arXiv:2606.20333, 2026.

[3] SkillOpt-Lite: Better and Faster Agent Self-evolution via One Line of Vibe. arXiv:2607.03451, 2026.

Thank you for conducting the additional experiments and providing this clarification.

In LLM and agent benchmarks, reproduction differences across studies are normal and can arise from differences in API versions, environments, or the stochasticity of LLM outputs. Precisely for this reason, having several SkillOpt entries differ in Table 1 while almost all other baseline numbers match the original papers raised concerns for me.

Given your explanation and clarification of these results, I have withdrawn the Ethics Flag and raised my rating from 1 to 2.

However, because my concerns regarding the paper's soundness and my main technical questions (Questions 2–5) have not yet been addressed, I cannot raise my score further at this stage. I look forward to your responses to those questions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.