acceptodds
Under review as a conference paper at ICLR 2027

COBRA-Skills: Contextual Bandit-Guided Evolution for Agent Skill Optimization

Abstract

Large language model (LLM) agents can benefit from reusable skills distilled from prior task experience, yet existing skill optimization methods often rely on costly execution-based evaluation and substantial task data. We introduce **COBRA-Skills**, an efficient framework that formulates skill optimization as budgeted sequential optimization over a dynamically evolving candidate space. COBRA-Skills couples contextual-bandit-guided prioritization with evidence-grounded skill evolution, selectively allocating evaluations to promising or informative candidates while continually refining the skill population from execution feedback. Across six heterogeneous agent benchmarks and three target models, COBRA-Skills consistently achieves the strongest average performance among compared methods, while reducing optimization cost by 55–58% and cost per point of improvement by 60–69% relative to SkillOpt, using only 50 unique optimization examples per benchmark. Further analyses show that COBRA-Skills remains robust to changes in the agent harness and performs effectively when the target model itself is used for skill generation and refinement.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent Skills, Bandits, Skill Evolution, Skill Optimization

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_kCs5OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a framework for optimizing agent skills using contextual bandits and evolutionary search. It uses a mlp with a LinearUCB bonus to select promising skills, and periodically generates or refines skills based on execution feedback. Experiments on six benchmarks and three models show improvements over existing skill optimization methods, with lower optimization costs.

Critical strengths
  1. The method achieves strong average performance across six benchmarks and three models, while reducing optimization costs compared with SkillOpt.

  2. The paper includes component ablations, self-teaching experiments, and evaluations under different agent harnesses.

Critical weaknesses
  1. Similar bandit-based optimization ideas have also been explored in prompt optimization.
  2. The bandit mechanism needs more detailed analysis.More comparisons with simpler selection strategies, such as greedy reward prediction or UCB without the neural predictor, would help justify the design. the predictor could be also a overfit iid model.
Questions for authors
  1. Could you compare with simpler selection methods and provide more detailed ablations of the bandit components?

  2. Could you compare against skilopt under the same optimization data and computational budget?

4 comments

Response to Reviewer kCs5 (Part 1/2)

Thank you for your careful review and insightful feedback.


Response to W1

W1. Similar bandit-based optimization ideas have also been explored in prompt optimization.

We agree that bandit-based optimization has precedents in prompt optimization. In our setting, skill optimization requires generating and refining reusable task procedures from execution trajectories, including those involving tool use and environment interaction, such as spreadsheet manipulation. Both executing candidate skills for evaluation and generating or refining them with a teaching model can be costly. Our contribution is to couple bandit-guided evaluation allocation with evidence-grounded skill evolution: the bandit prioritizes which skills to execute, and the resulting feedback is reused for subsequent refinement and population updates. This coupling supports adaptive search over an evolving space of task-level skills under a limited optimization budget.


Response to W2 and Q1

W2. The bandit mechanism needs more detailed analysis.More comparisons with simpler selection strategies, such as greedy reward prediction or UCB without the neural predictor, would help justify the design. the predictor could be also a overfit iid model.

Q1. Could you compare with simpler selection methods and provide more detailed ablations of the bandit components?

In addition to the random-selection baseline already reported in Table 3, we added two component ablations: NN-Greedy, which selects by predicted reward alone, and UCB-only, which selects by the uncertainty bonus alone. We kept the evolution loop unchanged. Each new ablation is averaged over two independent runs on Qwen3.6-35B-A3B. Random and full COBRA-Skills results are from the original papers.

Selection ruleSearchQASpreadsheetDocVQALiveMathSocialMazeALFWorldAverage
NN-Greedy86.550.083.037.071.564.065.3
UCB-only (uncertainty bonus only)84.549.582.550.592.565.070.8
Random (w/o Bandit, Table 3)86.745.782.352.094.267.071.3
Full COBRA-Skills84.349.783.755.395.572.373.5

Full COBRA-Skills achieves the highest average held-out accuracy. Our objective is final skill quality, and the comparisons with UCB-only and NN-Greedy support the practical value of neural prediction and exploration, respectively. These results do not rule out predictor overfitting, but provide evidence that the predictor contributes useful guidance within the full algorithm. The 2.2-point improvement over random selection further supports the combined selection rule.

▼▼ Continued in Part 2/2. ▼▼

Response to Reviewer kCs5 (Part 2/2)

▲▲ Continued from Part 1/2. ▲▲

Response to Q2

Q2. Could you compare against skillopt under the same optimization data and computational budget?

Optimization data. We additionally evaluated SkillOpt-50 on Qwen3.6-35B-A3B using exactly the same 50 optimization examples per benchmark as COBRA-Skills. Specifically, we replaced SkillOpt’s separate training and development sets with this shared pool, using it for both trajectory-based refinement and candidate evaluation. Final performance was evaluated on the same independent held-out test set. Results for SkillOpt-50 are reported as mean scores over three trials.

MethodSearchQASpreadsheetDocVQALiveMathSocialMazeALFWorldAvg.
SkillOpt (original)86.343.781.042.391.772.369.6
SkillOpt-5085.739.380.345.089.073.068.7
COBRA-Skills84.349.783.755.395.572.373.5

Under this shared 50-example setting, SkillOpt’s average performance is slightly lower than its original configuration, potentially due to the use of fewer optimization examples, while COBRA-Skills retains an advantage of approximately 4.8 points.

Performance at comparable computational budgets. Exact cost alignment between completed steps is difficult because the methods have different update granularities: a SkillOpt step includes multiple rollout, reflection, aggregation, and evaluation operations, while COBRA-Skills evaluates candidates round by round with scheduled evolution. We therefore compare intermediate checkpoints against their actual cumulative optimization costs. In Figures 5–7 (Appendix B.4, “Checkpoint Results”), each checkpoint reports the held-out performance of the skill that the method would return if stopped at that point. Costs include both target-model and teaching-model usage under fixed prices. These curves show that COBRA-Skills generally achieves stronger performance at comparable cumulative costs across most benchmarks, supporting its efficiency beyond the final endpoint comparison.


Thank you again for your helpful suggestions. We hope our responses and additional results address your concerns and encourage a more favorable assessment of our paper.

Thanks for your response. I have increased my score from 2 to 3.

AuthorsOpenReview ↗

Thank You for Your Updated Assessment

Thank you very much for considering our clarifications and additional results and for updating your assessment. We appreciate your thoughtful review and constructive feedback.

Reviewer_ygqPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies skill optimization for LLM agents and formulates it as budgeted sequential optimization over a dynamically evolving candidate space. The proposed COBRA-Skills maintains a population of candidate skills. In each round, an MLP predicts each skill's reward from its text embedding, a LinUCB-style uncertainty bonus is added, and the top-priority skill is evaluated on the optimization set. The skills are then updated by a teacher model (GPT-5.5) through regeneration, rollout mutation, and crossover, and the lowest-scoring skills are removed. Across six benchmarks and three target models, COBRA-Skills improves over the no-skill baseline by 13.1, 26.9, and 22.5 points, outperforms SkillOpt by 3.9, 3.2, and 1.3 points, and reduces total cost by 55–58%.

Critical strengths
  1. Efficiency. On all three target models, total cost drops by 55–58% relative to SkillOpt, using only 50 optimization examples.
  2. Effective skill generation. A single trajectory grounded generation already improves over the no-skill baseline by 5.8, 18.3, and 8.8 points.
  3. Broad experiments. The evaluation covers six benchmarks, three target models, two external harnesses, and a self teaching setting, and the method remains effective across them.
Critical weaknesses
  1. Questionable bandit design. The budget used over the whole optimization process is nearly sufficient to evaluate every skill once (in the ablation). Moreover, the same skill may be selected repeatedly, even though re-evaluation provides little new information. In addition, the paper does not show whether the embedding-based priority score reflects skills' observed performance.
  2. Inconsistent comparison setup. The paper and the baselines start from different points: SkillOpt and Trace2Skill start from their own initial skills, while COBRA-Skills generates its initial skills with its own regeneration procedure. On Qwen, Best-of-30 using only this initialization already outperforms SkillOpt.
  3. Missing analytical experiments. Gemma rises from 9.5 to 88.5 on SocialMaze, and GPT-Nano from 30.0 to 70.7 on DocVQA; it is unclear where these substantial improvements come from. The paper also does not ablate the three evolution operators (regeneration, rollout mutation, crossover) individually, so their respective roles remain unknown.
Questions for authors
  1. Does the priority score assigned before evaluation (MLP predicted reward plus uncertainty bonus) actually reflect candidate skills’ performance? Some analytical experiments could be added.
  2. Under the schedule in Alg. 1, the total number of candidate skills encountered during optimization (about 30, see Sec. 5.1) is roughly equal to the 30 round evaluation budget. Since the budget can nearly evaluate every skill once, why is bandit scoring needed to choose which skill to evaluate?
  3. Alg. 1 scores all skills in the population each round, including already evaluated ones, and selects the highest-scoring one, so the same skill may be selected repeatedly, even though evaluating the same skill provides little new information. How many repeated selections actually occur within the 30 rounds, and what is the purpose of re-evaluation?
  4. On Qwen3.6-35B-A3B, Best-of-30 without the bandit or evolution (Table 3, 71.0) already outperforms SkillOpt (Table 1, 69.6). Could the advantage over SkillOpt mainly from COBRA’s initial skill generation? Could you rerun SkillOpt starting from COBRA’s initial skills?
5 comments

Response to Reviewer ygqP (Part 1/3)

We appreciate your thoughtful review and insightful suggestions for strengthening our work.


Response to W1, Q2, Q3

W1. Questionable bandit design. The budget used over the whole optimization process is nearly sufficient to evaluate every skill once (in the ablation). Moreover, the same skill may be selected repeatedly, even though re-evaluation provides little new information.
Q2. Under the schedule in Alg. 1, the total number of candidate skills encountered during optimization (about 30, see Sec. 5.1) is roughly equal to the 30 round evaluation budget. Since the budget can nearly evaluate every skill once, why is bandit scoring needed to choose which skill to evaluate?
Q3. Alg. 1 scores all skills in the population each round, including already evaluated ones, and selects the highest-scoring one, so the same skill may be selected repeatedly, even though evaluating the same skill provides little new information. How many repeated selections actually occur within the 30 rounds, and what is the purpose of re-evaluation?

Evaluation decisions shape the evolving candidate pool, while repeated evaluations follow standard contextual bandit practice, refining reward estimates and supplying feedback for skill evolution. Despite these repetitions, COBRA-Skills achieves the strongest average performance among the compared methods.

Q2: Why is bandit selection needed? Each run generates 28 candidates: 10 initial skills and 18 introduced progressively through evolution. They are not all available initially, and evaluation feedback guides subsequent mutation, crossover, and population retention. Standard contextual bandits allow repeated arm selection. Permitting skill revisits follows this classical formulation more closely and lets the bandit balance exploring new skills with refining estimates of promising ones within the 30-round budget.

Q3: How often are skills re-evaluated? Below are averages over the three main-experiment trials with Qwen3.6-35B-A3B, with each evaluation using the same 50-example optimization set.

BenchmarkDistinct skills evaluatedSkills evaluated ≥2 timesRepeat evaluationsMean score difference (pp)
SearchQA24.05.36.00.7
SpreadsheetBench13.05.717.05.0
DocVQA21.35.38.70.9
LiveMath9.36.320.75.4
SocialMaze12.76.317.32.7
ALFWorld21.05.09.04.2

Repeat evaluations exclude each skill’s first evaluation. Score difference is the mean pairwise absolute difference between evaluations of the same skill, averaged equally over repeatedly evaluated skills and then over trials. SocialMaze uses soft reward.

Why re-evaluation helps. Variability is larger on LiveMath, SpreadsheetBench, and ALFWorld. For example, two unchanged skills in one LiveMath run obtained:

SkillThree evaluation scores (%)Mean (%)
Skill A54, 60, 4653.3
Skill B62, 50, 6458.7

Both exhibit a 14-point range on the same optimization set, and their relative ranking changes: the second observations favor Skill A, while the first and third favor Skill B. Repeated observations help reduce sensitivity to individual rollout outcomes; our final selection uses each skill’s mean observed reward.

How repeated feedback is used. Each re-evaluation updates the reward predictor and exploration design matrix; its trajectories can inform mutation and its rewards can inform crossover at scheduled updates. The exploration bonus decreases for repeatedly evaluated skills, while high predicted reward can still favor their selection, reflecting the exploration–exploitation balance. This bonus reflects embedding coverage rather than directly estimating within-skill score variance.

Despite repeated evaluations, COBRA-Skills achieves the highest average performance among all compared methods for each of the three target models (Table 1). Complementarily, our selection ablation shows that replacing bandit selection with random selection while retaining evolution reduces average performance from 73.5 to 71.3, supporting the value of adaptive selection.

▼▼ Continued in Part 2/3. ▼▼

Response to Reviewer ygqP (Part 2/3)

▲▲ Continued from Part 1/3. ▲▲

Response to W2 and Q4

W2. Inconsistent comparison setup. The paper and the baselines start from different points: SkillOpt and Trace2Skill start from their own initial skills, while COBRA-Skills generates its initial skills with its own regeneration procedure. On Qwen, Best-of-30 using only this initialization already outperforms SkillOpt.
Q4. On Qwen3.6-35B-A3B, Best-of-30 without the bandit or evolution (Table 3, 71.0) already outperforms SkillOpt (Table 1, 69.6). Could the advantage over SkillOpt mainly from COBRA’s initial skill generation? Could you rerun SkillOpt starting from COBRA’s initial skills?

COBRA-Skills automatically generates its initial skills through regeneration, which is an integral component of our method. In the original comparison, we retained SkillOpt’s and Trace2Skill’s respective initialization procedures to preserve their original designs. The strong Best-of-30 result also indicates that our regeneration procedure can produce high-quality candidates, providing promising starting points for subsequent optimization.

To examine the contribution of initialization, we additionally ran SkillOpt with COBRA-generated initial skills. The following results are averaged over two runs for this new configuration:

MethodSearchQASpreadsheetDocVQALiveMathSocialMazeALFWorldAvg.
SkillOpt (original)86.343.781.042.391.772.369.6
SkillOpt + COBRA initialization87.047.581.545.092.371.070.7
COBRA-Skills84.349.783.755.395.572.373.5

COBRA initialization improves SkillOpt’s average performance by 1.1 points, confirming that initialization contributes to the gains. However, most of the original 3.9-point gap remains (2.8 points), suggesting that initialization is not the main explanation for COBRA-Skills’ advantage in this setting.


Response to W3 -Part1

W3. Missing analytical experiments. Gemma rises from 9.5 to 88.5 on SocialMaze, and GPT-Nano from 30.0 to 70.7 on DocVQA; it is unclear where these substantial improvements come from.

Trajectory analysis suggests two distinct task-specific bottlenecks. On SocialMaze, Gemma often conflates its stated identity with its true role, impairing both self-role and culprit identification. The optimized skill addresses this misunderstanding through explicit role disambiguation and global consistency checks.

On DocVQA, GPT-Nano frequently surrounds otherwise correct answers with explanatory text, whereas Qwen and Gemma more often return concise spans under the same baseline prompts. Generated skills encourage answer-span-only outputs, reducing errors under the strict matching criterion. This analysis points to improved answer-format compliance as a major contributor. Notably, the initial LLM Skill already achieves 70.7, matching the full method’s average score, indicating that most of the observed gain is already attained during initial skill generation.


Response to W3 -Part2

W3. The paper also does not ablate the three evolution operators (regeneration, rollout mutation, crossover) individually, so their respective roles remain unknown.

We additionally ablated each evolution operator individually on Qwen3.6-35B-A3B. Results for each ablation are averaged over two independent trials.

MethodSearchQASpreadsheetDocVQALiveMathSocialMazeALFWorldAvg.
w/o Regeneration85.548.082.049.095.566.071.0
w/o Rollout Mutation87.544.583.549.593.568.571.2
w/o Crossover87.050.083.050.095.571.072.8
COBRA-Skills84.349.783.755.395.572.373.5

Removing any operator reduces average performance. Regeneration and rollout mutation have larger effects, with decreases of 2.5 and 2.3 points, respectively, while removing crossover yields a smaller decrease of 0.7 points. These results support the contributions of all three operators, with regeneration and rollout mutation contributing more strongly in this setting.

▼▼ Continued in Part 3/3. ▼▼

Response to Reviewer ygqP (Part 3/3)

▲▲ Continued from Part 2/3. ▲▲

Response to W1 and Q1:

W1. In addition, the paper does not show whether the embedding-based priority score reflects skills' observed performance. Q1. Does the priority score assigned before evaluation (MLP predicted reward plus uncertainty bonus) actually reflect candidate skills’ performance? Some analytical experiments could be added.

We agree that analyzing candidate selection would help clarify the usefulness of the priority score. We therefore generated eight additional skills using regeneration for each benchmark and evaluated their rewards. These skills were excluded from the optimization population, and their rewards were not used for NN training, UCB updates, or evolution.

We then ran COBRA-Skills normally. At rounds 0, 5, 10, and 15, we selected two of the eight held-out skills using either uniform random selection, NN predictions alone, or the full NN+UCB priority. At round 0, all three methods use random selection with the same random seed, resulting in identical scores. Below, we report the actual average reward of the two selected skills, averaged over three trials.

BenchmarkRoundRandomNN-onlyNN+UCB
DocVQA080.380.380.3
DocVQA583.383.382.3
DocVQA1082.383.384.3
DocVQA1582.083.385.0
LiveMath035.035.035.0
LiveMath533.335.745.0
LiveMath1034.736.345.0
LiveMath1538.036.745.0

The identical round-0 scores reflect the shared random selection and seed. At round 5, the full priority performs slightly below random selection and the NN-only baselines on DocVQA, consistent with an early emphasis on exploration. The subsequent gains are consistent with a shift from exploration-dominated selection toward exploitation as NN predictions become more reliable: by round 15, the full priority selects skills scoring 3.0 points higher on DocVQA and 7.0 points higher on LiveMath than random selection. These results support the priority score’s usefulness for selecting candidates whose rewards have not been observed during optimization, complementing our selection-strategy ablations.


Thank you once again for your thoughtful review. We hope our clarifications and additional experiments resolve your concerns and support a more positive assessment of our paper.

Reviewer_RAYBOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes COBRA-Skills, which treats agent skill optimization as a budgeted search problem. It includes a contextual bandit that decides which candidate skills are worth evaluating, while evidence-grounded evolution. The authors evaluate across six benchmarks and three target models, it achieves the best average performance while cutting optimization cost by 55-58% versus SkillOpt and using only 50 optimization examples per benchmark.

Critical strengths
  • The proposed method is cost-efficient. In the result table 2, COBRA-Skills wins on cost per point of gain for all three models.

  • The proposed method generalizes across harnesses. The results hold under Claude Code and Codex harnesses, with self-teaching (73.5 to 72.5, at roughly half the cost), and in cross-model skill transfer (34 of 36 transfers beat the no-skill baseline).

Critical weaknesses
  • The cost win depends on the price ratio. Savings come almost from fewer teacher tokens (GPT-5.5), while COBRA uses more target-model tokens than SkillOpt on Qwen and Gemma. That is not the problem when targets are cheap open models, but the conclusion could shrink or flip if the target is expensive or the teacher is cheap.

  • The baselines chosen are limited. Only two skill-optimization baselines are compared, and Trace2Skill is weak. Related evolutionary methods the paper cites (EvoSkill, CoEvoSkills, SkillProx) are not compared. Baseline budgets were chosen by the authors rather than using official configs, and SkillOpt numbers are reproductions.

Questions for authors
  • How ν was chosen. Fig. 9 in Appendix that reports test-set averages across ν, and the peak (73.5) equals the headline result. Was ν selected using test performance?

  • Which bandit component matter? Does the gain come from the neural predictor, the LinearUCB bonus, or simply from the evolution loop plus any reasonable selection rule?

3 comments

Response to Reviewer RAYB (Part 1/2)

Thank you for your insightful comments and constructive suggestions.


W1. The cost win depends on the price ratio. Savings come almost from fewer teacher tokens (GPT-5.5), while COBRA uses more target-model tokens than SkillOpt on Qwen and Gemma. That is not the problem when targets are cheap open models, but the conclusion could shrink or flip if the target is expensive or the teacher is cheap.

We agree that monetary savings depend on the relative prices of the teacher and target models. Our reported savings are primarily driven by lower teaching-model token usage.

We also evaluate self-teaching, where both methods use Qwen3.6-35B-A3B for skill generation and task execution, with identical pricing for both roles (Section 5.2, lines 437–443). In this setting, COBRA-Skills still achieves lower optimization cost than SkillOpt (USD 27.5 vs. USD 32.4, a 15.1% reduction; Figure 4) while attaining higher average performance (72.5 vs. 68.4, Table 7). This shows that the observed cost advantage also holds without an expensive external teacher.

In practical skill optimization, using a more capable and typically more expensive teacher is a reasonable choice: it can provide stronger skill generation and refinement while keeping the target model inexpensive to deploy. This design is consistent with SkillOpt (Yang et al., 2026a) and related skill-learning frameworks such as SkillRL (Xia et al., 2026), which use stronger models to optimize or distill skills for smaller target models. The teacher is used during optimization, so its cost can be amortized over repeated deployment of the resulting skills.

These results support COBRA-Skills’ cost efficiency both with a stronger external teacher and under self-teaching. We will clarify that the reported savings are specific to the evaluated model and pricing configurations and do not imply a cost advantage under every teacher-to-target price ratio.


Response to W2 Part1

W2. The baselines chosen are limited. Only two skill-optimization baselines are compared, and Trace2Skill is weak. Related evolutionary methods the paper cites (EvoSkill, CoEvoSkills, SkillProx) are not compared.

We initially focused on Trace2Skill and SkillOpt because they directly address reusable skill construction and optimization from task-execution feedback, closely matching our task-level setting.

We agree that comparisons with additional evolutionary methods would strengthen the evaluation. Following your suggestion, we additionally evaluated EvoSkill with Qwen3.6-35B-A3B as the target model, using its default hyperparameters and conducting two independent runs.

MetricSearchQASpreadsheetDocVQALiveMathSocialMazeALFWorldAvg. / Total
Accuracy (%)83.543.081.048.091.065.568.7
Optimization cost ($)3.323.93.817.09.522.880.3

EvoSkill achieves an average accuracy of 68.7%, compared with 73.5% for COBRA-Skills, while incurring a higher optimization cost (USD 80.3 vs. USD 54.1). This additional comparison supports COBRA-Skills’ effectiveness and cost efficiency against another evolutionary skill-optimization baseline.


Response to W2 Part2

W2. Baseline budgets were chosen by the authors rather than using official configs, and SkillOpt numbers are reproductions.

We reduced the baseline data budgets because the original SkillOpt setting incurs substantial optimization cost (details on this cost are provided below).

Shared experimental setting. Our reproduction ensures fair comparisons by using exactly the same target models, teaching model, agent harness, benchmark evaluators, and held-out test sets for SkillOpt and COBRA-Skills. We reduced benchmark-specific data budgets while retaining SkillOpt’s released training–selection procedure (lines 788–797). The baselines still use 53–170 optimization examples, compared with 50 for COBRA-Skills, and SkillOpt’s optimization cost remains higher across all three models (Table 2).

▼▼ Continued in Part 2/2. ▼▼

Response to Reviewer RAYB (Part 2/2)

▲▲ Continued from Part 1/2. ▲▲

Response to W2 Part2 (continued)

New results: Same optimization data. We additionally evaluated SkillOpt-50 on Qwen3.6-35B-A3B using exactly the same 50 optimization examples per benchmark as COBRA-Skills, shared between trajectory-based refinement and candidate evaluation. The table reports mean scores (%) over three trials.

MethodSearchQASpreadsheetDocVQALiveMathSocialMazeALFWorldAvg.
SkillOpt (original)86.343.781.042.391.772.369.6
SkillOpt-5085.739.380.345.089.073.068.7
COBRA-Skills84.349.783.755.395.572.373.5

COBRA-Skills retains higher average performance under this shared data budget (73.5 vs. 68.7).

Comparable computational budgets. To address the concern about our choice of optimization budgets, we compare performance throughout optimization. We additionally provided checkpoint performance–cost curves (Figures 5–7; lines 947–953). Since optimization steps differ between methods, these curves compare held-out performance against actual cumulative cost, showing generally stronger performance for COBRA-Skills at comparable costs across most benchmarks.

Cost of the original setting. As discussed in Appendix A.2 (lines 806–809 and 840–848), running all released configurations across three target models and three independent trials would incur substantial cost. Specifically, SkillOpt reports 526.7M training tokens across six benchmarks with GPT-5.5 as both student and teacher, which is equivalent to at least $2,633.50 under our uncached pricing for each independent run.


Questions For Authors:

Q1. How ν was chosen. Fig. 9 in Appendix that reports test-set averages across ν, and the peak (73.5) equals the headline result. Was ν selected using test performance?

ν was selected using optimization-set rewards, not test-set performance. We initially set ν = 0.1 heuristically and compared it with ν = 0.05 and 0.3 in preliminary LiveMath experiments. Both alternatives achieved lower optimization-set rewards within the pilot budget, so we retained ν = 0.1 and kept it fixed across subsequent main experiments without task-specific retuning.

Figure 9 was produced after ν = 0.1 had been fixed, as a sensitivity analysis; its 73.5 point reuses the main-experiment runs. It evaluates ν ∈ {0.03, 0.06, 0.1, 0.2, 0.3}, a different grid from the preliminary LiveMath pilot {0.05, 0.1, 0.3}. We will revise Appendix C.2 to make this chronology explicit and clarify that the test-performance peak was not used to select ν.

We set λ = 0.03 heuristically without tuning. As an initial scale check, with unit-normalized skill embeddings, ν = 0.1 gives an initial exploration bonus of ν/λ≈0.58ν/\sqrt{λ}\approx0.58, comparable in scale to rewards normalized to [0,1][0,1].


Response to Q2

Q2. Which bandit component matter? Does the gain come from the neural predictor, the LinearUCB bonus, or simply from the evolution loop plus any reasonable selection rule?

Following your suggestion, we added NN-only greedy and LinearUCB bonus only variants alongside the random-selection ablation already reported in Table 3, keeping the evolution loop unchanged. The two new variants are averaged over two independent runs on Qwen3.6-35B-A3B; the results in the last two rows are from the original paper.

Selection strategySearchQASpreadsheetDocVQALiveMathSocialMazeALFWorldAvg.
NN-only greedy86.550.083.037.071.564.065.3
LinearUCB bonus only84.549.582.550.592.565.070.8
Random (w/o Bandit, Table 3)86.745.782.352.094.267.071.3
COBRA-Skills (NN + LinearUCB bonus)84.349.783.755.395.572.373.5

Neither the neural predictor nor the LinearUCB bonus alone outperforms random selection on average, whereas their combination achieves the highest average performance among the tested selection rules. This supports the benefit of combining reward prediction with exploration. The 2.2-point improvement over random selection, which retains the evolution loop, further suggests that adaptive candidate selection contributes beyond evolution alone.


Thank you again for your constructive feedback. We hope our clarifications and additional results address your concerns and lead to a more favorable evaluation of our paper.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal

Dear Reviewer RAYB,

Thank you again for your review. We would appreciate your consideration of our rebuttal in Parts 1/2 and 2/2, which addresses your concerns through the following clarifications and additional results:

  • Cost savings: Self-teaching achieves 15.1% lower optimization cost than SkillOpt while attaining higher performance, demonstrating savings without an expensive external teacher.
  • Additional baseline: We added EvoSkill, which achieves 68.7 average performance versus 73.5 for COBRA-Skills, at higher optimization cost.
  • Comparison fairness: We clarified the baseline budget choices and added SkillOpt-50 using the same 50 optimization examples. COBRA-Skills retains a 4.8-point average advantage. We also discuss comparisons at comparable cumulative costs.
  • Hyperparameter selection: We clarified that ν was selected using optimization-set rewards before the Figure 9 sensitivity analysis, without using test performance.
  • Bandit components: We added NN-only and exploration-bonus-only ablations. Their combination performs best on average and exceeds random selection by 2.2 points while retaining the same evolution loop.

We hope these responses address your concerns and would welcome any further questions. Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.