- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a framework for optimizing agent skills using contextual bandits and evolutionary search. It uses a mlp with a LinearUCB bonus to select promising skills, and periodically generates or refines skills based on execution feedback. Experiments on six benchmarks and three models show improvements over existing skill optimization methods, with lower optimization costs.
Critical strengths
-
The method achieves strong average performance across six benchmarks and three models, while reducing optimization costs compared with SkillOpt.
-
The paper includes component ablations, self-teaching experiments, and evaluations under different agent harnesses.
Critical weaknesses
- Similar bandit-based optimization ideas have also been explored in prompt optimization.
- The bandit mechanism needs more detailed analysis.More comparisons with simpler selection strategies, such as greedy reward prediction or UCB without the neural predictor, would help justify the design. the predictor could be also a overfit iid model.
Questions for authors
-
Could you compare with simpler selection methods and provide more detailed ablations of the bandit components?
-
Could you compare against skilopt under the same optimization data and computational budget?
4 comments
Response to Reviewer kCs5 (Part 2/2)
▲▲ Continued from Part 1/2. ▲▲
Response to Q2
Q2. Could you compare against skillopt under the same optimization data and computational budget?
Optimization data. We additionally evaluated SkillOpt-50 on Qwen3.6-35B-A3B using exactly the same 50 optimization examples per benchmark as COBRA-Skills. Specifically, we replaced SkillOpt’s separate training and development sets with this shared pool, using it for both trajectory-based refinement and candidate evaluation. Final performance was evaluated on the same independent held-out test set. Results for SkillOpt-50 are reported as mean scores over three trials.
| Method | SearchQA | Spreadsheet | DocVQA | LiveMath | SocialMaze | ALFWorld | Avg. |
|---|---|---|---|---|---|---|---|
| SkillOpt (original) | 86.3 | 43.7 | 81.0 | 42.3 | 91.7 | 72.3 | 69.6 |
| SkillOpt-50 | 85.7 | 39.3 | 80.3 | 45.0 | 89.0 | 73.0 | 68.7 |
| COBRA-Skills | 84.3 | 49.7 | 83.7 | 55.3 | 95.5 | 72.3 | 73.5 |
Under this shared 50-example setting, SkillOpt’s average performance is slightly lower than its original configuration, potentially due to the use of fewer optimization examples, while COBRA-Skills retains an advantage of approximately 4.8 points.
Performance at comparable computational budgets. Exact cost alignment between completed steps is difficult because the methods have different update granularities: a SkillOpt step includes multiple rollout, reflection, aggregation, and evaluation operations, while COBRA-Skills evaluates candidates round by round with scheduled evolution. We therefore compare intermediate checkpoints against their actual cumulative optimization costs. In Figures 5–7 (Appendix B.4, “Checkpoint Results”), each checkpoint reports the held-out performance of the skill that the method would return if stopped at that point. Costs include both target-model and teaching-model usage under fixed prices. These curves show that COBRA-Skills generally achieves stronger performance at comparable cumulative costs across most benchmarks, supporting its efficiency beyond the final endpoint comparison.
Thank you again for your helpful suggestions. We hope our responses and additional results address your concerns and encourage a more favorable assessment of our paper.
Thanks for your response. I have increased my score from 2 to 3.
Thank You for Your Updated Assessment
Thank you very much for considering our clarifications and additional results and for updating your assessment. We appreciate your thoughtful review and constructive feedback.
Response to Reviewer kCs5 (Part 1/2)
Thank you for your careful review and insightful feedback.
Response to W1
We agree that bandit-based optimization has precedents in prompt optimization. In our setting, skill optimization requires generating and refining reusable task procedures from execution trajectories, including those involving tool use and environment interaction, such as spreadsheet manipulation. Both executing candidate skills for evaluation and generating or refining them with a teaching model can be costly. Our contribution is to couple bandit-guided evaluation allocation with evidence-grounded skill evolution: the bandit prioritizes which skills to execute, and the resulting feedback is reused for subsequent refinement and population updates. This coupling supports adaptive search over an evolving space of task-level skills under a limited optimization budget.
Response to W2 and Q1
In addition to the random-selection baseline already reported in Table 3, we added two component ablations: NN-Greedy, which selects by predicted reward alone, and UCB-only, which selects by the uncertainty bonus alone. We kept the evolution loop unchanged. Each new ablation is averaged over two independent runs on Qwen3.6-35B-A3B. Random and full COBRA-Skills results are from the original papers.
Full COBRA-Skills achieves the highest average held-out accuracy. Our objective is final skill quality, and the comparisons with UCB-only and NN-Greedy support the practical value of neural prediction and exploration, respectively. These results do not rule out predictor overfitting, but provide evidence that the predictor contributes useful guidance within the full algorithm. The 2.2-point improvement over random selection further supports the combined selection rule.
▼▼ Continued in Part 2/2. ▼▼