- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
this paper proposes SkillEvolver, which lets agents create and improve skills through a few rounds of trial and error. it generates different strategies, compares successful and failed attempts, and updates skills based on the results. experiments on SkillsBench show improvements over human-written skills across three models.
Critical strengths
the experimental improvements over human-written skills are interesting, and the strategy diversification ablation shows some useful results
Critical weaknesses
- i find the novelty quite weak. generating skills from execution traces, comparing successes and failures, and refining skills through repeated testing have already been explored by SkillGen, EvoSkill, Anthropic's skill-creator and other recent work. the main difference here seems to be asking agents to try different strategies on a single task. i dont see enough new ideas to justify another framework built around essentially the same loop.
- more importantly, the paper doesnt compare against any of these closely related methods. beating static human-written skills says very little about whether SkillEvolver is better than existing automated skill creation methods. the authors argue that prior methods use different settings, but this is not a convincing reason to skip the most important comparisons. also, the claimed single-task setting still requires an LLM-generated training variant with verifier feedback, which makes the practical motivation less convincing.
Questions for authors
- what exactly is the technical novelty compared with SkillGen, EvoSkill and Anthropic's skill-creator, beyond strategy diversification and the single-task setting?
- why not adapt these existing methods to the same trial budget? without such comparisons, how can we tell whether the proposed method actually offers anything new?