- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces SelfEvoSkill, an iterative framework that revises LLM agent skills without external outcome supervision (such as task rewards or benchmark verifier feedback) by auditing behavioral differences between matched execution pairs.
Critical strengths
- Proposes an effective skill-revision framework that operates without external outcome supervision (such as task rewards or benchmark verifier feedback), solving the challenge of high-cost or missing evaluation signals.
- Outperforms no-skill (+33.0 pp) and curated-skill (+17.2 pp) baselines across 89 diverse tasks in 8 domains, with ablations confirming the necessity of paired references and multi-round iterative audits.
Critical weaknesses
- The evaluation is restricted to a single benchmark (SkillsBench, 89 tasks), leaving its generalizability to more diverse agent environments unverified.
- The method relies on benchmark-provided, human-curated skills as the initial baseline (). In practice, high-quality expert-crafted skills are scarce and expensive to obtain, and these curated skills already provide strong structural priors. The framework does not evaluate starting from low-quality, noisy, or automatically synthesized skills, limiting its applicability in cold-start scenarios where human-curated skills are unavailable.
- The LLM testing in the method is very limited. It is suggested to test more different LLMs from different companies and test whether the skill can be effectively transferred to other agents or LLMs.
Questions for authors
- How does the method locate exactly which paragraph, or which part of the skill, is harmful? How do you guarantee the attribution is correct?