- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper keeps the LLM frozen and instead evolves an external library of natural-language skills for a graphic design agent. The library grows by adding skills for recurring uncovered subtasks, and improves by revising existing skills based on their failures and successes. A replay gate decides which changes are kept. Experiments show better execution success and positive win rates over a no-skill agent on several benchmarks. The ablation suggests that adding and revising skills work better together than alone.
Critical strengths
- The setting is realistic and hard: long-horizon tool use, editable outputs, and no reliable success oracle.
- The approach is practical. It needs no weight updates, adds little overhead, and the paper reports negative results honestly.
Critical weaknesses
- Limited novelty. The main ideas are close to existing skill and memory evolution work (e.g., ACE, MACLA, SkillAudit). The more distinctive parts are not well validated. The replay gate is never ablated. The ablation behind the "combination" claim also seems confounded, because the full system revises the newly added skills, while the "new skills only" arm does not.
- Moderate and hard-to-interpret gains. The same automatic judge drives evolution and evaluates the internal results, and there is no human evaluation. The improvement is mainly in task completion and execution success, while aesthetic quality barely changes. It is also unclear how failed runs are handled in some comparisons.
Questions for authors
- How does the system perform without the replay gate, or with a simpler acceptance rule?
- Do the main results hold with an independent judge, or with human evaluation?