- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper proposes EvoGenome, a framework that represents agent skills as typed modules for planning, tool use, execution, verification, and recovery. It combines constrained mutation, crossover, and module transfer with trace- and lineage-based credit assignment to guide skill evolution. The authors report improved task success and executability, with fewer regressions, on SkillsBench and a dynamic tool-use environment.
Critical strengths
The paper studies a practically relevant problem: interference between edits and difficulties in reusing skill modules. Its proposed decomposition of planning, verification, and recovery into independently optimizable modules is a potentially useful design choice.
Critical weaknesses
- Missing strong baselines: The paper lacks direct comparisons with GEPA, EvoSkill, and other closely related methods, making its advantages over existing approaches difficult to establish.
- Weak credit assignment: Allocating overall gains based on module activation and verifier evidence does not isolate individual contributions when multiple modules are modified jointly.
- Questionable search strategy: Prioritizing edits to high-credit modules assumes that past contributions predict further improvement, an assumption the paper does not adequately justify.
- Poor writing and presentation: The figures, in particular, look templated and hastily assembled, giving the paper a pronounced “vibe-coded” feel rather than clearly communicating the method.
- Inconsistent experimental protocol: Unresolved inconsistencies in data splits, sampling sizes, and budget accounting undermine the credibility and reproducibility of the results.
Questions for authors
- Why are GEPA, EvoSkill, and other strong baselines missing? How do you rule out localized editing and additional checks as the primary sources of improvement?
- When multiple modules are modified jointly, how does your credit assignment identify harmful edits rather than incorrectly reward them for an overall gain?
- Why should high-credit modules be prioritized for further modification rather than preserved? What experimental evidence supports this choice?
- If the evolution split contains only 20% of the development tasks, how can you sample 32 tasks without replacement? Please provide the dataset version, split sizes, and task lists.
- What are the separate budgets for generated candidates, actual environment executions, and model calls? Why do the failure rates and reductions reported in Table 11 and the main text not align?