- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Lifetime Skill-Learning (LSL), a framework that evolves not only agent skills but also the skill-learning algorithm itself while keeping the foundation model and host agent frozen, which comprises two levels: 1) Inner Loop: an executable lifecycle meta-program controls trajectory selection, failure diagnosis, edit-operator selection, candidate generation, budget allocation, validation, compression, deployment, and rollback to construct and refine domain skills, while recording a replayable provenance graph; 2) Outer Loop: a population-based program-evolution procedure mutates these meta-programs and selects them with a fitness that combines cross-task regret, anytime performance, cost, transfer, safety violations, and program complexity. Experiments with a frozen Qwen2.5-7B-Instruct on seven domains, four for meta-training and three unseen ones for meta-testing, show that the evolved lifecycle outperforms fixed refinement loops, expert-designed policies, a single-prompt meta-skill, and population-based training, and ablations examine the contributions of adaptive budgeting, operator selection, rollback, and multi-domain meta-training.
Critical strengths
- Treating the skill-learning procedure itself as an optimizable object is a reasonable direction, and the emphasis on versioning, provenance, and rollback makes the learned skills auditable.
- The evaluation adopts a cross-domain meta-train/meta-test split and reports multiple metrics beyond final success, including anytime performance, cost, and safety-related statistics.
Critical weaknesses
-
Limited novelty. The core motivation, evolving the skill-learning procedure rather than only the skill, has already been proposed: [1] jointly evolves skills and the skill-evolving strategy via meta-skills, [2] evolves meta-skills that govern diagnosis, budget allocation, and editing on a slow loop, and [3] (cited but not compared) frames skill learning as a meta-skill. Moreover, the inner loop (diagnosis, bounded edits, validation gate, rollback) closely mirrors [5], and the outer loop is a standard population-based program search. The only difference, representing the lifecycle as a program rather than a textual meta-skill, is not shown to be beneficial.
-
The method is overly abstract, and the key artifact is never shown. 1) No evolved lifecycle program is presented anywhere, so it is unclear what was actually learned at the meta level; 2) the program DSL, mutation operators, and initial population are only described in words, and , , , , and are not formally defined; 3) the fitness term penalizes transfer, contradicting the stated goal, and the leftover "% Revised LaTeX code" in Section 3.5 suggests an unfinished manuscript.
-
The benchmark is self-constructed and unverifiable. All seven domains are built by the authors without sources, task examples, or release plans, so the results cannot be compared with any published number. Public benchmarks used by prior skill-evolution work, such as SkillsBench, SpreadsheetBench, OfficeQA, and ALFWorld, should be adopted.
-
The baselines are weak and all author-designed. None of the closely related methods [1, 2, 3, 5, 6, 7] or optimizers such as GEPA [8] and ADAS is compared. ExpertPolicy is not specified beyond "designed by experienced agent developers", and there is no no-skill reference to show the absolute benefit of skill learning.
-
The reported budgets and protocols are internally inconsistent. 1) Each method is said to use at most 1,200 meta-training executions, yet 60 generations of successive halving over 8/24/60 instances require about 16,800 evaluations; 2) Appendix D.1 says no LLM judge is used, while Appendices B.2–B.3 use the backbone itself as the judge for web synthesis; 3) Table 3 omits GreedyProgram, and LIFETIME's utility is lower than three baselines, contradicting the efficiency claim; 4) the alternative model in Table 4 is never named. Releasing code and raw logs is necessary to verify the results.
-
A single outdated backbone plays every role. Qwen2.5-7B-Instruct serves as executor, diagnoser, skill writer, program mutator, and judge. It is doubtful that a 7B model can reliably mutate programs, and the conclusions may not hold for current models, given that [5] shows the gains of skill optimization vary substantially across models.
[1] You Live More Than Once: Towards Hierarchical Skill Meta-Evolving. arXiv:2605.28390, 2026.
[2] MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution. arXiv:2607.05297, 2026.
[3] SkillEvolver: Skill Learning as a Meta-Skill. arXiv:2605.10500, 2026.
[4] Learning to Evolve: A Self-Improving Framework for Multi-Agent Systems via Textual Parameter Graph Optimization. Findings of ACL, 2026.
[5] SkillOpt: Executive Strategy for Self-Evolving Agent Skills. arXiv:2605.23904, 2026.
[6] EvoSkill: Automated Skill Discovery for Multi-Agent Systems. arXiv:2603.02766, 2026.
[7] Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills. arXiv:2603.25158, 2026.
[8] GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning. arXiv:2507.19457, 2025.
Questions for authors
-
Could you show the final selected program and contrast it with ExpertPolicy? How similar are the programs selected across the five seeds, and what fraction of mutations are rejected by static checks before evaluation?
-
How exactly are the "1,200 meta-training executions" counted? Do they include all inner-loop executions triggered by successive halving over 60 generations, and does PBT receive the same total number of environment interactions and model calls?
-
If the executable program is replaced by a textual meta-skill or an optimization-experience memory, as in [1, 2, 4], and evolved under the same budget, how does it perform? This comparison is necessary to justify the program representation.