- Rating
- 3(+1)
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SKILL2, a method that improves LLM agent skill generation by continuously optimizing a meta-skill. Instead of directly optimizing the generated Skills, the method focuses on improving the skill generation procedure itself. It updates the MetaSkill based on execution feedback, and uses historical rules to detect conflicts and validate new updates.
they evaluate the method on four benchmarks with two different executor model sizes. The results show some improvements over existing skill generation baselines.
Critical strengths
- Fairly comprehensive experimental evaluation.
they have a relatively full experiments, and compare it with several existing skill generation methods. The improvements are relatively consistent across different settings, which suggests that the method has some practical value.
- Useful analysis of the meta-skill update process.
Besides comparing the final performance, they also study how performance changes under a fixed trajectory setting, whether the learned meta-skill can transfer to unseen trajectories, and the effect of interference resolution.
Critical weaknesses
-
Limited novelty The main idea of optimizing meta-level instructions based on execution feedback seems like to be explored in works such as PromptBreeder, MetaSkill-Evolve. While skill2 focuses on skill generation, its algorithmic difference from these methods is not very clear. In particular, the actions like SELECT, REPRESENT, SCOPE, and COMPOSE seem more like structured prompt engineering rather than a new learning mechanism.
-
Missing important ablation comparisons. Although the paper compares with TextGrad and GEPA, these baselines mainly optimize the generated Skills rather than the same MetaSkill representation. Without a comparison under the same MetaSkill representation, feedback data, and computational budget, it is hard to tell where the improvements actually come from.
Questions for authors
-
Compared with MetaSkill-Evolve and other meta-level optimization methods, what is the main algorithmic novelty of skill2, beyond changing the optimization target to skill generation? I understand for the current stage, research ideas related to skills are like making the engineering greater.
-
Could you compare against GEPA or TextGrad directly optimizing the same MetaSkill, under the same feedback data and computational budget?
-
Could you provide longer-term experiments with continuously arriving tasks and trajectories to better support the claims about continual learning and generalization? [like a dynamic optimization process]
7 comments
Response to Reviewer fV5M (2/4)
2. How does SKILL² differ from existing meta-level methods?
Our intuition is that better generation guidance should produce a better new skill from the same trajectory evidence. SKILL² therefore removes the direct inheritance of previous skills while retaining execution experience through MetaSkill updates.
-
Fixed-evidence regeneration removes direct dependence on the previous skill.
- At round , SKILL² generates a skill as , where is the fixed trajectory pool and is the current MetaSkill.
- The generator is not directly given the previous skill or its newly collected execution trajectories. Each skill is generated afresh from the same trajectory evidence, rather than obtained by continually editing the previous skill.
- Thus, given the current MetaSkill and fixed trajectory pool, skill generation does not directly depend on the previous skill-editing path.
-
Execution feedback improves the MetaSkill, allowing the system to learn across fresh generations.
- After executing the current skill, the updater analyzes the skill and its execution feedback to identify a generation decision that may need to change.
- The MetaSkill represents these decisions through four editable components: SELECT chooses trajectory evidence, REPRESENT turns it into reusable rules, SCOPE specifies when the rules apply, and COMPOSE coordinates them.
- Once accepted, an edit remains in the MetaSkill and guides subsequent skill generation. Thus, execution experience can improve future skills through the MetaSkill without directly carrying the previous skill forward.
The contribution is not the textual structure itself, but the procedure that uses execution feedback to update a specific generation decision and tests the update through a newly generated skill.
- We agree that structured textual workflows are not new: ReAct organizes reasoning and actions [6], while Self-Refine organizes feedback and refinement [7]. We do not claim novelty from the four component names themselves.
- In SKILL², the components provide localized targets for feedback-driven updates. Each proposal specifies a generation decision to change, the intended effect on the generated skill, and the behavior to preserve.
- The resolver checks the proposal against previously validated principles and their applicability conditions. A candidate MetaSkill then generates a fresh skill from the same trajectory pool.
- The update is accepted only if the intended change is verified, validation utility improves, and regressions remain within the allowed limit: , , and .
For example, in a recorded ALFWorld cooling task, the updater proposed a COMPOSE edit that introduced execution-stage routing to coordinate existing rules. The regenerated skill implemented this change and improved paired validation from 6/15 to 8/15, with two additional successes and no regressions (Table R3).
In this way, SKILL² learns from execution experience through the MetaSkill while evaluating each update through a freshly generated skill.
3. How does SKILL² perform against other methods?
-
SKILL² outperformed GEPA and TextGrad when all three optimized the same MetaSkill. We adapted GEPA and TextGrad to the same four-component MetaSkill. The three methods shared the initialization, trajectory evidence, feedback access, verifier, acceptance rule, evaluation pipeline, and computational budget. Across the six settings in Table R4, SKILL² exceeded the stronger MetaSkill baseline by 3.15–22.27%.
Table R4 also reports our PromptBreeder adaptation and MetaSkill-Evolve reimplementation. For example, on ALFWorld with Qwen3-8B, SKILL² achieved 75.55%, compared with 46.72% for PromptBreeder and 49.27% for MetaSkill-Evolve.
-
The evolved MetaSkill generated stronger skills from unseen trajectory evidence. In a separate ALFWorld transfer experiment, both the initial and evolved MetaSkills generated skills from the same unseen target trajectories. The evolved MetaSkill increased test success from 42.70% to 65.82%.
In the main Qwen3-8B settings, the static initial MetaSkill scored below No-Skill on all four benchmarks, whereas SKILL² exceeded No-Skill by 11.92–33.82% (Table R5). Together, these results show that the evolved MetaSkill retains useful guidance for generating new skills.
We thank the reviewer again for the valuable feedback, which helped us clarify the distinction between the MetaSkill representation and the procedure used to improve it.
Response to Reviewer fV5M (3/4)
(W2, Q2) GEPA and TextGrad on the same MetaSkill
We thank the reviewer for suggesting a direct comparison with GEPA and TextGrad under the same MetaSkill representation, feedback, and computational budget.
Under the matched MetaSkill setting, SKILL² outperformed GEPA and TextGrad in all six evaluated settings (Table R4).
-
SKILL² outperformed both optimizers under the shared MetaSkill protocol. The three methods used the same initialization, four-component representation, trajectory pool, access to execution feedback, verifier, acceptance rule, computational budget, and downstream evaluation protocol. Each candidate MetaSkill was evaluated through a freshly generated skill. Across the six settings in Table R4, SKILL² led the stronger MetaSkill baseline by 3.15–22.27%.
-
Moving the optimization target to the MetaSkill improved both optimizers in all six settings. Relative to their original skill-level results in Table 1, TextGrad gained 0.36–10.21% and GEPA gained 2.07–10.50%.
-
SKILL² also exceeded both reimplemented meta-level baselines across the six settings. On ALFWorld with Qwen3-8B, SKILL² scored 75.55%, compared with 46.72% for the PromptBreeder adaptation and 49.27% for MetaSkill-Evolve, using the same three seeds and 274 held-out games.
(Q3) Can the MetaSkill adapt as new tasks and trajectories arrive?
We appreciate the reviewer’s suggestion to evaluate continual adaptation as new tasks and trajectories arrive over a longer sequence. This helps us examine continual adaptation beyond the fixed-trajectory setting in the original experiments.
Continuing to update the MetaSkill outperformed freezing or restarting it at both benchmark shifts in our long-horizon transfer task.
-
Our three-benchmark task tests adaptation as new tasks and trajectories arrive. We designed the sequence WTQ WikiSQL TabFact to evaluate the MetaSkill across two successive shifts. Continual carries forward the MetaSkill and its validated history and continues updating at each stage. Frozen keeps the WTQ-trained MetaSkill fixed, and Restart starts each new stage from . We evaluate each incoming benchmark and revisit earlier benchmarks using fixed probes.
-
Continual achieved the highest score on both incoming benchmarks. It reached 81.11% on WikiSQL and 81.12% on TabFact, outperforming Frozen by 4.64% and 3.42%, and Restart by 3.34% and 0.84%, respectively (Table R6).
-
Continual largely retained performance on earlier benchmarks across the two shifts. By the final stage, its scores on WTQ and WikiSQL were 0.62% and 0.53% below their respective post-learning scores (Table R6).
References
[1] Trace2Skill: Distill Trajectory-Local Lessons into Transferable Agent Skills. EMNLP 2026.
[2] Reflection in the Dark: Exposing and Escaping the Black Box in Reflective Prompt Optimization. ACL 2026 Student Research Workshop.
[3] MetaSkill-Evolve: Recursive Self-Improvement of LLM Agents via Two-Timescale Meta-Skill Evolution. arXiv preprint, 2026.
[4] Meta Context Engineering via Agentic Skill Evolution. ICML 2026.
[5] Promptbreeder: Self-Referential Self-Improvement via Prompt Evolution. ICML 2024.
[6] ReAct: Synergizing Reasoning and Acting in Language Models. ICLR 2023.
[7] Self-Refine: Iterative Refinement with Self-Feedback. NeurIPS 2023.
Experimental Tables
Table R1. Percentage of generated skills containing patches for previously observed execution errors, across four benchmarks and three seeds.
| Method | Skills containing ad-hoc patches |
|---|---|
| PromptBreeder | 72.2% |
| MetaSkill-Evolve | 78.9% |
| SKILL² | 16.4% |
Table R2. Effects of meta-level guidance and inner-loop optimization on ALFWorld with Qwen3-8B.
| Method | Initial meta guidance | Evolved meta guidance | Fixed meta + inner-loop optimization | Full method |
|---|---|---|---|---|
| PromptBreeder | 43.07% | 43.80% | 44.16% | 46.72% |
| MetaSkill-Evolve | 43.80% | 44.53% | 47.45% | 49.27% |
Table R3. Paired validation of the recorded COMPOSE update.
| Incumbent successes | Candidate successes | Failure success | Success failure |
|---|---|---|---|
| 6/15 | 8/15 | 2 | 0 |
Response to Reviewer fV5M (1/4)
We thank you for your thorough review and valuable comments on the algorithmic contribution, baseline comparisons, and continual adaptation. We provide our point-by-point responses below. If you have any remaining concerns or questions, please let us know; we would be happy to discuss them further.
(W1, Q1) Algorithmic contribution and the role of the four components
We thank the reviewer for asking us to clarify the algorithmic contribution beyond changing the optimization target. This question helped us better explain both the motivation behind SKILL² and the role of its four components.
SKILL² learns how to generate new skills by updating a persistent MetaSkill from execution feedback. Its contribution lies in the complete procedure for attributing, resolving, and validating updates to generation decisions, rather than in the component names alone.
Below, we explain what motivates SKILL², how it differs from existing meta-level methods, and what evidence supports its contribution.
1. What motivates SKILL²?
Execution feedback must be interpreted before it can guide a change to the generation procedure. Feedback is observed when a generated skill is executed, but it does not directly identify which generation decision should change. Existing work illustrates this difficulty:
These observations motivate two requirements for improving generation guidance: identify the generation decision associated with the feedback, and check the effect of a proposed change on existing behavior.
The performance of an evolved artifact can reflect both meta-level guidance and accumulated inner-loop history. PromptBreeder evolves task prompts and mutation prompts [5]. MetaSkill-Evolve uses a meta-skill to guide successive edits to an existing task skill [3]. MCE co-evolves context-engineering skills and context artifacts using their execution and evaluation history [4].
For an inner loop that directly uses a previous skill and its execution experience, a representative update can be written as Sti+1=G(Sti,Rti,Fti;Mt), where Mt is the meta-level guidance, Sti is the current skill, and Rti and Fti are its execution trajectory and feedback.
In this setting, the next skill can reflect both the MetaSkill and the previous skill and execution history. Consequently, an improvement in the resulting skill does not by itself identify how much comes from better meta-level guidance.
Generated skills can contain instructions tied to previously observed failures. We define an ad-hoc patch as an instruction in a generated skill that addresses an execution error observed in an earlier trajectory. Although such patches can address observed failures, patches tied too closely to specific cases may not improve performance on unseen tasks. In our experiments, skills generated by PromptBreeder and MetaSkill-Evolve contain such patches more frequently than those generated by SKILL² (Table R1), while SKILL² achieves higher held-out performance on ALFWorld with Qwen3-8B (Table R4).
Final skill performance alone does not isolate the contribution of meta-level guidance.