- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Agent skills are a simple/lightweight way to adapt the capabilities of LLM agents by encoding reusable procedures as instructions and workflows, without updating the model's underlying parameters. The authors note that one issue with these skills is that they are often handwritten or distilled from execution trajectories. This static approach to fixed skills means they don’t generalize well, are not malleable enough to accommodate updates for new tasks or failure cases, and need to be rewritten across sessions to prevent skill errors in future tasks. Previous work targets validation gates (held-out and replay) to evaluate and filter candidate skills that don’t meet predefined criteria, but these gates are expensive and impose a substantial cost during skill evolution. The authors propose FreeEvo as a gate-free self-evolution framework in which a skill is first decomposed into modular skill atoms. Accuracy atoms and fixed-efficiency atoms are thereby used to encode task-specific procedures and avoid redundant reasoning. They show that across three models and four benchmarks, FreeEvo matches or beats the performance of SkillOpt and GRASP while also reducing evolution tokens and test-time output tokens by a large percentage. Interestingly, the skills evolved by one model or dataset also show signs of transfer to others.
Critical strengths
-
The authors identify an existing bottleneck in the process of skill evolution (namely, costly gates) and propose an interesting framework to address it. FreeEvo is shown to work well on a set of models and across a variety of benchmarks spanning different domains. The results are consistent across models/datasets.
-
The cross-dataset transfer results are also consistent and are shown to reduce output tokens while improving accuracy.
-
The authors disclose the use of AI tools in the writing of the paper. As is, there was an obvious effort to make the ideas and results human-readable.
Critical weaknesses
W1: Appendix C.1.2 appears to have placeholders “[XX]” for the execution settings and the optimizer configurations. Presumably, these were meant to be filled out and are critical details for reproducibility.
W2: Some of the performance ranges noted in the abstract appear to be inconsistent with downstream results. For instance, the abstract notes “at test time, the evolved agent improves task performance by up to 23.1 percentage points while using 6.4%–77.9% fewer output tokens.”, but this appears to 1) understate the max gain (Table 1, FLM ALFWorld shows as 32.3), and 2) slightly overstate the lower bound on token savings (where DeepSeek and GLM TriviaQA results show -2.8 and -3.3%, respectively). This is fairly minor, but should be corrected, as it also appears verbatim in the introduction (L103-104).
W3: Some close alternatives mentioned in the appendix are ACE and gate-free SkillGrad. These are not used as baselines in the main paper, however.
W4: Extending W3, some of the main mechanisms included in the FreeEvo framework are similar to ACE. For instance, the skill-atom representation. The efficiency attribution is specific to FreeEvo, but it is questionable that no reference to ACE is made until the appendix.
Questions for authors
-
Can the authors run an experiment to show how much of FreeEvo’s gains come from the fixed efficiency atom alone? The ablations don’t appear to cover this case, but it would help to isolate the value of the FreeEvo framework.
-
Could the authors clarify which model is used as the optimizer in the main experiments? Pursuant to W1, this is not mentioned in the appendix, but is a rather critical detail. GPT4,4 is mentioned in passing, but it is uncertain what experiments can be attributed to which optimizer model.
-
Could the authors include the baselines mentioned in W3 and compare them against FreeEvo?
Presentation/Other:
-
I commend the authors for their clear visuals and tables, which are mostly self-contained and accompanied by digestible captions. Table 1 could benefit from a brief explanation of what the different columns (Evo and Test) mean.
-
Figure 4 has a very soft font woven in that can be difficult to read without zooming.
-
Figures 2 & 3 appear upstream of any experimental setup or results (even before background and motivation), which creates some flow issues for the reader.
-
The abstract states "across three models and seven benchmarks", which feels like a bit of a stretch. The full method comparison is conducted across 3 models and 4 datasets, while additional results showing transfer-only comparisons use 3 alternative datasets (AIME ‘25 and ‘26, along with MATH-500, ran over Qwen)
-
Other work (REUSERL) treats skills as reusable procedures and also uses a compactness signal. Could the authors distinguish themselves from this previous literature?
18 comments
Response to Weakness 2 (Consistency of Reported Performance and Token Savings)
Weakness 2: Some of the performance ranges noted in the abstract appear to be inconsistent with downstream results. For instance, the abstract notes “at test time, the evolved agent improves task performance by up to 23.1 percentage points while using 6.4%–77.9% fewer output tokens.”, but this appears to 1) understate the max gain (Table 1, FLM ALFWorld shows as 32.3), and 2) slightly overstate the lower bound on token savings (where DeepSeek and GLM TriviaQA results show -2.8 and -3.3%, respectively). This is fairly minor, but should be corrected, as it also appears verbatim in the introduction (L103-104).
Thank you for catching this inconsistency. The original summary was based only on the Qwen3.6-35B-A3B results. We have corrected both the Abstract and Introduction to cover all 12 model–benchmark settings in Table 1: up to 32.3 percentage points in task performance and 2.8%–77.9% fewer output tokens, relative to No Skill.
Response to Weakness 3 & Question 3 (Comparisons with ACE and SkillGrad)
Weakness 3: Some close alternatives mentioned in the appendix are ACE and gate-free SkillGrad. These are not used as baselines in the main paper, however.
Question 3: Could the authors include the baselines mentioned in W3 and compare them against FreeEvo?
Thank you for this suggestion. We have added direct comparisons with ACE and gate-free SkillGrad to Section 4.3, Tab. 2 of the revised main paper and report the corresponding results below.
Experimental setup. We compare the methods on Olympiad, GPQA-D, and TriviaQA using Qwen3.6-35B-A3B for execution and GPT-5.5 for optimization, with shared data splits and execution settings.
Performance is accuracy for Olympiad and GPQA-D and exact match (EM) for TriviaQA, in percent. Evo counts evolution input + output tokens, including execution and optimizer calls. Test-out counts output tokens per complete test evaluation, and Test-total counts test input + output tokens. Token costs are in millions (M); bold marks the best value within each benchmark.
| Benchmark | Method | Performance (%) ↑ | Evo (M) ↓ | Test-out (M) ↓ | Test-total (M) ↓ |
|---|---|---|---|---|---|
| Olympiad | ACE | 83.1 | 32.07 | 12.52 | 19.24 |
| Olympiad | SkillGrad | 79.7 | 8.78 | 13.28 | 16.36 |
| Olympiad | FreeEvo | 84.6 | 13.12 | 10.03 | 10.52 |
| GPQA-D | ACE | 80.4 | 4.86 | 0.79 | 1.70 |
| GPQA-D | SkillGrad | 76.8 | 3.93 | 0.83 | 2.11 |
| GPQA-D | FreeEvo | 82.6 | 1.41 | 0.62 | 0.79 |
| TriviaQA | ACE | 83.4 | 31.47 | 1.08 | 34.40 |
| TriviaQA | SkillGrad | 82.2 | 8.71 | 1.19 | 27.61 |
| TriviaQA | FreeEvo | 85.9 | 11.22 | 0.84 | 19.91 |
FreeEvo improves performance over ACE by 1.5–2.5 percentage points and over SkillGrad by 3.7–5.8 points, while using fewer test output and total tokens on all three benchmarks. Its evolution cost is lower than ACE's on all three tasks and SkillGrad's on GPQA-D. SkillGrad uses fewer evolution tokens on Olympiad and TriviaQA, but trails FreeEvo in performance by 4.9 and 3.7 points, respectively. These comparisons support FreeEvo's effectiveness against both experience curation and alternative gate-free skill updates.
Response to Weakness 1 & Question 2 (Execution Settings and Optimizer Configuration)
Thank you for pointing out the missing details. All experiments use GPT-5.5 as the optimizer, with medium reasoning effort and an output limit of 8,192 tokens per call. We have completed Appendix C.1.2 with these optimizer settings and the execution configurations, including decoding parameters, output limits, and the ALFWorld step limit.