acceptodds
Under review as a conference paper at ICLR 2027

Think Less, Evolve Better: Unlocking Gate-Free Skill Self-Evolution in LLM Agents

Abstract

Skill self-evolution enables language model agents to refine reusable instructions from execution experience without updating model parameters. Existing methods often rely on costly validation gates to filter harmful revisions, yet removing these gates allows harmful updates to accumulate. We observe that accuracy-degrading revisions often lengthen reasoning, while fixed guidance against redundant reasoning mitigates degradation during ungated evolution. These observations suggest that reasoning efficiency can identify harmful guidance from existing execution trajectories. Motivated by this insight, we propose FreeEvo, a framework that eliminates validation gates by turning reasoning efficiency feedback into instruction selection. FreeEvo decomposes each skill into task-specific accuracy atoms and a fixed efficiency atom that discourages redundant reasoning. The optimizer uses this guidance to attribute unnecessary reasoning, useful procedures, and task errors to individual accuracy atoms. Evidence accumulated across distinct examples then determines atom activation and permanent retirement without additional candidate validation rollouts. Across three models and seven benchmarks spanning four task domains, FreeEvo matches or outperforms existing SOTA methods while reducing evolution tokens by up to 93.6%. At test time, the evolved agent improves task performance by up to 23.1 percentage points while using 6.4%–77.9% fewer output tokens. Our code will be released upon publication.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent, Self-Evolving, Large Language Models

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_X3naOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Agent skills are a simple/lightweight way to adapt the capabilities of LLM agents by encoding reusable procedures as instructions and workflows, without updating the model's underlying parameters. The authors note that one issue with these skills is that they are often handwritten or distilled from execution trajectories. This static approach to fixed skills means they don’t generalize well, are not malleable enough to accommodate updates for new tasks or failure cases, and need to be rewritten across sessions to prevent skill errors in future tasks. Previous work targets validation gates (held-out and replay) to evaluate and filter candidate skills that don’t meet predefined criteria, but these gates are expensive and impose a substantial cost during skill evolution. The authors propose FreeEvo as a gate-free self-evolution framework in which a skill is first decomposed into modular skill atoms. Accuracy atoms and fixed-efficiency atoms are thereby used to encode task-specific procedures and avoid redundant reasoning. They show that across three models and four benchmarks, FreeEvo matches or beats the performance of SkillOpt and GRASP while also reducing evolution tokens and test-time output tokens by a large percentage. Interestingly, the skills evolved by one model or dataset also show signs of transfer to others.

Critical strengths
  • The authors identify an existing bottleneck in the process of skill evolution (namely, costly gates) and propose an interesting framework to address it. FreeEvo is shown to work well on a set of models and across a variety of benchmarks spanning different domains. The results are consistent across models/datasets.

  • The cross-dataset transfer results are also consistent and are shown to reduce output tokens while improving accuracy.

  • The authors disclose the use of AI tools in the writing of the paper. As is, there was an obvious effort to make the ideas and results human-readable.

Critical weaknesses

W1: Appendix C.1.2 appears to have placeholders “[XX]” for the execution settings and the optimizer configurations. Presumably, these were meant to be filled out and are critical details for reproducibility.

W2: Some of the performance ranges noted in the abstract appear to be inconsistent with downstream results. For instance, the abstract notes “at test time, the evolved agent improves task performance by up to 23.1 percentage points while using 6.4%–77.9% fewer output tokens.”, but this appears to 1) understate the max gain (Table 1, FLM ALFWorld shows as 32.3), and 2) slightly overstate the lower bound on token savings (where DeepSeek and GLM TriviaQA results show -2.8 and -3.3%, respectively). This is fairly minor, but should be corrected, as it also appears verbatim in the introduction (L103-104).

W3: Some close alternatives mentioned in the appendix are ACE and gate-free SkillGrad. These are not used as baselines in the main paper, however.

W4: Extending W3, some of the main mechanisms included in the FreeEvo framework are similar to ACE. For instance, the skill-atom representation. The efficiency attribution is specific to FreeEvo, but it is questionable that no reference to ACE is made until the appendix.

Questions for authors
  • Can the authors run an experiment to show how much of FreeEvo’s gains come from the fixed efficiency atom alone? The ablations don’t appear to cover this case, but it would help to isolate the value of the FreeEvo framework.

  • Could the authors clarify which model is used as the optimizer in the main experiments? Pursuant to W1, this is not mentioned in the appendix, but is a rather critical detail. GPT4,4 is mentioned in passing, but it is uncertain what experiments can be attributed to which optimizer model.

  • Could the authors include the baselines mentioned in W3 and compare them against FreeEvo?

Presentation/Other:

  • I commend the authors for their clear visuals and tables, which are mostly self-contained and accompanied by digestible captions. Table 1 could benefit from a brief explanation of what the different columns (Evo and Test) mean.

  • Figure 4 has a very soft font woven in that can be difficult to read without zooming.

  • Figures 2 & 3 appear upstream of any experimental setup or results (even before background and motivation), which creates some flow issues for the reader.

  • The abstract states "across three models and seven benchmarks", which feels like a bit of a stretch. The full method comparison is conducted across 3 models and 4 datasets, while additional results showing transfer-only comparisons use 3 alternative datasets (AIME ‘25 and ‘26, along with MATH-500, ran over Qwen)

  • Other work (REUSERL) treats skills as reusable procedures and also uses a compactness signal. Could the authors distinguish themselves from this previous literature?

18 comments
AuthorsOpenReview ↗

Response to Weakness 1 & Question 2 (Execution Settings and Optimizer Configuration)

Weakness 1: Appendix C.1.2 appears to have placeholders “[XX]” for the execution settings and the optimizer configurations. Presumably, these were meant to be filled out and are critical details for reproducibility.

Question 2: Could the authors clarify which model is used as the optimizer in the main experiments? Pursuant to W1, this is not mentioned in the appendix, but is a rather critical detail. GPT4,4 is mentioned in passing, but it is uncertain what experiments can be attributed to which optimizer model.

Thank you for pointing out the missing details. All experiments use GPT-5.5 as the optimizer, with medium reasoning effort and an output limit of 8,192 tokens per call. We have completed Appendix C.1.2 with these optimizer settings and the execution configurations, including decoding parameters, output limits, and the ALFWorld step limit.

AuthorsOpenReview ↗

Response to Weakness 2 (Consistency of Reported Performance and Token Savings)

Weakness 2: Some of the performance ranges noted in the abstract appear to be inconsistent with downstream results. For instance, the abstract notes “at test time, the evolved agent improves task performance by up to 23.1 percentage points while using 6.4%–77.9% fewer output tokens.”, but this appears to 1) understate the max gain (Table 1, FLM ALFWorld shows as 32.3), and 2) slightly overstate the lower bound on token savings (where DeepSeek and GLM TriviaQA results show -2.8 and -3.3%, respectively). This is fairly minor, but should be corrected, as it also appears verbatim in the introduction (L103-104).

Thank you for catching this inconsistency. The original summary was based only on the Qwen3.6-35B-A3B results. We have corrected both the Abstract and Introduction to cover all 12 model–benchmark settings in Table 1: up to 32.3 percentage points in task performance and 2.8%–77.9% fewer output tokens, relative to No Skill.

AuthorsOpenReview ↗

Response to Weakness 3 & Question 3 (Comparisons with ACE and SkillGrad)

Weakness 3: Some close alternatives mentioned in the appendix are ACE and gate-free SkillGrad. These are not used as baselines in the main paper, however.

Question 3: Could the authors include the baselines mentioned in W3 and compare them against FreeEvo?

Thank you for this suggestion. We have added direct comparisons with ACE and gate-free SkillGrad to Section 4.3, Tab. 2 of the revised main paper and report the corresponding results below.

Experimental setup. We compare the methods on Olympiad, GPQA-D, and TriviaQA using Qwen3.6-35B-A3B for execution and GPT-5.5 for optimization, with shared data splits and execution settings.

Performance is accuracy for Olympiad and GPQA-D and exact match (EM) for TriviaQA, in percent. Evo counts evolution input + output tokens, including execution and optimizer calls. Test-out counts output tokens per complete test evaluation, and Test-total counts test input + output tokens. Token costs are in millions (M); bold marks the best value within each benchmark.

BenchmarkMethodPerformance (%) ↑Evo (M) ↓Test-out (M) ↓Test-total (M) ↓
OlympiadACE83.132.0712.5219.24
OlympiadSkillGrad79.78.7813.2816.36
OlympiadFreeEvo84.613.1210.0310.52
GPQA-DACE80.44.860.791.70
GPQA-DSkillGrad76.83.930.832.11
GPQA-DFreeEvo82.61.410.620.79
TriviaQAACE83.431.471.0834.40
TriviaQASkillGrad82.28.711.1927.61
TriviaQAFreeEvo85.911.220.8419.91

FreeEvo improves performance over ACE by 1.5–2.5 percentage points and over SkillGrad by 3.7–5.8 points, while using fewer test output and total tokens on all three benchmarks. Its evolution cost is lower than ACE's on all three tasks and SkillGrad's on GPQA-D. SkillGrad uses fewer evolution tokens on Olympiad and TriviaQA, but trails FreeEvo in performance by 4.9 and 3.7 points, respectively. These comparisons support FreeEvo's effectiveness against both experience curation and alternative gate-free skill updates.

Reviewer_Vs21OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the cost of evolving reusable instructions for language-model agents and proposes FreeEvo. Specifically, it represents skills as instruction atoms and uses task-outcome and reasoning-efficiency evidence from existing trajectories to activate or permanently retire them without additional candidate-validation rollouts. Experiments across three models and four domains report lower evolution costs with competitive accuracy. Additional ablations and frozen-skill transfer experiments evaluate the proposed selection mechanism and its generalization.

Critical strengths
  • Table 2 separates fixed efficiency guidance, efficiency attribution, and retirement. Removing attribution reduces accuracy by 1.5, 0.7, and 0.9 points on the three Qwen tasks. This supports an incremental effect within FreeEvo, although it does not establish superiority over earlier curation mechanisms.
  • Evolution costs include execution, optimizer calls, validation, and checkpoint selection. The substantial savings therefore address real accounting categories. Frozen-skill transfer adds useful breadth, and the paper acknowledges that output-token savings do not transfer consistently across models.
Critical weaknesses
  • The evaluation does not isolate the proposed advance over close prior art. Previous works [1, 2] have already maintained itemized instructions and curated them using evidence. ExpeL [1] updates insight-level support and removes weak entries; ACE [2] uses identified bullets, helpful/harmful counters, and local curation. Both are cited, as is gate-free SkillGrad [3], but none is an experimental comparator. Table 1 does include matched SkillOpt and GRASP gate-removal variants, so the efficiency evidence extends beyond comparisons with gated systems. These ablations do not test native itemized curation or SkillGrad's alternative evolution rule. A matched comparison with one such method, with and without FreeEvo's efficiency signal, would better isolate the increment. The missing TAG [4] further anticipates evidence-based retirement, although its paired-utility evidence and UCB rule differ. This narrows novelty; it does not demonstrate full-method duplication.
  • Instruction-level attribution accuracy is not measured, and empirical precision is unclear. Positive evidence can endorse an unexecuted corrective instruction, allowing the optimizer to retrospectively support its own proposal. Table 2 provides downstream ablation evidence, but distinct-example counting does not establish correct attribution; targeted replay or blinded auditing would test it. Appendix C.1.2 also leaves optimizer and two execution-model configurations unspecified. The four reported repetitions evaluate frozen skills; independent complete evolution runs are not reported. Without complete settings and evolution-level uncertainty, it is unclear whether smaller accuracy effects exceed sampling variation. The cost savings remain useful, but do not establish the correctness of individual retirement decisions.

References

[1] ExpeL: LLM Agents Are Experiential Learners

[2] Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models

[3] SkillGrad: Optimizing Agent Skills Like Gradient Descent

[4] A Control Architecture for Training-Free Memory Use

Questions for authors
  • Clarification: Which execution and optimizer configurations were used, and were complete evolution runs independently repeated?
  • Requested analysis: Report paired accuracy uncertainty, separating frozen-skill evaluation variation from evolution variation.
  • Requested comparison: Under the same execution model and token accounting, how does FreeEvo compare with itemized gate-free curation, before and after adding its efficiency attribution?
  • Requested validation: How often do sampled attribution assignments survive blinded checking or targeted replay, particularly false retirement of useful atoms?
  • Attribution clarification: Please distinguish the proposed efficiency and retirement rules from the uncited Dynamic Cheatsheet [1], which already considers efficiency when updating/removing strategies, and ReMe [2], which deletes experiences using accumulated recall utility. Neither implements FreeEvo's exact distinct-example state machine.

References

[1] Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory

[2] Remember Me, Refine Me: A Dynamic Procedural Memory Framework for Experience-Driven Agent Evolution

11 comments
AuthorsOpenReview ↗

Response to Weakness 1 & Question 5 (1/4) (Increment over Prior Curation Methods and Efficiency-Based Retirement)

Weakness 1: The evaluation does not isolate the proposed advance over close prior art. Previous works [1, 2] have already maintained itemized instructions and curated them using evidence. ExpeL [1] updates insight-level support and removes weak entries; ACE [2] uses identified bullets, helpful/harmful counters, and local curation. Both are cited, as is gate-free SkillGrad [3], but none is an experimental comparator. Table 1 does include matched SkillOpt and GRASP gate-removal variants, so the efficiency evidence extends beyond comparisons with gated systems. These ablations do not test native itemized curation or SkillGrad's alternative evolution rule. A matched comparison with one such method, with and without FreeEvo's efficiency signal, would better isolate the increment. The missing TAG [4] further anticipates evidence-based retirement, although its paired-utility evidence and UCB rule differ. This narrows novelty; it does not demonstrate full-method duplication.

Question 5: Attribution clarification: Please distinguish the proposed efficiency and retirement rules from the uncited Dynamic Cheatsheet [1], which already considers efficiency when updating/removing strategies, and ReMe [2], which deletes experiences using accumulated recall utility. Neither implements FreeEvo's exact distinct-example state machine.

Thank you for identifying these related methods. We would like to clarify that FreeEvo's central contribution lies in transforming reasoning efficiency into instruction-level selection evidence to enable stable skill self-evolution without candidate-validation rollouts. Atom-level decomposition, evidence accumulation, and retirement criteria are simple yet effective mechanisms designed to realize this core idea, rather than constituting the primary novelty themselves.

Below, we clarify our baseline selection, distinguish FreeEvo's contributions from prior work, and provide additional experimental comparisons. These clarifications have been incorporated into Appendix B.4, and the new experimental results are presented in Section 4.3 (Table 2) of the revised manuscript. All revisions are highlighted in blue\color{blue}{\text{blue}}.

(1) Rationale for the baseline selection in Table 1

Table 1 deliberately selects representative skill self-evolution methods from both peer-reviewed research and the open-source community. This selection follows the paper's central objective, unlocking gate-free skill self-evolution in LLM agents. We selected GRASP from the EMNLP 2026 Main Conference, the most recent major NLP conference to announce acceptance decisions before our submission deadline, as a timely peer-reviewed baseline. We also included SkillOpt, a prominent open-source framework with over 19,000 GitHub stars. Both directly address the same skill self-evolution problem as FreeEvo, making them relevant and representative baselines.

Together, they cover the two gate mechanisms illustrated in Fig. 1: replay-based validation in GRASP and held-out validation in SkillOpt. We additionally include matched w/o gate variants to establish why simply removing validation is insufficient and to test whether FreeEvo can preserve effective evolution without that overhead. Table 1 therefore evaluates both the necessity of the problem we address and the effectiveness of our solution against representative existing approaches. The additional comparisons in part (3) address native itemized curation and alternative gate-free updates. To isolate the incremental contribution of efficiency attribution, we report a matched ExpeL comparison in our response to Question 3, also addressing Weakness 1.

AuthorsOpenReview ↗

Response to Weakness 1 & Question 5 (2/4) (Increment over Prior Curation Methods and Efficiency-Based Retirement)

(2) Clarifying the core contribution and the role of itemized curation

The reviewer's novelty concerns involve three aspects: itemized experience curation and evidence-based retirement [1-4], efficiency-aware updates [5], and gate-free evolution [6]. These aspects play different roles in our work. Our core contribution is to identify reasoning efficiency as an actionable signal for stabilizing skill evolution and to turn that signal into instruction-selection decisions without additional validation rollouts. Itemized curation and the associated activation/retirement rules provide a simple, efficient mechanism for implementing this objective. The motivation and implementation goal below explain this distinction.

Motivation. FreeEvo addresses a fundamental challenge in skill self-evolution: skill updates can either improve or degrade performance, making ungated evolution unstable. Existing methods mitigate this risk through validation gates that require additional rollouts. Our controlled gate-removal experiments demonstrate that simply removing these gates can degrade performance. We further observe that accuracy-degrading updates often coincide with increased reasoning length. This motivates our central question: Can reasoning efficiency serve as actionable evidence from existing trajectories to stabilize evolution without additional validation?

Implementation Objective. FreeEvo extends reasoning efficiency from execution guidance to instruction selection. A fixed efficiency atom guides execution and provides a reference for identifying redundant reasoning. The optimizer then examines concrete trajectory evidence, attributes observed inefficiency to relevant accuracy atoms, and uses the accumulated evidence to guide their activation or retirement. Itemized atoms enable these targeted updates, providing a lightweight mechanism for translating efficiency evidence into skill evolution decisions. The within-FreeEvo ablation in our response to Question 3 isolates the contribution of this efficiency pathway while keeping the atom representation unchanged.

The comparisons below therefore distinguish shared implementation elements from the specific efficiency signal and selection objective that motivate FreeEvo.

(i) ExpeL [1]

The review notes that ExpeL updates insight-level support and removes weak entries. However, its underlying mechanisms differ from FreeEvo in two key aspects.

Insight-level support updates. ExpeL adjusts insight-level support based on relevance to task trajectories, without explicitly considering reasoning efficiency. In contrast, FreeEvo attributes positive and negative evidence to individual instructions based on both task outcomes and observed overthinking behaviors.

Removal of weak entries. In ExpeL, only the DOWNVOTE operation decreases support, while other operations increase it. Consequently, retiring an insight requires repeated downvotes to offset previously accumulated support, making retirement difficult for highly supported insights. In our reproduction experiments, retirement occurred zero times on GPQA-D and only once each on Olympiad and TriviaQA. In contrast, FreeEvo maintains separate positive and negative evidence counts and prioritizes negative evidence, enabling more timely retirement of harmful instructions without requiring accumulated positive support to be fully offset.

(ii) ACE [2]

The review notes that ACE uses identified bullets, helpful/harmful counters, and local curation. However, we identify important distinctions between ACE and FreeEvo, including discrepancies between ACE's methodological description and its actual implementation.

Identified bullets. ACE bullets represent brief memory entries extracted from historical trajectories, whereas FreeEvo skill atoms encode structured, reusable procedural knowledge.

Helpful/Harmful Counters. Although ACE mentions these counters in its methodology, we found no corresponding mechanism in its appendix prompts or official implementation.

Local Curation. Although ACE describes incremental counting for local updates, its appendix prompts and official implementation support only the ADD operation, without evidence-based retirement. Concerns about uncontrolled memory growth have also been raised in GitHub Issues #26 and #11.

In contrast, FreeEvo explicitly accumulates positive and negative evidence from both task outcomes and reasoning efficiency to guide the activation and retirement of individual skill atoms. This enables targeted skill curation, controlling library growth while maintaining task performance.

AuthorsOpenReview ↗

Response to Weakness 1 & Question 5 (3/4) (Increment over Prior Curation Methods and Efficiency-Based Retirement)

(iii) ReMe [3]

The review notes that ReMe deletes experiences using accumulated recall utility. However, its utility accumulation and deletion mechanisms operate exclusively during testing rather than training, leading to several important limitations.

First, ReMe updates an experience's utility only when that experience is retrieved through cosine similarity. This results in sparse evidence accumulation. Harmful experiences that are not frequently retrieved may persist for extended periods before being removed, potentially continuing to degrade test performance.

Second, this mechanism relies on ground-truth task outcomes from the test set to assess retrieval utility and determine deletion. Such dependence limits its applicability in real-world scenarios where ground-truth feedback is unavailable. In contrast, FreeEvo performs all skill evolution exclusively on the training set and evaluates frozen skills during testing, ensuring broader applicability and a fairer evaluation protocol.

Finally, ReMe relies solely on task success as its utility signal. FreeEvo instead incorporates bidirectional evidence attribution across both task accuracy and reasoning efficiency. The experimental results in the following table further demonstrate the advantages of FreeEvo.

(iv) TAG [4]

The review notes that TAG already incorporates evidence-based retirement. However, TAG is not a skill self-evolution method. Its retirement mechanism identifies and removes harmful entries from a predefined memory bank. In contrast, FreeEvo integrates retirement directly into the skill self-evolution process. Specifically, retirement serves as an instruction selection mechanism that extends efficiency signals from execution guidance to instruction selection, enabling the evolving skill library to eliminate harmful instructions.

We did not include an experimental comparison with TAG because its official implementation is not publicly available.

(v) Dynamic Cheatsheet [5]

The review notes that Dynamic Cheatsheet considers efficiency when updating or removing strategies. However, Dynamic Cheatsheet does not explicitly use inference efficiency as an evidence signal for strategy selection or retirement. Instead, efficiency is considered primarily through natural language instructions in the Curator's prompt, which encourages reflection on whether existing strategies could be improved. The relevant excerpt is reproduced below:

For every new problem encountered:
1. Evaluate the Solution’s Effectiveness  
   - Was the applied strategy optimal?
   - Could the solution be improved, generalized, or made more efficient?
   - Does the cheatsheet already contain a similar strategy, or should a new one be added?

Thus, Dynamic Cheatsheet encourages efficiency-aware reflection without explicitly identifying inefficient reasoning or attributing it to individual strategies. In contrast, FreeEvo operationalizes reasoning efficiency as an explicit signal for skill self-evolution. Its fixed efficiency atom provides a reference for identifying overthinking in execution trajectories, while atom-level evidence attribution links observed inefficiency to individual instructions and guides their retirement. FreeEvo therefore extends efficiency from qualitative execution guidance to evidence-based instruction selection, enabling stable skill evolution without additional validation rollouts.

(vi) SkillGrad [6]

The review notes that SkillGrad provides an alternative gate-free evolution rule. However, SkillGrad explicitly attributes its omission of validation gates to cost and latency considerations (Appendix A.1), rather than a mechanism specifically designed to ensure stable evolution without validation. Moreover, SkillGrad itself reports performance degradation under excessive evolution (Section 5.1, Figure 2(b)), indicating that its gate-free evolution remains unstable over extended iterations.

In practice, SkillGrad directly adopts updated skills without validation and limits evolution to a small, fixed number of iterations. While this reduces the risk of degradation, it also constrains the potential benefits of continued evolution, as further demonstrated by the experimental results in the following table.

In contrast, FreeEvo explicitly addresses the stability of gate-free evolution through efficiency-aware evidence attribution and dynamic instruction retirement, enabling sustained skill evolution without additional validation rollouts.

Reviewer_VxT6OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces FreeEvo, a gate-free framework for self-evolving agent skills. Motivated by the observation that harmful skill revisions often increase reasoning length, FreeEvo decomposes skills into accuracy atoms and a fixed efficiency atom, then uses execution feedback to activate or permanently retire individual instructions. Experiments across three models and multiple reasoning and interactive benchmarks show improved task performance with substantially lower evolution costs. Ablations and transfer experiments further examine the contribution and portability of the learned skills.

Critical strengths
  • I think the motivation is quite insightful. Instead of designing a cheaper validation gate, the paper asks whether existing execution trajectories already contain enough information to identify harmful skill updates. This is an interesting perspective on reducing the cost of continual skill improvement.
  • The accuracy efficiency trade-off is impressive. FreeEvo achieves higher accuracy than SkillOpt while using substantially fewer evolution tokens. I also like that Table 2 separates fixed efficiency guidance from efficiency-based attribution, rather than treating the entire framework as a single component.
Critical weaknesses
  • I'm not convinced that reasoning inefficiency reliably identifies harmful instructions. The initial observation is correlational: many accuracy-degrading updates increase output length, but longer reasoning is not necessarily redundant, and shorter reasoning can still be incorrect. Since the optimizer attributes inefficiency to individual atoms without controlled interventions, it is difficult to know whether the retired instructions actually caused the observed behavior.
  • The largest performance gain is also the least explained. In Table 1, FreeEvo improves Qwen's ALFWorld success rate from 61.2% to 84.3%, much larger than its improvements on most reasoning tasks. Yet Table 2 excludes ALFWorld from the component ablations. I would particularly like to know whether this gain comes from the fixed efficiency atom, accumulated accuracy atoms, or their interaction.
Questions for authors
  • Could you analyze cases where efficiency-based attribution incorrectly penalizes a useful instruction? How often does the system retire an atom that would have improved accuracy despite increasing reasoning length?
  • Can you provide the component ablation on ALFWorld? I think understanding the source of its 23.1 point improvement is important to evaluating the method.
  • Why are accuracy atoms generated only during the first epoch, and why is retirement irreversible? Could useful instructions be prematurely removed before enough evidence accumulates?
11 comments
AuthorsOpenReview ↗

Response to Weakness 1 (1/3) (Efficiency Signals and Instruction-Level Attribution)

Weakness 1: I'm not convinced that reasoning inefficiency reliably identifies harmful instructions. The initial observation is correlational: many accuracy-degrading updates increase output length, but longer reasoning is not necessarily redundant, and shorter reasoning can still be incorrect. Since the optimizer attributes inefficiency to individual atoms without controlled interventions, it is difficult to know whether the retired instructions actually caused the observed behavior.

Thank you for the opportunity to clarify our motivation and how FreeEvo uses efficiency feedback. We have incorporated all these details into Appendices E.1–E.2 in our rebuttal revision, with changes highlighted in blue\color{blue}{\text{blue}}.

(1) Why efficiency and task performance can improve together in skill-guided agentic reasoning

We agree that reasoning length alone cannot determine reasoning quality: different valid solution paths can require different amounts of reasoning, and a short response can still be incorrect. Our motivation comes from the role of skills in reusing procedural experience. Without relevant experience, an agent may need to explore solution paths, test alternatives, and reflect through an extended chain of thought or agent loop. When similar tasks share useful procedures and key solution steps, a high-quality skill lets the agent reuse this experience instead of rediscovering it. Avoiding repeated exploration can therefore improve task performance and reasoning efficiency together. These joint gains have already been supported by experimental results in ReasoningBank [1] (Tab. 1), SkillsBench [2] (Tabs. 12), SAGE [3] (Tab. 2), and TRS [4] (Tab. 1).

This perspective also explains the motivation behind Fig. 3(a). To assess whether efficiency feedback can help replace a costly validation gate, we ask: when an efficiency signal flags an update as potentially harmful, how often does that update actually reduce accuracy? Using increased output length as an initial proxy for reduced efficiency, the relevant quantity is P(Δacc<0∣Δlength>0)P(\Delta\mathrm{acc}<0\mid\Delta\mathrm{length}>0), shown in the first probability column below. Both changes are measured for a SkillOpt candidate update relative to its incumbent skill.

BenchmarkP(Δacc<0∣Δlength>0)P(\Delta\mathrm{acc}<0\mid\Delta\mathrm{length}>0)P(Δlength>0∣Δacc<0)P(\Delta\mathrm{length}>0\mid\Delta\mathrm{acc}<0)
Olympiad80.0%66.7%
TriviaQA93.5%80.6%

Among updates flagged by this efficiency proxy, 80.0% on Olympiad and 93.5% on TriviaQA also reduce accuracy and would therefore be rejected by the accuracy gate. This conditional probability measures how reliably an efficiency warning identifies the harmful updates that validation is intended to filter. It thus supports the motivation for FreeEvo: use efficiency evidence from existing execution trajectories to help identify harmful guidance and stabilize evolution without additional validation rollouts.

(2) Clarifying the role of reasoning length in FreeEvo

FreeEvo does not use reasoning length as the criterion for identifying overthinking or harmful atoms. In the preliminary observation in Fig. 3(a), length served as a convenient proxy for examining the relationship between efficiency and accuracy. Because length can vary with the reasoning path and the amount of necessary verification, FreeEvo uses a length increase only to draw the optimizer's attention to possible inefficiency. The actual attribution requires inspecting the reasoning content, as reflected in the two stages below.

Execution-time guidance. A fixed, domain-specific efficiency atom guides the agent on when to reuse completed reasoning and when further work is warranted. Its generation prompt in Appendix D.1 states:

Identify concrete reasoning behaviors that repeat work without resolving a new uncertainty. Write an actionable instruction that tells the agent when to continue, reuse an established result, or stop.
Preserve checks and exploration needed to solve the task correctly.

This guidance defines useful versus redundant reasoning in terms of whether further work resolves an uncertainty, without prescribing a target reasoning length. Necessary checks and exploration are explicitly preserved. Fig. 6 gives an example of the resulting atom.

AuthorsOpenReview ↗

Response to Weakness 1 (2/3) (Efficiency Signals and Instruction-Level Attribution)

Evolution-time attribution. The optimizer uses the efficiency atom to assess execution trajectories. For repeated examples, a length increase serves only as a cue for closer inspection, as explicitly specified in Appendix D.4:

For a repeated example, compare its current execution with the previous execution and inspect the output-length change. Treat an increase as a cue for inspection, not as sufficient evidence of redundancy. If there is no previous execution, assess the current trajectory against the efficiency atom.

To determine whether overthinking actually occurred, the optimizer must examine the reasoning content against the domain-specific guidance in the efficiency atom. The attribution prompt specifies the behavioral distinction it must make:

Look for concrete behavior such as repeating a resolved check without new information, retrying an unchanged failed approach, or repeatedly switching between conflicting instructions. Distinguish this from necessary verification, productive exploration, and additional work that resolves a genuine uncertainty.

Negative efficiency evidence is assigned only when the optimizer identifies such redundant behavior and links it to a specific accuracy atom included in the executed skill. Unsupported or ambiguous assignments are omitted. Thus, length changes direct the optimizer's attention, while concrete trajectory evidence and its connection to an instruction determine the attribution.

(3) Reliability of efficiency attribution

The concern about attribution accuracy has two parts: detecting genuine redundancy and identifying the instruction responsible for it. In our experiments, we observed errors in both:

  • False-positive detection: the optimizer identifies overthinking without supporting trajectory evidence.
  • Wrong-atom attribution: the optimizer correctly identifies redundant reasoning but attributes it to the wrong atom.

Specifying rules for overthinking does not by itself ensure correct attribution: the optimizer may misapply a rule or link the observed behavior to the wrong instruction. FreeEvo therefore requires concrete evidence for both the redundant behavior and the implicated atom. For each negative assignment, the optimizer must identify the atom (atom_id), quote the relevant trajectory passage (trajectory_quote) and instruction (instruction_quote), and explain their connection (diagnostic) in a JSON object (Appendix D.4). This makes explicit what behavior is being penalized and why that particular atom is held responsible.

To measure how reliably this procedure identifies the responsible atoms, we used execution records from complete evolution runs on Olympiad, GPQA-D, TriviaQA, and ALFWorld, with Qwen3.6-35B-A3B as the execution model and GPT-5.5 as the optimizer. The audit checks whether independent judges support the assignments, while replay tests whether changing the attributed instruction produces the expected effect on execution.

AuthorsOpenReview ↗

Response to Weakness 1 (3/3) (Efficiency Signals and Instruction-Level Attribution)

(i) Blinded audit of efficiency attribution.

We retained the original optimizer inputs, hid FreeEvo's assignments, and randomized the order of accuracy atoms. Two independent frontier judges, GPT-6 Astra and Claude Opus 5.5, received these inputs and the same attribution prompt. Each case checks whether a specific atom contributed to redundant reasoning in a given execution. We checked FreeEvo's negative efficiency assignments for errors and sampled cases where it assigned no efficiency evidence to check for missed attributions.

Judge Consensus reports the proportion of cases on which both judges agreed. Error counts and precision use this consensus subset. Precision is the fraction of FreeEvo's nonzero assignments matching both judges in evidence type, polarity, and atom identity. The table reports FreeEvo's evidence-based assignments as Detailed. We also include Direct attribution, which outputs the decision without the detailed quotes and diagnostic explanation, to assess the contribution of the evidence requirement.

In the table, False Positive (FP) denotes unsupported detection, False Negative (FN) denotes missed evidence, and Wrong Atom (WA) denotes attribution to the wrong atom. Error entries list FP / FN / WA counts; consensus and precision entries give numerator/denominator (%).

BenchmarkJudge ConsensusDirect FP / FN / WADetailed FP / FN / WADirect PrecisionDetailed Precision
Olympiad356/375 (94.9%)14 / 10 / 41 / 1 / 172/90 (80.0%)81/83 (97.6%)
GPQA-D108/114 (94.7%)3 / 3 / 10 / 1 / 125/29 (86.2%)27/28 (96.4%)
TriviaQA1082/1120 (96.6%)48 / 37 / 56 / 9 / 4214/267 (80.1%)242/252 (96.0%)
ALFWorld106/117 (90.6%)2 / 4 / 21 / 1 / 229/33 (87.9%)32/35 (91.4%)
Overall1652/1726 (95.7%)67 / 54 / 128 / 12 / 8340/419 (81.1%)382/398 (96.0%)

Independent judges supported both the redundancy judgment and the responsible-atom assignment in 382/398 cases (96.0%). The 16 mismatches comprised eight unsupported detections and eight wrong-atom assignments. This directly quantifies the reliability of FreeEvo's instruction-level efficiency attribution on the consensus-audited cases. The direct-attribution comparison further supports the evidence requirement: precision increased from 81.1% to 96.0% (+14.9 percentage points), with false positives decreasing from 67 to 8 and wrong-atom assignments from 12 to 8.

(ii) Controlled replay of atoms retired with negative efficiency evidence.

To address the concern that the retired instruction may not have caused the observed inefficiency, we directly tested its effect through removal. We selected retired atoms that had received negative efficiency evidence (EFF−) and replayed the task that produced the evidence under With Atom and Without Atom configurations. We fixed the task, execution model, other accuracy atoms, fixed efficiency atom, and decoding settings, changing only the target atom's inclusion. Each configuration was evaluated independently four times.

Let ΔK\Delta K be the success count under Without Atom minus that under With Atom across the four executions. Token changes compare the arithmetic means of the four outputs per configuration. We classify each removal as follows:

  • Supported: ΔK≥0\Delta K\geq0 and mean output tokens decrease by at least 8%.
  • Contradicted: ΔK<0\Delta K<0 with no token reduction, or ΔK≤0\Delta K\leq0 with at least an 8% token increase.
  • Inconclusive: all other cases, including performance–efficiency trade-offs.

Each entry reports the number of cases in that category divided by all replayed EFF− cases. Inconclusive cases remain in the denominator of the support rate.

BenchmarkEFF− SupportedEFF− ContradictedEFF− Inconclusive
Olympiad30/35 (85.7%)1/35 (2.9%)4/35 (11.4%)
GPQA-D10/12 (83.3%)0/12 (0.0%)2/12 (16.7%)
TriviaQA83/105 (79.0%)6/105 (5.7%)16/105 (15.2%)
ALFWorld13/15 (86.7%)1/15 (6.7%)1/15 (6.7%)
Overall136/167 (81.4%)8/167 (4.8%)23/167 (13.8%)

Removing the implicated atom reduced output tokens by at least 8% without lowering observed task success in 136/167 cases (81.4%). Because only the target atom's inclusion changed, these results provide intervention-based support for its attributed efficiency penalty and the resulting retirement decision in those cases. The remaining 31 cases were contradicted or inconclusive under the stated criteria. Together with the audit, this directly evaluates whether FreeEvo identifies the responsible instruction and whether its removal has the expected effect.

[1] ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory. arXiv:2509.25140.

[2] SkillsBench: Benchmarking How Well Agent Skills Work Across Diverse Tasks. arXiv:2602.12670v1.

[3] Reinforcement Learning for Self-Improving Agent with Skill Library. arXiv:2512.17102.

[4] Thinking with Reasoning Skills: Fewer Tokens, More Accuracy. arXiv:2604.21764.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.