acceptodds
Under review as a conference paper at ICLR 2027

Test Time Computing for Evolving Agent Skill Optimization

Abstract

Large language model (LLM) agents increasingly rely on reusable skills that en- code procedural knowledge for specialized tasks. Recent work has begun to op- timize such skills automatically from execution feedback, but existing methods primarily focus on how to revise an individual skill, while the outer search over alternative refinement paths remains comparatively underexplored. Early revi- sion choices can therefore narrow subsequent exploration, and simply spending more optimization computation along the same trajectory may not recover alter- native directions. We introduce EvoBranch, a plug-in evolution framework that separates the native skill-revision mechanism from the search strategy around it. EvoBranch reuses an existing skill optimizer without modifying its revision pro- cedure, while broadening exploration through two complementary mechanisms. Branch-local optimization uses execution trajectories and branch-specific history to produce multiple targeted revisions from each parent skill, while global evolu- tion generates additional candidates across branches through mutation, crossover, and regeneration. Validation-based Top-M beam selection then determines which refinement paths receive further optimization. We evaluate EvoBranch with 3 skill optimizers, 3 LLM backbones, and 4 benchmarks. EvoBranch im- proves the average performance of all nine model-optimizer combinations, with gains of up to 51.4 percentage points .Compute-controlled comparisons further show that longer single trajectories and independent restarts do not reproduce these gains.Anonymous code is available at: https://anonymous.4open. science/r/EvoBranch-for-skill-optimization-DA3F/

Primary area:
optimization
Keywords:
LLM agents, skill optimization, test-time computation, skill evolution

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_apLfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces EvoBranch, an outer optimization framework that wraps existing skill optimizers with branch-specific history, multiple local revisions, global evolutionary proposals, and validation-based beam selection. It evaluates the framework with three optimizers, three backbone models, and four benchmarks, aiming to show that organizing optimization across competing refinement paths improves skill optimization.

Critical strengths
  1. Separating native proposal mechanisms from outer search has practical value. The shared adapter design supports SkillOpt, SSO, and SkillGen, while branch-specific issue and revision histories provide a structured alternative to independent restarts.

  2. The appendix provides unusually informative diagnostic material. Detailed costs, repeated runs, final skills, transfer results, and benchmark-specific shortcut analyses allow readers to examine the sources and limitations of the reported gains. These observations could support a useful contribution on the behavior of skill optimization.

Critical weaknesses
  1. The complete results do not establish the claimed advantage of the search strategy under comparable conditions. The abstract and main text state that longer trajectories and restarts do not reproduce the gains. However, Table 10 reports that a native SkillOpt trajectory truncated to the budget achieves 69.2%, compared with EvoBranch’s 67.3%, using fewer calls (963 versus 1,047) and tokens (5.32M versus 5.36M). Appendix B.3 explicitly acknowledges that equal-compute superiority has not been demonstrated. Table 11 also shows substantial variation: full EvoBranch obtains 50.5% and 67.3% in identical configurations, while removing global evolution yields 70.1% and 63.6%. Furthermore, the SSO comparison changes the selection supervision from a label-free pairwise judge to labeled validation accuracy. Together, these findings prevent attributing the reported gains specifically to better search organization.

  2. The strongest improvements largely reflect benchmark shortcuts rather than stronger task-solving skills. Appendix B.5 shows that the SkillOpt–Luna LiveMath improvement from 41.1% to 67.3% comes entirely from exploiting a meta-option that is always correct when present: correct answers increase from 10 to 41 out of 48 such questions, while performance on the other 59 questions decreases from 34 to 31. On ChemLLMBench, always answering “No” achieves 80.4%, and the authors report that every 80.4% entry exhibits this constant behavior; no configuration exceeds that baseline. The highlighted maximum gain of 51.4 percentage points reaches this majority-class result. These findings do not show that the optimization algorithm is unusable, but substantially weaken the evidence for meaningful, reusable skill improvement.

  3. Skill-MAS: Evolving Meta-Skill for Automatic Multi-Agent Systems is relevant prior work on refining a persistent skill using multiple rollouts and contrastive feedback. Discussing it would help distinguish multiple execution trajectories under one incumbent skill from EvoBranch’s simultaneous refinement paths with branch-specific histories and cross-branch evolution.

Questions for authors

See Weakness

Reviewer_5DXjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates how to optimize reusable LLM Agent Skills by increasing test-time compute. The authors propose EvoBranch, a multi-branch evolutionary framework that can integrate existing skill optimizers. The method separates skill revision from the outer search process and allocates optimization compute across multiple candidate skills through tree-structured multi-branch parallel optimization and Top-M selection.

The experimental results show that EvoBranch improves the average performance across all nine model–optimizer combinations.

Critical strengths
  1. This paper focuses on how to allocate optimization compute across multiple skill refinement paths. EvoBranch can integrate different existing skill optimizers, offering practical value.

  2. The paper conducts relatively extensive experiments and provides ablation analyses of the main components. The authors also report optimization-time token usage and LLM call counts, providing a basis for analyzing the actual computational cost of the method.

Critical weaknesses
  1. Limited methodological novelty and insufficient differentiation from existing evolutionary prompt/skill optimization methods.

Although this paper focuses on an important research question—how to organize limited test-time compute to prevent skill optimization from prematurely converging to a low-quality refinement path when the LLM's skill revision capability is fixed—optimizing candidate prompts or skills using execution feedback and expanding the search space through beam search, mutation, crossover, and multi-candidate selection have already been explored in ProTeGi, Promptbreeder, GEPA, and EvoSkill.

In particular, GEPA already employs execution-trajectory-based reflection and candidate combination, while EvoSkill has explored multi-candidate skill evolution.

  1. Insufficient validation of the advantages of Branch-specific History over existing approaches.

Branch-specific History is presented as an important design feature distinguishing EvoBranch from existing methods such as EvoSkill. The authors compare variants without History, History-aware Partition, and Attempt-aware Proposal, but do not directly compare Branch-specific History against EvoSkill's Shared Global History. More ablation experiments are needed to demonstrate the superiority of the proposed method over existing approaches when addressing the same research problem.

Questions for authors

Q1. What is the core algorithmic novelty of EvoBranch compared with existing evolutionary prompt/skill optimization methods such as GEPA and EvoSkill? Could the authors provide direct experimental comparisons under matched computational budgets?

Q2. What are the unique advantages of Branch-specific History over Shared Global History? Could the authors provide additional ablation experiments to demonstrate that maintaining independent histories for different branches is indeed superior to sharing historical experiences across all branches?

Reviewer_jNLJOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This manuscript proposes EvoBranch, a search framework that wraps existing optimizers for reusable LLM skills. It maintains multiple candidate branches, records branch-specific optimization history, partitions execution evidence to produce targeted revisions, and generates further candidates through mutation, crossover, and regeneration. Validation scores determine which branches survive. Empirical experiments cover three optimizers, three backbone models, and four benchmarks, with additional compute comparisons, ablations, and transfer experiments. The main results report higher average accuracy for all nine model–optimizer combinations.

Critical strengths
  1. This is a practical, reusable integration design. The separation between native proposal generation and candidate search is useful, and the adapters for three different optimizers demonstrate concrete engineering applicability. Branch-specific histories and evidence partitioning provide plausible mechanisms for avoiding repeated unsuccessful revisions.
  2. Substantial diagnostic disclosure. The appendices provide optimization costs, repeated runs, final skills, and analyses of benchmark-specific behavior. These details make it possible to assess what the system actually learns and identify where aggregate accuracy is misleading.
Critical weaknesses
  1. The central claim of an advantage beyond additional compute is not established, and the main text contradicts the fuller results. The abstract and Section 4.3 state that longer trajectories and independent restarts cannot reproduce EvoBranch’s gains. However, Table 10 reports native SkillOpt reaching 69.2% with 963 calls and 5.32M tokens, compared with EvoBranch’s 67.3% with 1,047 calls and 5.36M tokens. Moreover, identical EvoBranch configurations achieve 50.5% and 67.3% on LiveMath, while Table 11 reports repeated ablations without a consistent aggregate analysis. These results do not support the strong causal conclusions about search breadth or component contributions.
Questions for authors
  1. How should the abstract and Section 4.3 be reconciled with Table 10’s successful within-budget native baseline? What narrower claim do the complete results support?
  2. Why are the last runs selected for the main ablation table, and what conclusions remain when all repeated runs are aggregated with uncertainty estimates?
  3. How much of the SSO improvement remains when a sequential SSO baseline receives the same labeled validation selection and optimization budget?
  4. What advantage remains after controlling for the LiveMath answer-option shortcut and evaluating ChemLLMBench using class-sensitive metrics and the constant-majority baseline?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.