acceptodds
Under review as a conference paper at ICLR 2027

MetaSkill: Reusable Problem-Solving Methods for Adaptive Skill Orchestration in LLM Agents

Abstract

Reusable skills provide specialized procedures, but large language model (LLM) agents must still determine how to organize and adapt their use. We introduce MetaSkill, a two-layer framework for reusing problem-solving methods across tasks and domains. The method layer selects from 26 manually authored methods and binds abstract roles to task objects. The execution layer uses the instantiated method to identify intermediate objectives, coordinate native skills, and guide repair or method reselection. On 89 SkillsBench tasks and 995 ToolHop examples, MetaSkill outperforms both evaluated baselines in all six configurations, with accuracy gains of 3.51–13.48 percentage points over the strongest baseline. Ablations support explicit method-guided orchestration beyond direct method injection and further gains from method reselection.

Primary area:
foundation or frontier models, including LLMs
Keywords:
LLM agents, skill orchestration, problem-solving methods, problem representation, tool use, adaptive planning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4TWeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MetaSkill, a two-layer framework for LLM agents equipped with reusable skills. The Method layer maintains a library of 26 manually authored problem-solving methods and each method specifies entry conditions, exclusions, role slots, state variables, policy rules, invariants, review checks, repair rules, and termination criteria. At task time, the agent selects a method and binds its abstract roles to task objects. The Execution layer uses the instantiated method to determine intermediate results, select and coordinate native skills, and check outcomes. On failure, it either repairs execution locally or reselects the method.

Experiments cover 89 SkillsBench tasks and 995 ToolHop examples, with two agent frameworks and two LLMs. MetaSkill outperforms BoT and AgentSkillOS in all six configurations, by 3.51–13.48 pp over the strongest baseline. Ablations on SkillsBench indicate that method-guided orchestration outperforms injecting method definitions as context, and that method reselection contributes 4.50 pp.

Critical strengths
  1. The paper targets a relevant gap. Reasoning-template approaches offer reusable strategies that are not tied to concrete skill execution. Skill-composition approaches organize execution without method-level guidance on what to check or when to revise.
Critical weaknesses
  1. The central claim is not isolated. Direct injection removes both the method library and the explicit orchestration loop, and there is no "orchestration without methods" control, so the gains cannot be attributed to methodological knowledge. The existing numbers point the other way: injecting all 26 methods (55.06) scores below BoT (60.67), and MetaSkill without reselection exactly matches BoT, which leaves a simple check-and-retry loop as an equally plausible explanation.

  2. The ToolHop error analysis contradicts the proposed mechanism. MetaSkill's advantage comes from eliminating incomplete executions (69 -> 0), while completed-but-incorrect answers rise (284 -> 329). Among completed executions, MetaSkill is no more accurate than BoT (64.7% vs. 65.8%), so the gain reflects finishing more reliably rather than better dependency reasoning.

  3. The SkillsBench evidence is statistically weak. Margins over the strongest baseline amount to only 4-12 of 89 tasks, from a single run with no variance estimates or significance tests. All ablation claims, including the 4.50 pp (4-task) reselection effect, rest solely on this benchmark.

Questions for authors
  1. How does the full orchestration loop perform without any method library, in the Table 5 configuration? This control would show whether the gains come from methodological knowledge or from orchestration alone.
  2. On ToolHop, what is each method's accuracy among completed executions, and how are incomplete executions distributed across chain lengths? Figure 2 suggests the advantage comes mainly from completing execution rather than from more accurate answers.
  3. Are the SkillsBench results from a single run, and can you report variance and paired significance tests across multiple runs? The margins correspond to only 4–12 of 89 tasks.
  4. Which tasks were the 26 methods derived from, and do any overlap with the SkillsBench evaluation tasks?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 4TWe

Q1: Isolating the contribution of method knowledge

We added two controls on the same 89 SkillsBench tasks using PI Agent with DeepSeek-V4.1-Flash. The gap is full MetaSkill task accuracy minus the accuracy of each configuration.

ConfigurationTask accuracy (%)Gap to full MetaSkill (%)
Generic orchestration without a method library (new)57.307.87
Online method generation (new)53.9311.24
Direct method injection55.0610.11
Full MetaSkill65.17—

The no-library control retains planning, online skill selection, execution, checking, and bounded repair. Its gap to full MetaSkill measures the contribution of adding curated methods together with their role bindings and decision rules. Generating a method for each task under the same budget also yields lower accuracy, supporting the value of reusing knowledge across tasks. Direct injection supplies the curated definitions as text. Its lower accuracy shows the additional benefit of applying those definitions to guide execution.

Q2: Completed accuracy and incomplete executions

All methods in the ToolHop experiment share a maximum of nine execution steps. An execution is classified as incomplete if it has not produced a final answer file meeting the submission requirements before the budget expires. This differs from submitting an incorrect answer. Incompleteness therefore reflects the ability to organize useful calls and deliver an answer within a finite budget, rather than an external failure independent of the solving strategy.

Some baseline executions exhaust steps on indirect, redundant, or erroneous tool calls without obtaining information needed downstream. MetaSkill binds roles to the entities being resolved, uses method rules to constrain call dependencies and required results, and reuses available results. These constraints direct limited steps toward useful progress. Reducing incomplete executions is therefore a benefit of method-guided tool orchestration under the shared step budget, supporting the proposed mechanism rather than contradicting it.

Q3: Repeated runs and statistical tests

The SkillsBench results are from single runs. We have not completed independent reruns or paired significance tests, and do not report cross-run variance or statistical significance.

Improvements across all six configurations show a consistent direction across models, agents, and benchmarks; the added controls examine the source of the gain. Neither measures run-to-run variability. The results support improvements in the evaluated settings, with stability across independent runs still to be quantified. We will conduct independent repeat runs to assess the stability of these results.

Q4: Method sources and evaluation-task overlap

Using only public task descriptions and native Skills as source material, we built the library through LLM-assisted analysis and human synthesis. No execution trajectories, failure outcomes, or scoring feedback were used in construction. We first organized seven families by the main work needed to complete tasks, using task goals and Skill-based solution approaches to scope cross-task comparisons. Within each family, we compared applicability conditions, key dependencies, and decision principles to extract reusable logic.

Task objects became bindable roles; shared relations guiding progress, checking, and adjustment were retained, while implementations stayed in native Skills. LLMs assisted analysis and candidate drafting; humans compared candidates, checked sources, revised, and selected 26 structured MetaSkills across the seven families. Section 4.1.1 and Appendix A clarify the process and library in the revision.

For example, separate-nuisance-estimate shares reference checks across economic detrending and video analysis while leaving estimators in native Skills. Methods encode no reference answers, target outputs, or complete solutions.

Evaluation fixes definitions; runtime adaptations do not update them. ToolHop is not a construction source. Applying the library there improves accuracy by 3.51-4.72% over the strongest baseline, supporting reuse beyond SkillsBench source procedures.

Thanks for the detailed reply. The new Q1 controls are useful, but since the method's contribution remains confounded with the much higher token budget and the Q2 breakdown is still missing, I will keep my original score.

Reviewer_E9WVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

MetaSkill is a two-layer framework designed for reusing problem-solving methods across tasks and domains. The method layer selects applicable methods from 26 manually-written approaches and maps the abstract roles to specific task objects, converting general principles into guidance for the current task. The execution layer determines intermediate goals based on the instantiated methods, selects and coordinates native skills, manages the dependencies between steps, and checks the execution results. When the results do not meet the requirements, the system adjusts the input, skills, or role bindings according to the problem; if the applicable premise of the current method does not hold, a new method is selected. During this process, the system retains the still valid intermediate results, re-examines or recalculates the parts affected by the modifications, thereby achieving dynamic skill arrangement based on feedback.

Critical strengths
  1. The method has cross-task reusability value. MetaSkill separates the problem-solving principles from the specific skill operations. By using applicable conditions, abstract roles, and decision rules to represent the method, tasks from different domains can share similar reasoning structures while retaining their respective professional operation procedures.

  2. The framework binds roles and maps the general method to the objects in the current task. Based on this, it determines intermediate goals, organizes skill dependencies, and sets checking conditions. This enables the method knowledge to directly influence the agent's actions, rather than merely being incorporated as background information into the context.

  3. The feedback mechanism can distinguish between local execution problems and method selection problems. The system retains valid intermediate results and only rechecks or recalculates the affected parts, making the execution process have targeted repair and adjustment capabilities.

Critical weaknesses
  1. The core proposition of cross-task and cross-domain reusability proposed in the paper lacks sufficient verification. 26 methods were manually extracted from public task descriptions and skill processes, but the paper did not clearly explain the source of the method library construction and the isolation relationship between the evaluation tasks. Therefore, the current results are difficult to distinguish between the method knowledge designed for the evaluation tasks and the truly transferable problem-solving principles.

  2. The performance gains demonstrated in the experiments are confused with additional computations and repair opportunities. In some configurations, MetaSkill consumed significantly more tokens. For example, under the DeepSeek/PI Agent settings, it increased by approximately 88% compared to the strongest baseline. The paper lacks clear budget comparisons, so it is not possible to fully determine how much of the performance improvement comes from the method design and how much from additional reasoning, checking, and repair opportunities.

  3. The key ablation of the paper failed to isolate the contribution of the method representation itself. The comparison between the complete system and the direct injection of method descriptions simultaneously changed multiple factors such as method instantiation, skill arrangement, inspection, and repair. Existing experiments have not clearly answered: when keeping the execution framework and budget consistent, do the designs of abstract roles, method-specific rules, etc. still bring independent benefits?

Questions for authors
  1. What are the sources of the 26 manually-written methods? Did they use evaluation tasks or execution feedback during their development? Can the authors explain the separation between library construction and testing, and evaluate the frozen library on unseen tasks or domains?

  2. Under a fixed computing budget and review/fix opportunities, can MetaSkill still maintain its advantage? Can the method be decomposed, the fixed execution framework be fixed, and specific method role bindings and decision rules be removed or replaced to isolate their contributions?

  3. How many times did method re-selection occur, and how frequently did it improve or harm performance?

2 comments
AuthorsOpenReview ↗

Response to Reviewer E9WV

Q1: Library construction and cross-dataset transfer

Using only public task descriptions and native Skills as source material, we built the library through LLM-assisted analysis and human synthesis. No execution trajectories, failure results, or scoring feedback were used in construction or revision. We first organized seven families by the main work needed to complete tasks, using task goals and Skill-based solution approaches to scope cross-task comparisons. Within each family, we compared applicability conditions, key dependencies, and decision principles to extract reusable logic.

Task objects became bindable roles; shared relations guiding progress, checking, and adjustment were retained, while implementations stayed in native Skills. LLMs assisted analysis and candidate drafting; humans compared candidates, checked sources, revised, and selected 26 structured MetaSkills across the seven families. Section 4.1.1 and Appendix A describe this process and list the methods and families in the revision.

For example, economic detrending and video analysis share the rule of estimating a reference before interpreting residuals, while using different filters and motion estimators. Roles and rules preserve this logic; native Skills supply implementations.

The library was constructed from SkillsBench's public task descriptions and native Skills through the abstraction process above. Library construction and testing are separate processes: test results were not used to iteratively revise or optimize the library. Its definitions remain fixed during evaluation; task-local adjustments do not update the library.

ToolHop evaluates the same fixed library on a different dataset that was not used in construction. Across its 995 examples, MetaSkill improves strict accuracy over the strongest baseline by 3.51-4.72%, demonstrating reuse with different tools and task objects beyond the SkillsBench construction sources.

Q2: Fixed budgets and method-component controls

Execution steps and token expenditure have fixed upper limits, with bounded review/repair opportunities. All exploration and operations occur within these preset limits for normal task completion. Online generation also uses the same task budget.

Higher token use with stronger models and more exploratory agents reflects necessary Skill calls, checks, and repairs. Method rules improve call precision and correct ineffective operations to obtain usable results: diagnosing a missing month contribution enables a corrected tool call, while CAD checks validate preservation after repair. GPT-4.1-mini also exceeds BoT with fewer tokens under both agents, so higher expenditure is not a uniform requirement.

We added two controls within the execution framework to isolate reusable method guidance. Both are included in the revised experiments on 89 SkillsBench tasks with DeepSeek/PI:

ConfigurationAccuracy (%)
Generic orchestration without the method library57.30
Online task-local method generation53.93
Full MetaSkill65.17

Generic orchestration preserves planning, online skill selection, execution, checks, and bounded repair, while removing method definitions, bindings, and method-specific rules. Full MetaSkill improves over it by 7.87%. Online generation replaces the curated library with task-local guidance under the same budget; full MetaSkill improves by 11.24%. Together with direct injection (55.06%), these comparisons support applying reusable knowledge through task-bound roles and decision rules to set intermediate objectives, checks, and repair directions.

Q3: Method reselection and outcomes

Method reselection improves recovery when the current approach is unsuitable. It is separate from local repair and role rebinding, with at most one switch per task (Appendix B). Each configuration contains 89 tasks; DS denotes DeepSeek.

ConfigurationTasks with reselectionUltimately successfulStill unsuccessful
PI Agent + DS642
Claude Code + DS752
Claude Code + GPT-4.1-mini15213
PI Agent + GPT-4.1-mini12111

With DeepSeek, 4/6 and 5/7 replanned tasks succeed; GPT-4.1-mini recovers 2/15 and 1/12, reflecting execution capability. The remaining cases stay unsuccessful; no correct outcome is turned into failure in these reported cases. Disabling reselection lowers DeepSeek/PI accuracy from 65.17% to 60.67%, a 4.50% drop, supporting changing the approach when local repair is insufficient.

Thank you for the clarifications and additional experiments. I appreciate the new evidence, though I still have reservations about how strongly it supports the paper’s central claims, like which SkillsBench evaluation task descriptions, if any, were used to build the method library. I will maintain my original score.

Reviewer_1LtjOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper investigates meta skill for LLM agents, which teaches LLM general knowledge about how to solve a task. The proposed metaskill is a manually written skill library that is later used by the LLM agents in execution time for task-specific adaptation. The experiments on skillsbench and toolhop show that it helps improve agent performance and outperform several baselines.

Critical strengths
  • Clear motivation: the meta-skill method is clear and intuitive. The experiments also show improvement.
Critical weaknesses
  • Potential leakage: the metaskills are written based on public task instructions and skills. It's unclear whether the methods are written using the same tasks that are used for evaluation. If so, the comparison with baselines is not fair. It would be beneficial to involve a potential leakage analysis.

  • Higer computation cost: the proposed method often uses more tokens than baselines. For example, it requires 60-90% more tokens on SkillsBench as reported in the experiment section. The additional token cost may come from the repair and reselection steps. It's unclear whether the performance improvement comes from the meta-skills or simply from more computation. It would be beneficial to compare the methods under same token budget.

  • Limited analysis of the skill library: The paper does not explain clearly how each of the skill in the meta skill library are selected and written. It also does not report how often each skill is selected for downstream tasks. Therefore, it's unclear how these skills improve the downstreawm performance.

  • Limited baselines: the main tables does not include agent performance that directly uses task specific skills, e.g. the SkillsBench has a set of human curated skills for each task. Including baselines some other baselines like SkillOpt should also be beneficial.

Questions for authors

Please see weakness. And I feel this paper is purely generated by LLM agent. For example, the main paper is so short that it barely fit 9-page limit and the introduction is so short and informless, which makes me think that it's even worse than an undergraduate course work. Can author clarify whether such a low quality paper is really human written or you're using an LLM auto-research framework for this paper?

2 comments
AuthorsOpenReview ↗

Response to Reviewer 1Ltj

Q1: Potential leakage

Using only public task descriptions and native Skills as source material, we built the library through LLM-assisted analysis and human synthesis. No execution trajectories, failure results, or scoring feedback were used in construction. We first organized seven families by the main work needed to complete tasks, using task goals and Skill-based solution approaches to scope cross-task comparisons. Within each family, we compared applicability conditions, key dependencies, and decision principles to extract reusable logic.

Task objects became bindable roles; shared relations guiding progress, checking, and adjustment were retained, while implementations stayed in native Skills. LLMs assisted analysis and candidate drafting; humans compared candidates, checked sources, revised, and selected 26 structured MetaSkills across the seven families. Section 4.1.1 and Appendix A explain the process and library in the revision.

Methods contain no reference answers, target outputs, or complete solutions. Agents still instantiate methods and compute answers. SkillsBench tests whether organizing public task and Skill knowledge improves its application.

Q2: Token cost and computing budget

Higher token use with stronger models and more exploratory agents reflects necessary execution, checking, and repair. DeepSeek with Claude Code can perform more such work. Method guidance makes calls more precise and productive: diagnosing a missing month contribution enables a corrected tool call, turning an ineffective operation into a usable result; CAD checks validate preservation after repair. These operations advance task completion.

Runs remain bounded by preset task limits. Higher cost is not universal: GPT-4.1-mini exceeds BoT with fewer tokens under both agents; with Claude Code, accuracy improves by 5.61% while token use falls by 12.1%. Revised Tables 2-3 report costs and accuracy.

Q3: Method construction and selection frequency

As detailed in Q1, seven families scope comparisons of task-solving processes; LLM-assisted analysis and human synthesis extract reusable roles and rules within each family. At runtime, applicability guides selection and roles bind to task objects.

In the 89-task DeepSeek/Claude Code campaign, counting each original execution once, 88 initial accepted plans selected 20 of the 26 methods; one task had no accepted plan.

MethodsSelections per method
rule-model-recompute30
parse-map-reconcile; localize-preserve-repair10
separate-nuisance-estimate; identify-calibrate-generalize; map-preserve-migrate4
align-synchronize-fuse; segment-transform-reassemble; filter-compare-rank; feedback-error-stabilize3
constrain-propagate-search; partition-balance-merge; template-bind-instantiate; receding-horizon-act-replan2
contrast-test-eliminate; trace-link-resolve; triangulate-resolve-synthesize; recover-from-redundancy; contract-decompose-integrate; measure-bottleneck-optimize1
Remaining six methods0

Rule-model-recompute spans energy, planning, and finance; localize-preserve-repair spans CAD, documents, and code. Methods guide different procedures across domains; ablations test their contribution.

Q4: Additional baselines

The revised main comparison includes WithSkill, previously reported as the skills-only ablation, and CoM, alongside BoT and AgentSkillOS. Across all four SkillsBench configurations, MetaSkill exceeds WithSkill by 7.87-14.61% and CoM by 6.74-20.23% (Table 2). The WithSkill comparison establishes gains beyond access to native procedures. The CoM comparison tests general methodology selection and iteration, while the execution-framework controls examine the contribution of MetaSkill's explicit application of method knowledge.

The SkillsBench-based SkillOpt: Trajectory-Derived, Verifier-Grounded Compilation of LLM-Agent Skills uses verifier source and feedback for offline same-task optimization. Its verifier access and repeated-task objective differ from ours; CoM directly tests methodological guidance.

Q5: AI use and authorship

We, the authors, rather than AI, designed, organized, and conducted this research. We did not use an autonomous research framework. We invested substantial time and effort in method design, source comparison, library curation, manually arranging and executing experiments, diagnosing implementation issues, and analyzing and verifying results. LLMs assisted analysis, candidate drafting, writing and polishing, and figures; the authors checked sources, revised rules, selected methods, and made the research decisions.

The revision expands motivation and prior work, details construction, and adds baselines and controls. The method library, experiments, analysis, and manuscript remain the authors' work and responsibility.

Thanks for the detailed response and additional results. I'm glad to see the paper being largely improved in the revision version. However, given the immature initial version of this paper, I would like to keep my original rating so that authors can improve the paper quality in next submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.