acceptodds
Under review as a conference paper at ICLR 2027

SkillsWiki: Taming Ever-Growing LLM Agent Skills with a Self-Evolving Hierarchical Wiki

Abstract

Large language model agents increasingly rely on a large and diverse repertoire of skills to execute long-horizon tasks, yet existing frameworks manage these skills as a flat, isolated, and static list retrieved by embedding similarity. This design scales poorly as the skill library grows, ignores the relational structure among skills, and discards the experience accumulated across tasks. We propose SkillsWiki, a unified framework that organizes agent skills into a three-layer wiki. A skill taxonomy constructed by large language models materializes skill categories as a hierarchical directory tree with summarization nodes, enabling coarse-to-fine retrieval. A skill relation graph, autonomously discovered by the agent through wiki exploration, links related skills and couples a shared cloud-side wiki with a local workspace wiki. A skill package library distills proven skill compositions from historical execution traces and surfaces them only when an associated skill is activated, enabling context-triggered reuse. The entire structure is maintained as a lightweight, file-based wiki that remains readable and editable by human developers. Experiments on three agent benchmarks with multiple backbone models show that SkillsWiki consistently improves task success rate and reduces execution steps over strong baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Skill Management, Self-Evolving Hierarchical Wiki, LLM Agents

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_iPraOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SkillsWiki organizes agent skills into a hierarchical taxonomy, a relation graph, and experience-grounded packages. Agents navigate category summaries and expand their skill sets through associated packages and relations. Experiments across three interactive environments and three backbone models report improved rewards, with component ablations supporting the usefulness of the proposed modules.

Critical strengths

SkillsWikii is practical and easy to follow. Linking composition strategies and failure guards to skills and surfacing them upon activation, provides a useful mechanism.

SkillsWikii shows improvements across multiple environments and models, provide encouraging evidence that the system’s benefits extend beyond a single model or task.

Critical weaknesses

Novelty is insufficiently. SkillNet already introduces the same three-layer architecture and automatic relation construction, and its later version includes Wiki-based exploratory skill selection. SkillWiki also studies continuously evolving skill infrastructurem, which is omitted in this paper.

Removing entire packages in the ablations does not distinguish the benefit of additional experience from that of the proposed association and activation mechanism.

Online efficiency is insufficient discuess. The paper doesn't compare retrieval latency, LLM calls, token consumption, or monetary cost. Exploratory retrieval may introduce additional inference overhead, so environment steps alone do not substantiate overall efficiency claims.

Questions for authors

With identical package content and reading budgets, does skill-triggered retrieval still outperform global experience retrieval?

Under the same skill library, model, and budget, what are each method’s online retrieval latency, token consumption, and end-to-end task cost?

1 comment

We appreciate your valuable feedback and provide a detailed point-by-point response below.

W1: novelty of SkillsWiki

We recognize the contributions of SkillNet and SkillWiki in this field. However, the framework we have proposed still has its own unique contributions:

  • SkillsWiki materializes the three layers as a single persistent, human-readable and human-editable file-based wiki with a read-only cloud copy and on-demand local materialization.
  • SkillsWiki maintains the taxonomy by signature-keyed insertion with merge/split refactor and a global reference check.
  • SkillsWiki discovers relations by having the agent explore the complete wiki, with a natural-language justification on every edge and a separation of shared cloud-side relations from environment-specific local ones.
  • SkillsWiki attaches experience-grounded packages to individual skills, absorbing both successful and failed experience. These packages surface only on activation and evolve as new traces arrive.

W2 & Q1: isolation of the package activation mechanism

Removing entire packages indeed cannot isolate the package activation mechanism. However, it needs to be clarified that skill packages are themselves designed for agents. They are used by agents to summarize experience from historical trajectories and to guide similar task execution. In the SkillsWiki retrieval mechanism, the roles of skill taxonomy, relations, and packages work coherently and are fully explored freely by the agent. Since the Wiki structure is an integrated whole, the agent's retrieval only involves the presence or absence of components, rather than non-existent sub-retrieval strategies.

W3 & Q2: online efficiency

We add a systematic comparison of online inference cost across all methods on MiniMax-m2.7, reporting input/output token consumption on ALFWorld, WebShop, and ScienceWorld (dev and test). The results show that SkillsWiki incurs substantially higher token usage than ReAct, ExpeL, and SkillNet, which we acknowledge as the cost of exploratory retrieval and discuss as a trade-off against its stronger task performance. However, with skill taxonomy, skill relations, and skill packages, SkillsWiki can indeed match better skill combinations, resulting in higher task rewards and fewer steps in most settings.

MethodALFWorld (dev)ALFWorld (test)WebShopScienceWorld (dev)ScienceWorld (test)
ReAct42,428 / 2,12443,608 / 2,2398,916 / 89038,397 / 1,98341,080 / 2,139
ExpeL74,484 / 1,50575,504 / 1,647227,772 / 3,53666,702 / 2,22752,480 / 1,345
SkillNet37,426 / 1,56942,409 / 1,810222,644 / 5,883122,889 / 2,809185,367 / 2,547
SkillWiki422,187 / 4,088417,415 / 4,209764,960 / 10,744439,286 / 4,482481,529 / 4,933
Reviewer_UpDBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses the problem that LLM agents' skill libraries are treated as a flat, isolated, and static list retrieved by embedding similarity. It proposes SkillsWiki, which organizes skills into a three-layer wiki: (1) a skill taxonomy constructed by an LLM classifier, where the directory tree itself is the category system and internal nodes carry LLM-generated summaries, enabling coarse-to-fine retrieval; (2) a directed skill relation graph autonomously discovered by the agent through wiki exploration, whose edges carry natural-language justifications and which couples a shared cloud-side wiki with a local workspace wiki; and (3) a skill package library distilled from historical execution traces, surfaced lazily only when an associated skill is activated. The whole structure is materialized as plain files (readme.md / relations.md / packages.md) that remain human-readable and editable. At inference time the agent performs wiki-based exploratory retrieval: taxonomy-guided descent → package lookup → relation-guided expansion. Experiments are run on ALFWorld, WebShop, and ScienceWorld with MiniMax-M2.7, GLM-5.1, and GPT-5.4-mini as backbones, comparing against ReAct, ExpeL, and SkillNet, and reporting task score R (↑) and mean environment steps S (↓). The central claim is that SkillsWiki consistently improves success rate and reduces execution steps across all datasets, all splits, and all backbones.

Critical strengths
  1. The problem is clearly identified, and the three-layer wiki abstraction is genuinely novel and deployable.** Evidence: §1 reduces skill management to three deficiencies (no hierarchy, no relations, no experience), with the three-panel contrast in Figure 1; §3 formalizes the wiki as W = (T, G, P, I); §4 materializes the three layers as readme.md / relations.md / packages.md and designs classification caching with per-skill-signature idempotent incremental updates, plus a read-only cloud library with on-demand local materialization (C.2). Importance: this is not another embedding index but a deployable, human-auditable form of skill governance — the abstraction retains value for the community even if the final conclusions need to be revised.

  2. Coverage is broad and the disclosure is unusually transparent. Evidence: Table 1 covers 3 benchmarks × 3 backbones × Seen/Unseen, i.e. 15 configurations, and R is the best in all 15/15 columns; the Unseen columns improve in step with the Seen columns (e.g., MiniMax ALFWorld Unseen 82.09 vs. SkillNet 72.18), indicating that the gain is not memorization of training instances; Tables 3/4/5 disclose construction cost, environment, and hyperparameters; Appendix D discloses all prompts; and §6.5 uses a "byte-identical 61-skill library" as a controlled comparison, illustrated by the paint-mixing case in Figure 4. Importance: the success-rate improvement is real and holds across model families and domains, and the artifacts most easily hidden (prompts and cost) are disclosed.

Critical weaknesses
  1. The two most important empirical claims run opposite to the paper's own tables. Problem. (a) The abstract and §6.1 claim that SkillsWiki "reduces execution steps over strong baselines" and achieves "the lowest average step count in most settings", but Table 1 does not support this. (b) §6.2 claims that "the largest drop occurs without packages" and positions packages as the most critical component, but Table 2 does not support this.
  2. Fairness of the comparison and reproducibility information are insufficient, which could undermine the attribution of the entire set of results. Problem. Three key settings are unspecified: the provenance and sharing of the skill library, how the baselines were implemented, and whether the baselines and SkillsWiki were given the same tool permissions.
Questions for authors
  1. The statement in §6.1 that SkillsWiki achieves "the lowest average step count in most settings" does not match Table 1 (only 5/15 columns are better than SkillNet; on GPT-5.4-mini it is worse in 5/5 columns). Should this claim be restated? Also, in the GPT-5.4-mini + WebShop column, SkillsWiki's S = 26.14 is exactly equal to ReAct's — is this a transcription error?
  2. What criterion does §6.2 use to conclude that packages are most critical — per-column or benchmark-averaged? Table 2 shows that w/o relations has the largest drop in WebShop and both ScienceWorld splits (3/5 columns), and also the largest mean drop. How do the authors reconcile this?
  3. §6.5 refers to SkillNet's "tool-free agent", while Table 5 shows that SkillsWiki has Read/Glob/Grep/Bash/Write and acceptEdits. In the main experiments, do all baselines run in the same harness, with the same tool permissions and the same tuning budget as SkillsWiki?
  4. Where does the raw skill set come from? How many skills are there per benchmark? Do all methods share the same skill library? Were ExpeL and SkillNet re-implemented or run from official code? Could you provide the per-method hyperparameter configuration?
  5. Figure 3 and Table 1 report different values (e.g., 88.76 vs. 88.57, 16.63 vs. 17.13). What is the difference in protocol and sample size between the two?
2 comments

We are grateful for your constructive comments, which we address point by point in the following.

W1 & Q2: the two empirical claims contradicted by our own tables

We accept the correction. (a) SkillsWiki improves task score R in all 15 configurations and reduces environment steps in a subset of settings. However, even if SkillNet can achieve fewer steps in some settings, it falls significantly behind SkillsWiki in the reward, which reflects the task completion. (b) Taxonomy, relations and packages are all essential. Removing packages causes the largest reward drops on ALFWorld, while removing taxonomy or relations mainly hurts WebShop and unseen generalization, and the slightly lower step counts of some ablations do not offset their substantial reward losses.

W2: fairness of the comparison

We use exactly the same skills as SkillNet (37 for ALFWorld, 23 for WebShop and 61 for ScienceWorld). Experience is obtained by running the agent on tasks in the training split and summarizing the resulting trajectories. Both React and Expel methods use the original implementation. All methods use the same tools. In ALFWorld, the agent can perform actions such as “go to”, “open”, and “close”. In ScienceWorld, the agent can perform actions such as “open”, “close”, and “activate”. In WebShop, the agent can perform actions such as “search” and “click”. The relevant prompts can be found in Appendix D.

Q1: the step-count claim and the identical S = 26.14 cell

For the step-count claim, please see the response to W1. In the GPT-5.4-mini + WebShop column, ReAct and SkillsWiki indeed achieved the same S result.

Q3: tool permissions and tuning budget

SkillNet has designed a skill retrieval system based on keywords and LLMs, while SkillsWiki uses agents to autonomously discover relevant skills. They differ in skill retrieval, but are the same in actions interacting with the environment, as shown in Appendix D.

Q4: skill source and hyperparameter configuration

We use exactly the same skills as SkillNet (37 for ALFWorld, 23 for WebShop and 61 for ScienceWorld). Experience is obtained by running the agent on tasks in the training split and summarizing the resulting trajectories. Both React and Expel methods use the original implementation. The experience in Expel is obtained from the training set. The experiment and parameter settings have been described in Section 5 and Appendix A.

Q5: Figure 3 vs. Table 1 values

As mentioned in Chapter 6.3, the data in Figure 3 is based on the results of experiments where the skills were successfully retrieved. They are mainly used to determine whether the retrieved skills can actually help solve tasks. SkillsWiki's advantage comes from the quality of retrieved skills rather than their quantity. It leads on reward, task progress, and LLM-judged skill reward contribution, especially when several skills must be coordinated. SkillNet retrieves comparable numbers of skills but converts them into success less often. This reflects that the skills retrieved by SkillsWiki have inherent relevance and experiential structure, enabling tasks to be completed more accurately and effectively.

Thank you for your detailed response. But there are still several concerns, two points in particular: the tuning budget was not specified, and the criterion for calling packages "most critical" (per-column vs. benchmark-averaged) was not directly addressed, so I maintain my rating of 2.

Reviewer_RY4zOpenReview ↗
Rating
2(+1)
Confidence
4(+1)
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

Existing frameworks manage skills as flat, isolated, and static lists, relying on embedding similarity for retrieval. However, no existing framework jointly addresses (i) structured skill organization, (ii) autonomous skill-relation discovery, and (iii) experience-grounded skill packaging within a unified, scalable architecture. Inspired by the wiki paradigm, the paper proposes SkillsWiki, a hierarchical wiki-structured skill repository for LLM agents that consists of three components: a Skill Taxonomy, a Skill Relation Graph, and a Skill Package Library.

Critical strengths
  1. The paper identifies a valuable research problem: existing skill libraries are organized as flat, isolated, and static lists.
  2. The paper proposes a method that jointly supports structured skill organization, autonomous skill-relation discovery, and experience-grounded skill packaging.
  3. The paper draws inspiration from the wiki paradigm to construct a skill library.
Critical weaknesses
  1. The construction of the Skill Taxonomy relies on incremental expansion rather than continuous optimization of the entire taxonomy. How can the method ensure that the resulting Skill Taxonomy has an optimal structure?
  2. When the Skill Taxonomy undergoes merging, the paths of some skills may change. Since the construction of the Skill Relation Graph depends on the Skill Taxonomy, does the method merely update the affected skill paths, or does it also re-examine the completeness and correctness of the existing relations?
  3. Could the authors report the LLM input and output token consumption at different stages? Additionally, I suggest plotting how the cost of incorporating a new skill scales with the number of existing skills and historical trajectories, to demonstrate the maintenance overhead of the proposed structured skill library as it grows.
  4. I noticed that SkillNet has already introduced the same three components: a Skill Taxonomy, a Skill Relation Graph, and a Skill Package Library, along with automated relation discovery and wiki-based skill routing. What are the key novel contributions of this paper compared with SkillNet?
Questions for authors
  1. Although skill relations are derived from experience, how does the method ensure their correctness? Is there any mechanism for validating and correcting potentially incorrect relations?
  2. Please clarify the sources of the skills and experiences used by the proposed method to ensure a fair comparison in terms of the information available in the skill libraries.
4 comments

Thank you for your careful reading. Our point-by-point responses follow below.

W1: optimality of skill taxonomy

Our design target is maintainability and retrieval effectiveness. The Skill Taxonomy is the retrieval backbone of the wiki. It turns a flat skill list into a coarse-to-fine category system. Due to the progressive disclosure nature of the wiki structure, the agent descends from a task description to a small candidate set instead of embedding-matching against all skills. On the other hand, it gives every skill and category an interpretable, human-editable location, which is what makes readme files auditable.

W2: completeness and correctness of the existing relations

When paths change, all references (readme.md, relations.md, packages.md and the metadata index) to those paths are updated in the same operation. So category paths, incident relation entries and package attachments stay consistent. To ensure the completeness and correctness of the existing relations, the agent is required to perform a global reference check over the entire wiki after a merge.

W3: LLM input and output token consumption

The section 6.4 already reports time cost for the wiki-construction stages, which bounds the offline maintenance overhead. To further evaluate the overhead of the reasoning phase, we report LLM input/output token consumption for every backbone and benchmark setting (input / output tokens):

ModelALFWorld (dev)ALFWorld (test)WebShopScienceWorld (dev)ScienceWorld (test)
minimax-m2.7422,187 / 4,088417,415 / 4,209764,960 / 10,744439,286 / 4,482481,529 / 4,933
glm-5.1365,216 / 2,173390,602 / 2,278922,058 / 5,592503,704 / 2,757565,590 / 2,979
gpt-5.4-mini688,692 / 7,719674,319 / 6,5531,076,322 / 17,508559,561 / 9,260592,584 / 9,907

We also report the incremental construction cost for three randomly selected ALFWorld skills with GPT-5.4-mini, which averages 22.43 s and 6,103.33 input / 1,476.00 output tokens per skill, confirming that maintenance overhead stays low as the library grows.

Time (s)input tokensoutput tokens
increment 123.916,2041,482
Increment 221.135,9941,457
Increment 322.246,1121,489
avg22.436,103.331,476.00

W4: novelty relative to SkillNet

Our contribution is not the three-layer architecture itself, but the instantiation and maintenance protocol that turns those layers into a single persistent, self-evolving wiki. The directory tree serves as the category system, where internal nodes carry LLM-generated summaries for coarse-to-fine retrieval. Relations are discovered by the agent exploring the complete Skill Wiki, and every edge carries a natural-language justification, with shared cloud-side relations separated from environment-specific local ones. Packages are experience-grounded skill compositions distilled from historical traces, absorbing both successful and failed experience. Therefore, the novelty of SkillsWiki lies in an auditable and incrementally friendly wiki system where an agent can both read and write.

Q1: correctness of relations

Skill relations are not derived from experience: they are discovered by the agent while it explores the complete Skill Wiki, and each edge stores a natural-language justification recording why it was posited. The experience-grounded layer is the Skill Package Library. Packages are skill compositions summarized from historical experience, retrieved to provide a composition for similar tasks. They absorb both correct and incorrect experience, and continue to evolve as new experience arrives. During merging and synchronization, the agent performs the global reference check, so inconsistent relation entries are caught.

Q2: sources of skills and experiences

We use exactly the same skills as SkillNet (37 for ALFWorld, 23 for WebShop and 61 for ScienceWorld). Experience is obtained by running the agent on tasks in the training split and summarizing the resulting trajectories. Both React and Expel methods use the original implementation. The experience in Expel is obtained from the training set. The experiment and parameter settings have been described in Section 5 and Appendix A.

Thank you for your detailed response! I appreciate your clarifications, but I still have a few follow-up questions:

  1. By "continuous optimization of the entire taxonomy," I mean optimizing the categories in the coarse-to-fine category system itself. For example, some categories might be redundant and could be merged.

  2. Since time measurements are affected by request latency, that's why I hope the authors can also report token consumption to better reflect the cost. Does "LLM input/output token consumption for every backbone and benchmark setting" refer to the total input/output tokens used to build the SkillWiki for each benchmark, or the average tokens per skill addition? Are Increments 1–3 three consecutive steps? Do you have results showing the complete process of building a SkillWiki on a benchmark? I believe visualizing this process in the paper would help readers better understand how the costs evolve.

Q1: category optimization in the coarse-to-fine category system

I understand your concerns. After the original construction or incremental construction of the knowledge base is completed, SkillsWiki will perform a wiki health check, which includes verifying the completeness of compilations, the accuracy of references and so on. A directory merging check is also included in this process. An agent is tasked with identifying redundant directories, and the merging operations are carried out by scripts. For example, the skills “biology/molecular-biology/gene-expression/tissue-specific-expression/tissue-specific-analysis” and “biology/genomics/transcriptomics/gene-expression/gene-expression-atlas” both deal with the topic of gene expression; therefore, “tissue-specific-analysis” is merged under “biology/genomics/transcriptomics/gene-expression/”. The references related to the skills whose directories have been adjusted will be updated accordingly, and summaries of the merged directories will also be regenerated by LLMs.

Q2: incremental construction experiment

In our response to W3, we reported the results of single-increment experiments across three skills. These were not continuous incremental tests, but rather three separate results, each obtained by incrementally adding one skill to a wiki library that already contained 36 skills. We further expand the complete skill-increment construction experiments on the ALFWorld benchmark with GPT-5.4-mini. Thirty-seven skills are continuously added to the same skill wiki database. As shown in the table, the number of input/output tokens does indeed show an increasing trend as skills are incrementally constructed. However, considering that the construction process is offline and the cost is not high, this is acceptable. In the subsequent version, we will plot this table as a line chart to help readers better understand the incremental construction of SkillsWiki.

#Time (s)Input tokensOut tokens
122.762,527971
222.783,061919
323.943,7761,169
425.953,8381,340
520.603,0741,078
635.993,237953
724.303,4101,222
818.833,6411,080
919.913,8751,213
1031.833,9941,238
1138.893,9751,184
1224.744,5821,413
1327.294,5841,391
1432.114,8301,429
1532.413,8701,151
1629.714,2151,215
1718.994,5231,193
1825.104,5061,237
1923.054,2451,235
2024.774,7181,619
2134.334,8381,309
2220.724,8731,184
2328.955,6261,585
2427.195,2401,334
2527.195,6051,720
2622.815,0561,219
2729.716,1081,697
2829.376,4081,663
2925.266,0431,693
3034.046,0061,826
3130.146,1091,765
3224.885,6621,606
3323.905,4941,467
3426.695,5751,392
3523.456,3171,672
3637.416,8781,590
3727.586,2881,706

Thank you for your detailed response! It addresses most of my concerns, but I still have doubts about the paper's novelty.

The paper mainly proposes organizing existing skills into a wiki structure to facilitate skill retrieval during inference. In your clarification of the differences from SkillNet, the main distinction also seems to be the wiki-based organization. To my knowledge, several works have explored hierarchical skill banks, and [1–2] also adopt wiki-like structures.

Moreover, since the paper focuses on skill retrieval during inference, I would expect comparisons with simple embedding-based retrieval methods. The discussion of related skill retrieval methods is also limited, and the experiments only compare against two baselines (ExpeL and SkillNet).

Nevertheless, I appreciate the authors' efforts. I will raise my score to 2, although I still lean toward rejection.

[1] SkillWiki: A Living Knowledge Infrastructure for Agent Skills. arXiv 2026.

[2] RESOURCE2SKILL: Distilling Executable Agent Skills from Human-Created Multimodal Resources. arXiv 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.