acceptodds
Withdrawn from ICLR 2027

Ecological Skill Libraries: Co-Evolving Skills, Routers, and Library Topology

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language model agents increasingly rely on reusable skills that encode procedural knowledge, executable operations, and task-specific guidance. As skill libraries grow, their performance depends on skill quality, routing competition, context consumption, redundancy, conflicts, and complementary interactions. Existing approaches primarily optimize skills and routing decisions separately, which can misestimate each skill's marginal contribution and reduce library performance when the candidate population changes. We introduce EcoSkill, an ecological framework that models a skill library as a co-adapting community. Given a task distribution and an execution budget, EcoSkill jointly evolves skill content, applicability triggers, routing policies, library topology, and lifecycle decisions. Its central mechanism assigns ecological credit based on each skill's marginal contribution to community performance, incorporating task utility, context and routing costs, unique coverage, and inter-skill synergy. The framework estimates this credit using sampled coalition counterfactuals and approximate Shapley-style attribution, avoiding exhaustive evaluation of skill coalitions. The resulting signal coordinates skill mutation, retention, migration, merging, and extinction while preserving specialized and complementary capabilities. We evaluate EcoSkill across libraries of increasing scale, controlled skill corruptions, and task-distribution shifts using task performance, routing accuracy, context consumption, and conflict rate. This formulation maintains adaptive skill libraries by selecting members according to their contribution to the agent ecosystem.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Skill Library Evolution, Ecological Credit Assignment, Multi-Agent Skill Routing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_UQVvOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EcoSkill jointly evolves skills, routing policies, and library structure. It estimates skill contributions through sampled coalitions and evaluates performance under library growth, skill corruption, and task-distribution shifts.

Critical strengths

Evaluating skills through their interactions addresses a practical weakness of independent skill ranking. The ablations examine several relevant components.

Critical weaknesses
  1. Some averages in Table 1 do not match the scores for the three task groups. This makes the reported gains hard to check.

  2. It is unclear how skill scores are computed when a group of skills exceeds the context limit. This could affect which skills are kept or removed.

  3. The tests use only one base model and libraries of up to 500 skills. It is unclear whether the gains hold for other models or larger libraries

Questions for authors
  1. How are the averages in Table 1 calculated? Could you explain why they differ from the individual results?

  2. What happens when a sampled group of skills exceeds the context limit? Are skills removed or shortened, or is the group skipped? How does this affect their scores?

  3. Do the gains hold with another base model or a larger library? How does the cost of updating the library grow as more skills are added?

Reviewer_AjZCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces EcoSkill, a framework for managing reusable agent skills as an interacting community. With the base agent fixed, it jointly adapts skill content, applicability triggers, routing, directed topology, and lifecycle decisions. Ecological credit combines sampled coalition marginal utility with routing interference, unique coverage, and positive pairwise synergy. Experiments use three task suites with 300 tasks in total and libraries containing up to 500 available skills. At 250 skills, the paper reports 52.5% mean success for EcoSkill, alongside lower instruction conflict and context consumption. It also evaluates controlled library corruption, distribution shifts, component ablations, and sampling-budget sensitivity.

Critical strengths

1.The paper addresses a practical problem that standalone skill ranking misses: conditional utility, redundancy, competition for retrieval, and conflicts within a finite context budget.

  1. Joint adaptation of content, triggers, routing, topology, and lifecycle offers a coherent approach to library maintenance, rather than treating retrieval as a fixed preprocessing step.

3.The evaluation includes useful ablations, small-library attribution calibration, corruption tests, and shift recovery. The reported context reduction from 3.4k to 2.0k tokens relative to Independent-TopK is practically relevant.

4.Hardware, model, router, inference limits, prompt templates, and representative workflows are documented in the appendix.

Critical weaknesses

1.Figure 1 primarily depicts routing and selecting skills from the library for task execution. It does not clearly communicate the paper’s central co-evolution process described in the introduction and illustrated in Figure 2. As a result, Figure 1 may give readers the impression that EcoSkill is mainly a skill-retrieval method, rather than a framework for jointly adapting the skill ecosystem.

2.Aggregate results are not adequately explained. For EcoSkill, the three Table 1 suite scores (56.1, 49.0, 51.9) average to approximately 52.33%, rather than 52.5%. For EvoSkill-Style, (53.7, 46.2, 49.4) average to approximately 49.77%, rather than 49.3%. Weighting by suite task counts also does not reproduce the reported aggregates. This affects the stated 3.2-point primary improvement and requires corrected values or an explicit aggregation procedure.

3.The estimator is insufficiently specified under the deployment constraints. Full permutation prefixes can violate the eight-skill and token limits, and random prefixes need not include prerequisites. Unique coverage uses maxima over coalitions, while synergy requires four related coalition utilities; the appendix does not fully explain how these are estimated from the shared prefix record. Exact calibration is confined to ten-skill libraries.

4.Routing recall is described in the main text relative to skills used by successful reference trajectories, but Appendix B.2 defines a task-level indicator for retrieving at least one verifier-confirmed relevant skill. These are different metrics. The construction of the reference set and the reliability of conflict annotations also need clarification, especially since Appendix D.8 provides model-based annotation prompts.

  1. Reproducibility and external validity remain limited. The paper gives many settings but no complete task inventory or executable benchmark artifacts in the supplied manuscript. The approximately 90 held-out tasks, one backbone, and controlled corruption processes leave uncertainty about independent real-world libraries and higher-order interactions.

6.The reported 38.4 aggregate GPU-hours for evolution should be reconciled with the rollout counts implied by 64 task-permutation samples, n+1 evaluations per permutation, and up to 30 generations. A cost breakdown and an equally budgeted simpler alternative would help establish whether ecological credit is worth its overhead.

7.The nearest-16 restriction for synergy and several credit weights and lifecycle thresholds are important design choices. Sampling-budget and context-budget sensitivity are useful, but robustness to these other choices is not established.

Questions for authors

1.Could the authors revise Figure 1 to make the full EcoSkill co-evolution loop explicit? In particular, how does ecological credit feed into updates to skill content and triggers, routing, topology, and lifecycle decisions, and how are these updates evaluated and fed back into subsequent generations?

2.How is Avg. SR in Table 1 computed?

3.How are permutation-prefix coalitions evaluated when they exceed the token or eight-skill limit, omit prerequisites, or contain cycles? What distribution over feasible coalitions is used in practice?

4.Please provide executable estimators for unique coverage, interference, and synergy, including missing coalition values, additional rollouts, and sampling reuse. Can attribution calibration be extended beyond ten skills?

  1. Which routing-recall definition produces the tables? How are reference relevant skills obtained, and how are model-based conflict annotations validated independently?

  2. Can the task inventory, family splits, verifiers, seed/evolved skill libraries, and evaluation traces be released? Can results be tested on another backbone and an independently sourced library?

Reviewer_jeBeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents EcoSkill, a framework for managing evolving skill libraries by jointly optimizing skills, routing, topology, and lifecycle decisions. The main idea is to evaluate skills based on their contribution to the overall library rather than their standalone performance, using a Shapley-inspired credit assignment mechanism. The reported results are promising, particularly for larger libraries and under distribution shifts.

Critical strengths

The paper addresses a relevant problem. As skill libraries grow, redundancy, routing interference, and complementarity become increasingly important.

I like the idea of considering skill interactions when making evolution and retention decisions, rather than treating each skill independently.

The evaluation is reasonably comprehensive, covering different library sizes, corrupted skills, distribution shifts, and ablations.

Critical weaknesses

My main concern is the clarity and consistency of the proposed credit estimation procedure.

  1. Coalition feasibility. Algorithm 1 constructs coalitions by progressively adding skills from a permutation of the entire library. However, the implementation limits each coalition to eight skills and imposes a context budget. It is unclear how the algorithm handles coalitions that exceed these limits, or whether the actual estimator differs from the one described.
  2. Computational cost. The reported estimation cost is also difficult to reconcile with Algorithm 1. For example, 64 permutations over 10 skills would require 704 utility evaluations, whereas Table 7 reports a relative cost of only 0.125. It would help to clarify the actual number of rollouts and how the additional credit components are estimated.
  3. Experimental consistency. I noticed a few discrepancies in the results. Some averages in Table 1 do not match the per-task numbers, Figure 3 appears to have inconsistent method labels, and Table 9 reports an average of 8.1 retrieved skills despite the stated maximum of eight. These may be reporting issues, but they should be checked.
  4. Is the full framework necessary? The improvements are encouraging, but I would like to see a stronger comparison with simpler diversity-aware or context-matched retrieval methods. This would help establish how much of the gain comes from ecological credit assignment rather than more selective retrieval.
Questions for authors

See weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.