acceptodds
Under review as a conference paper at ICLR 2027

Skill Tree: Hierarchy-Guided Experience Consolidation for Self-Evolving Agent Skills

Abstract

Large language model (LLM) agents increasingly improve through reusable skills distilled from execution experience. Multi-trajectory skill evolution enables broader evidence aggregation, but many existing methods rely on LLM-generated consolidation, which can alter or omit useful details while leaving semantic redundancy unresolved. We propose Skill Tree, a framework that instead groups edits by their structural context, explicitly removes redundant guidance, and preserves useful content without generative rewriting. Skill Tree further organizes accumulated knowledge into a budgeted core skill and supplementary references, reorders core content under importance and workflow constraints, and retrieves task-relevant reference subtrees at inference time. Experiments across skill construction, out-of-distribution generalization, and cross-model reuse show substantial improvements in downstream task performance, with controlled ablations further demonstrating the importance of structure-aware consolidation, organization, and access.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agent Skill

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_dg5SOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Skill Tree uses Markdown hierarchy to consolidate trajectory-derived skill edits without rewriting retained content. It combines scoped deduplication, core/reference allocation, constrained reordering, and task-specific reference retrieval. Evaluation covers spreadsheet tasks, frozen-skill transfer to WikiTableQuestions, cross-model reuse, and separately evolved DocVQA skills.

Critical strengths
  1. Using the same document hierarchy to consolidate, organize, and retrieve experience is a coherent response to detail loss during skill rewriting. Eight component and structural ablations directly support the design, beyond the auxiliary similarity metrics.
  2. With the same trajectory pool as several strong baselines, Skill Tree leads on all four held-out Vrf settings, including gains above eight points in both Creation settings. Frozen WikiTQ evaluation and cross-family reuse provide additional evidence that the skills transfer beyond their construction tasks.
Critical weaknesses
  1. Appendix A.9 says domain priorities, workflow constraints, routing weights, and intent rules are fixed, but does not list them. These choices affect the actual algorithm, so their omission limits reproducibility and makes the effort needed for a new domain unclear.
  2. Full SpreadsheetBench Soft/Hard scores include the 200 evolution tasks. They mix seen-task improvement with generalization and should be separated when discussing robustness. This limits those particular claims, while the independent Vrf and frozen WikiTQ results remain meaningful.
  3. Actual context and construction costs are missing. A 1,500-line core and 40,000-character reference allowance do not reveal token exposure across methods. The existing structural controls support effectiveness, but users still need the performance–cost tradeoff to judge practical value.
  4. Embedding-based retention favors unchanged wording and cannot establish preservation of every condition or formula. NLI deletion errors and conflicting edits also need discussion. These gaps limit the mechanism explanation; downstream accuracy and the deduplication ablations already provide stronger evidence of usefulness.
  5. The cited ACE already uses localized edits and deterministic merging, and SkillReducer separates core guidance from references. Positioning should emphasize the shared hierarchy and its tested benefits, rather than imply that non-rewriting consolidation or core/reference separation is itself new.
  6. Minor corrections: resolve “Section ??” in Appendix A.6 and change “Subtrees form Reference File” to “Subtrees from Reference File” in Appendix A.12. Add uncertainty to the three-seed means, especially for the small DocVQA margins.
Questions for authors
  1. Can you provide the exact prompts and domain-specific rules, explain how they were selected, and identify what changed between spreadsheets and DocVQA?
  2. What are Soft/Hard scores excluding evolution tasks, and is the DocVQA split disjoint by document identity as well as question–image pair?
  3. What are the actual construction costs and execution-time prompt tokens for Skill Tree and the main baselines?
  4. How are contradictory edits or competing replacements handled, and what deletion errors appear in a small audit of existing NLI decisions?
2 comments

Thank you for the additional experiments. The held-out SpreadsheetBench results and document-disjoint DocVQA evaluation address my main concern about generalization. I am raising my rating from 2 to 3 (Weak acceptance).

Thank you very much for taking the time to revisit our rebuttal and for your support. We sincerely appreciate your thoughtful feedback and consideration!

Reviewer_p8JSOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2(+1)
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Skill Tree, a framework for updating agent skills from execution trajectories. It represents Markdown skill documents as trees based on their headings. The method groups edits by section, removes redundant content, and splits the remaining content between a main skill document and reference files. It then reorders the main document and retrieves relevant reference sections for new tasks. The authors evaluate the framework on spreadsheet and document question-answering tasks, including cross-model and out-of-distribution tests.

Critical strengths
  1. The paper is well organized and well written.
  2. The experiments are extensive, including multiple benchmarks, model scales, ablation studies, and analyses of synthetic data quality.
Critical weaknesses
  1. The novelty of this paper is limited. Its tree comes from the headings of an existing Markdown document. Most steps involve grouping edits by section, moving content between documents, changing section order, and retrieving relevant sections. It is unclear what the tree adds beyond keeping section labels and heading paths for these operations. The structural ablations do not fully answer this question. For example, replacing sections with paragraph chunks also changes how content is grouped and how much context is kept. The gains may therefore come from keeping related content together, rather than the tree structure itself. A stronger comparison should keep the same section context and input budget while using simpler grouping and retrieval rules.

  2. Several key design choices rely on fixed rules that are not sufficiently tested. For example, the method scores a section using its number of supporting cases and reference links, then uses the highest node score to rank the whole subtree. More reference links do not necessarily mean that a section is more useful. They may simply reflect how the document was split into files. Likewise, one useful node does not necessarily justify moving all the other content in its subtree forward. The reordering ablation only shows that the proposed order works better than the original order. It does not show that these scoring choices work better than simpler rules, such as using support counts alone or taking the average node score.

  3. Tree Merge focuses on removing repeated advice, but this does not ensure that the remaining advice is correct or useful. An edit can add new information while still being wrong or only applying to one training task. The described pipeline does not include a separate execution-based test of whether an edit improves task performance before keeping it. Keeping edits unchanged can therefore preserve errors as well as useful details. Also, edits are only compared within the same section, and the paper does not explain how conflicting advice across different sections is handled. The method needs a clearer check of experience quality, rather than mainly checking whether the experience is repeated.

  4. The paper provides limited examples of the actual skill tree and how it is used. Figure 1 shows small spreadsheet tips, while Appendix A.12 gives a few references about formula recalculation, FIFO processing, and string cleaning. These tips may be useful, but they do not show what the full tree contains or how its structure helps the agent. A complete case should show the original experience, the extracted edits, the tree before and after the update, and the content retrieved for a new task. It should also show how that content changes the agent’s actions. Without this, it is difficult to tell whether the gains come from the proposed organization or simply from giving the agent more instructions.

Questions for authors

See weaknesses

Reviewer_ZG7BOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses multi-trajectory skill evolution for LLM agents. It argues that LLM-based merging of trajectory-derived patches, as in Trace2Skill, ties two decisions together: what to keep, and how to rewrite it. This coupling can drop details while leaving semantic redundancy in place. Skill Tree instead uses the Markdown heading hierarchy of the skill document as a shared structure across the skill lifecycle, The framework has four modules. Tree Merge resolves each candidate patch to a target heading node and groups patches by target node, subject to operation and anchor compatibility. Within each group, natural language inference judgments of equivalence and entailment remove redundant edits, and the surviving edits are applied verbatim. Tree Shaping splits the consolidated skill into a persistently loaded main skill under a line budget (B_skill = 1500) and a reference tree. A greedy planner externalizes and then restores allocation groups according to heuristic retention priorities, keeping compact reference entries in the main skill. Tree Reordering permutes sibling subtrees of the main skill by a priority score, subject to structural and workflow constraints. The score combines log-normalized trajectory support and reference connectivity, maximized over each subtree. Finally, Inference-time Routing retrieves heading-rooted reference subtrees by deterministic weighted lexical matching, with an operation-intent supplementation step, under a per-task character budget (B_ref = 40,000), and injects them alongside the main skill before the first model call.

Critical strengths
  1. A deterministic, traceable pipeline with little dependence on the LLM. Outside patch distillation and the NLI (natural language inference) judge, every stage is deterministic: grouping, budgeted shaping, constrained reordering and lexical routing. Provenance from training cases is carried through all stages. As a result, the evolved skill can be audited: each retained instruction traces back to the trajectories that produced it, and routing adds no model calls at inference time.

  2. A well-isolated design principle, backed by the right control experiments. The paper separates the decision of which experience to retain from the regeneration of retained content. It restricts redundancy removal to structurally compatible contexts, meaning edits that target the same heading node with compatible operations and anchors. Prior multi-trajectory methods (Trace2Skill, SkillCAT, SkillOpt) fold both decisions into LLM-based merging. This framing gives a concrete, testable alternative, grounded in known context-collapse effects of iterative rewriting.

The structural ablations make the claim falsifiable. They keep each module’s function but remove its use of the hierarchy. Scope-agnostic deduplication drops Vrf (held-out SpreadsheetBench-Verified accuracy) by 9.5 points. Paragraph-level shaping units drop it by 12.5, and paragraph-chunk routing by 12.0.

Critical weaknesses
  1. The gains are not yet shown to come from held-out, statistically reliable evaluation. The largest claimed improvements, +8.8 to +14.5 points, are on the Hard metric, which is computed on the "full SpreadsheetBench" (912 tasks). SpreadsheetBench-Verified is a subset of this benchmark, and its first 200 tasks are used for skill evolution. The paper never states that these tasks are excluded from the Soft/Hard evaluation. The method’s design makes overlap particularly consequential: patches are applied verbatim, which plausibly preserves instance-specific details from training trajectories. The result pattern is consistent with this. In 35B Deepening, Soft is essentially tied with SkillCAT (39.11 vs. 39.02) while Hard is +12.07, and the method’s Hard/Soft ratio (~0.84 vs. ~0.53 for baselines) holds across every block.

On the clearly held-out Vrf split, the test set has only 200 tasks, so one standard error is about 3.4 points. Several margins fall within about one standard error, and the DocVQA gains are roughly 16 of 2,649 examples. No variances or paired tests are reported, and seed averaging appears inconsistent across methods. The skill budget B_skill also seems to be selected on the test split.

This is the main condition on my positive assessment. If Soft/Hard on tasks disjoint from evolution, together with variance estimates and validation-based hyperparameter selection, preserve a clear advantage, I am comfortable recommending acceptance; if the gains aren't substantial enough, I would lower my score.

  1. The evidence does not attribute the gains to the paper’s core mechanism. The paper’s central thesis is that structure-constrained filtering should replace rewriting-based merging. Yet the "w/o Tree Merge" variant, which substitutes the Trace2Skill merge, reaches 55.0 Vrf, while Trace2Skill itself reaches 29.67 in the same setting. So roughly 25 of the ~35-point gap over Trace2Skill comes from Shaping, Reordering and Routing, not from avoiding rewriting. Those three stages are themselves largely hand-specified: predefined domain priorities, operation intents, workflow-stage relations and field weights, none of which are listed.

Two further gaps compound this. Context budget is uncontrolled: Skill Tree exposes up to 1,500 lines plus 40,000 characters of routed references, and the baselines' skill or prompt lengths are never reported. The mechanistic metrics are also partly circular. Embedding-based retention rewards verbatim copying by construction, and grouping by target node raises semantic cohesion by definition.

This does not undermine the practical value of the full pipeline, but it weakens the paper’s central scientific claim as framed. I would like to see the contribution reframed around the combined structure-aware pipeline, or the consolidation principle isolated with a context-matched comparison.

Questions for authors
  1. Are the 200 SpreadsheetBench-Verified evolution tasks excluded from the "full SpreadsheetBench" (912-task) Soft/Hard evaluation? If not, please report Soft and Hard for all methods on tasks disjoint from the evolution set. Given the unusual Hard/Soft ratio (~0.84 vs. ~0.53 for baselines), this is the single most important question for my assessment.

  2. Please report mean ± standard deviation, or paired significance tests, for all methods in Tables 1–3 and Figure 3. What does each seed vary: skill construction, agent sampling, or both? Some baseline values (e.g., 70.33, 29.67) look averaged and others (e.g., 55.50, 61.50) do not; which baselines were run with three seeds? Are the Table 3 ablations single runs, and on which model and setting?

  3. "w/o Tree Merge" (Trace2Skill merge plus your Shaping, Reordering and Routing) reaches 55.0 Vrf, against 29.67 for Trace2Skill alone. How much of Trace2Skill’s deficit comes from skill length or organization rather than rewriting? Please also explain the −13.0 drop from removing sibling reordering alone, which changes no content and leaves the variant below the SkillCAT baseline.

  4. What are the average main-skill length and per-task prompt tokens for each method? Can you provide a comparison at a matched context budget?

  5. Several baseline numbers look anomalous to me. In the 35B setting, the Human-Written skill scores 9.67 Vrf (vs. 48.33 at 122B) and Trace2Skill scores 29.67. In Table 2, the SkillCAT and No-Skill WikiTableQuestions (WikiTQ) scores are identical (79.61). Can you explain these values, and reconcile the Trace2Skill numbers with those reported in the original paper under the same models?

  6. Please list the predefined domain priorities, operation intents and workflow-stage relations. How were they derived, and were they redefined for DocVQA?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.