acceptodds
Under review as a conference paper at ICLR 2027

Hierarchical Skill Evolution for Coding Agents

Abstract

Skills provide agents with reusable experience to guide future decisions without modifying the underlying model. However, coding trajectories are often entangled with task- and repository-specific contexts, making it challenging to distill procedural skills that generalize across diverse coding environments. To address this challenge, we present Hierarchical Skill Evolution (), which progressively distills procedural knowledge from coding trajectories through hierarchical evidence aggregation. first extracts task-level evidence from individual trajectories, then aggregates recurring evidence within the same repository to form repository-specific skill candidates, and finally consolidates consistent patterns across repositories into transferable skills across repositories. The evolution is driven by an external writer–evaluator loop that generates and assesses skills while keeping the underlying coding model frozen. The loop externally updates the writer and evaluator policies to improve subsequent skill generation and assessment. This hierarchy progressively filters task-specific details while preserving procedures that generalize across diverse coding environments. Successful trajectories yield reusable guidance for future tasks, while failed trajectories reveal recurring failure modes that guide avoidance and recovery. We evaluate on SWE-bench Verified and DeepSWE, using GLM-5.2 across Pi and Claude Code harnesses and GPT-5.5 and Opus 4.8 with Pi. Across both benchmarks, 10 skills evolved solely from SWE-Gym improve coding performance across all evaluated settings, demonstrating transfer across benchmarks, coding models, and agent harnesses. We further evolve the skill library from target-benchmark trajectories without verifier feedback, relying solely on the learned evaluator for candidate assessment and achieving additional gains through benchmark-tailored skills.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Agents, Self Evolution

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_GenYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Hierarchical Skill Evolution (HSE), a framework for extracting reusable procedural knowledge from coding-agent trajectories without updating the underlying coding model. The main idea is to aggregate experience progressively: individual trajectories are first converted into task-level evidence, evidence from the same repository is then summarized into repository-conditioned candidates, and recurring patterns across repositories are finally consolidated into transferable procedural or recovery skills. A writer–evaluator loop controls candidate construction and promotion.

The paper evaluates HSE on SWE-bench Verified and DeepSWE. The static setting is the most convincing result: ten skills evolved from SWE-Gym are frozen and transferred to the target benchmarks, where they improve GLM-5.2 under both Pi and Claude Code. The same skill pack also improves GPT-5.5 and Opus 4.8 under Pi. The paper further reports target-benchmark adaptation, where the skill library is repeatedly updated using trajectories collected from the target benchmark.

Overall, I think the paper studies an important problem and the static-transfer results are promising. My main concern is that the experiments do not yet cleanly establish that the proposed hierarchical evolution mechanism, rather than more general trajectory aggregation and repeated target-set experience, is responsible for the strongest gains.

Critical strengths

The strongest aspect of the paper is that the same compact skill library improves several coding-agent configurations without modifying the solver. This is practically attractive and avoids conflating skill learning with model fine-tuning.

I also appreciate that the authors evaluate more than aggregate performance. The procedural/recovery decomposition, cross-harness transfer, and controller/hierarchy ablations are useful attempts to understand why the method works rather than only reporting a higher SWE-bench number.

Critical weaknesses
  1. The target-adaptation experiment should be interpreted as adaptation rather than generalization. The largest improvements in the paper come from the eight-round target-benchmark adaptation experiment. However, each round performs a complete rollout over the target benchmark and then updates the skill library from those trajectories. The resulting library is subsequently evaluated on the same complete target task set. It is useful that verifier outcomes are not exposed to the writer or evaluator, but the trajectories themselves still contain substantial information about the target tasks: issue descriptions, inspected code, attempted patches, tests, and failure modes. After several rounds, the system has therefore seen a considerable amount of target-task experience. I think this is a valid test-time or continual-adaptation experiment, but it is qualitatively different from the static transfer result. In its current form, it is difficult to tell how much of the very large Gate-8 improvement comes from reusable skill evolution versus repeated exposure to the same target task population. A cleaner experiment would construct skills from one subset of target instances and evaluate on a disjoint subset. A leave-one-task-out version would also be informative.
  2. The hierarchy ablation does not cleanly isolate the value of hierarchical aggregation. The main methodological claim is that repository-level aggregation provides an important intermediate abstraction between individual trajectories and cross-repository skills. However, the “w/o Hierarchical Aggregation” experiment randomly shuffles task trajectories across repositories.
    I am not convinced this is the right counterfactual. Random shuffling destroys meaningful repository structure and creates artificial groups, so a performance drop is not particularly surprising. What I would really like to know is whether the proposed three-stage hierarchy is better than a flat but otherwise matched aggregation strategy. For example, directly aggregate L1 task evidence into transferable skills using the same trajectories, writer, evaluator, and final skill budget. Without such a comparison, Table 4 shows that incorrect grouping hurts, but does not fully establish that repository-conditioned aggregation is necessary.
  3. The controller contribution is still difficult to attribute. The evaluator uses manually defined evidence features and fixed thresholds, with relatively simple weight updates over time. The writer also evolves through constrained rule changes. Yet the paper ablates “controller updates” only as a single block. This leaves an important question unanswered: is the gain coming from adaptive writer behavior, adaptive evaluator weights, or simply from a reasonably designed fixed set of aggregation rules? I would like to see the two components separated. A fixed-writer/adaptive-evaluator variant and an adaptive-writer/fixed-evaluator variant would make the methodological contribution much clearer. Sensitivity to the evaluator thresholds would also help, since these thresholds directly determine which skills enter the library. A smaller issue is statistical reporting. The main GLM experiments use three runs, which is helpful, but some differences against the strongest baseline are modest—for example, less than one percentage point on SWE-bench Verified in some settings. A paired per-instance analysis or bootstrap confidence interval over the improvement would make these comparisons easier to interpret. The GPT-5.5 and Opus 4.8 results would also benefit from uncertainty reporting.
Questions for authors

1.Can the authors evaluate target adaptation using disjoint adaptation and evaluation task pools, or otherwise ensure that a task cannot benefit from skills influenced by its own earlier trajectories?

2.Can the authors add a matched flat-aggregation baseline that removes the repository level without randomly corrupting repository membership?

3.Can the writer update and evaluator update be ablated separately? How sensitive are the results to the evaluator weights and the fixed accept/revise thresholds?

4.Are the GPT-5.5 and Opus 4.8 results based on one run? If so, can paired uncertainty or additional runs be reported?

Reviewer_sXbzOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Hierarchical Skill Evolution (HSE), which converts coding-agent trajectories into reusable procedural and recovery skills through task-level, within-repository, and cross-repository aggregation. An external writer–evaluator mechanism constructs and screens candidates while the underlying coding model remains frozen. The paper evaluates both static transfer of skills learned on SWE-Gym and subsequent adaptation using trajectories from the target benchmarks.

Critical strengths
  • The task-to-repository-to-cross-repository organization is intuitively motivated, and explicitly incorporating recovery guidance is practically relevant.
  • The frozen skill library improves performance across the evaluated benchmark–harness settings, with additional positive cross-model results.
Critical weaknesses
  • Figure 2 is poorly designed, with an overcrowded layout, inconsistent font sizes, and small labels that impede readability. The figure requires substantial revision to communicate the method clearly and meet the presentation standard expected of an ICLR submission.

  • The largest gains come from repeatedly adapting to the evaluated tasks. Table 2 updates the skill library using trajectories from the same benchmark tasks on which performance is reported, over eight rounds. These results do not demonstrate generalization to unseen tasks. Withholding verifier labels does not remove task-specific information from prior trajectories. Without baselines matched for rollout budget and access to previous attempts, the gains cannot be attributed specifically to HSE.

  • The experiments do not establish why the proposed hierarchy is necessary. The demonstrated skills largely contain standard advice about localization, minimal edits, and validation. The paper lacks essential controls: a comparable procedural checklist, direct trajectory-to-skill summarization, and flat iterative skill updating. Shuffling repository assignments is an inadequate substitute, particularly because candidate eligibility depends on repeated repository-specific signals. The current evidence establishes neither the necessity nor the superiority of the three-level design.

  • Despite providing scoring thresholds and step sizes, the paper does not adequately define cross-repository pattern matching, writer-rule updates, evaluator update directions, or attribution of regressions to individual skills. It also leaves unclear how verifier-dependent evidence features are computed during adaptation without verifier feedback. These are central mechanisms, not incidental implementation details.

Questions for authors
  1. How much of the adaptation gain remains on tasks whose trajectories never enter skill construction?
  2. Does HSE outperform direct trajectory summarization or flat skill updating when experience access, rollout budget, and injected context are matched?
  3. How are evaluator update directions, skill-level regression attribution, and verifier-dependent features during target adaptation computed?
Reviewer_LyF4OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Hierarchical Skill Evolution (HSE), which aggregates task-level trajectory evidence into repository-conditioned candidates and then into cross-repository procedural and recovery skills for frozen coding agents. It also updates the writer rules and evaluator weights used during skill construction. Several experiments are conducted to evaluate the frozen skills produced on SWE-Gym transferred to SWE-bench Verified and DeepSWE across agent harnesses and models, along with experiments on target-benchmark adaptation and component ablations.

Critical strengths
  1. The idea of using repository structure as an intermediate level to aggregate evidence into reusable repository-level patterns and cross-repository procedural skills is intuitively sound.
  2. The experiment results demonstrate that HSE improves over the No Skills baseline on two benchmarks across two agent harness settings with GLM-5.2. Cross-model validation experiments are also carried out with Opus 4.8 and GLM-5.2, demonstrating improvement over the No Skills baseline. These experimental results provide empirical evidence that the learned skills can bring benefit to different models.
Critical weaknesses
  1. The novelty of HSE looks incremental and is not well established by the current experimental results. Several prior works [1,2,3,4] have already studied reusable experience, failure-derived guidance, and evolving external contexts and their construction strategies. Given this context, HSE's distinctive contribution is mainly the repository-conditioned evidence aggregation and adaptive candidate assessment. This incremental contribution still needs stronger evidence to support it, as the comparison with the evaluated baselines does not provide clear evidence that the performance gain purely comes from the specific design. The authors are encouraged to include more baselines to provide clear evidence supporting the contribution. A very simple but informative controlled experiment would be generating procedural skills without trajectories, while using the same other conditions (e.g., the same writer model, format, approximate length, etc.). Since learning reusable skills could help coding tasks is a well-established mechanism in the community, it is worth testing whether the proposed skill learning method improves upon readily generated procedural guidance.
  2. Several important details of HSE are not reported. For instance, it remains unclear how task-level outcome transitions are assigned to individual candidates, and how writer-rule edits are generated. These details matter a lot because outcome changes may reflect stochastic execution or multiple skill changes. Besides, the hyperparameter selection procedure and sensitivity analysis are also missing. Moreover, the acceptance criterion for updated states on the validation set is also unclear. Note that the validation set contains only 26 tasks, so the state acceptance could be sensitive to stochastic variation in validation outcomes. The complete prompts, skills, and training settings are not reported either, barring reproduction and the detailed understanding of HSE. The training of HSE relies on positive/negative support and outcome transition labels, but how these features are contructed of replaced without verifier feedback during tthe adaptation is also unclear.
  3. The ablation study does not cleanly ablate the hierarchy and evaluator adaptation. The ablation study compares HSE with the method with randomly shuffled trajectories across repositories, supporting repository grouping over random grouping. But it does not establish the advantage over a reasonable non-hierarchical alternative. A direct task-to-skill aggregation under comparable settings (e.g., budgets, output sizes, etc.) would provide more information about the advantage of hierarchy. Besides, the controller updates variant ablates the writer and evaluator together, so the reported gains do not reflect the effectiveness of either update mechanism.

References

  1. Chen et at., AutoManual: Constructing Instruction Manuals by LLM Agents via Interactive Environmental Learning, May 2024
  2. Ouyang et al., ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory, Sep 2025
  3. Zhang et al., Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models, Oct 2025
  4. Ye et al., Meta Context Engineering via Agentic Skill Evolution, Jan 2026
Questions for authors

Please refer to the weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.