acceptodds
Withdrawn from ICLR 2027

SkillSeg: Self-Evolving Agent Skills via Feedback Attribution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Language agents can improve without updating model parameters by revising an external library of reusable skills. However, when multiple skills are invoked within an episode, its terminal outcome does not specify whether the behavior associated with each skill provides evidence for retaining or revising that skill. We propose SkillSeg, a framework for skill-level feedback attribution over trajectory segments, each consisting of consecutive actions executed under the same active skill. For each segment, SkillSeg compares the likelihood of the observed actions under matched prompts with and without the active skill, then combines this contrast with the episode outcome relative to a historical baseline. The resulting sign routes the segment to positive or negative reflection for the corresponding skill, and a likelihood-ratio gate screens the proposed library update. We evaluate SkillSeg with frozen Qwen3.5-9B agents on ALFWorld, WebShop, and AppWorld. SkillSeg achieves 85.07 ± 1.49% success on ALFWorld, compared with 79.10 ± 1.49% for a matched segment-level baseline that directly inherits episode labels. It also achieves 36.51 ± 2.48% success on AppWorld and a score of 55.48 ± 3.64 with 20.00 ± 1.00% success on WebShop, attaining the highest means among the completed same-backbone methods across these benchmarks. These results support skill-level feedback attribution as a useful mechanism for guiding reusable skill evolution with frozen language models.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
language agents, skill evolution, segment feedback, credit assignment, frozen LLMs

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 3 reviews, above 0% of the others.
Reviewer_r8ZrOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SkillSeg, a framework for improving agent skills through segment-level feedback. It compares action likelihoods with and without the active skill, and combines this with the episode outcome to decide whether a skill should be reinforced or revised. Experiments on ALFWorld, WebShop, and AppWorld show improvements over several skill optimization baselines.

Critical strengths
  1. Good empirical results. The method has strong results on ALFWorld and AppWorld compared with several existing baselines.
  2. Useful feedback ablations. The paper compares different feedback strategies, including trajectory-level labels, segment-level labels, and likelihood-based feedback. These comparisons are helpful.
Critical weaknesses
  1. The feedback attribution mechanism is not very convincing.The likelihood contrast only measures whether a skill makes the recorded actions more likely, not whether those actions are actually useful. For example, a helpful skill in a failed episode can still receive negative feedback. I don't think this really solves the credit assignment problem

  2. The main scoring rule is quite simple and not sufficiently justified. For binary rewards, the sign of the historical advantage is mostly determined by the episode success or failure, so the historical baseline does not really change the feedback polarity. Also, the improvement over the C-only baseline is relatively small (85.07% vs. 83.08% on ALFWorld). It is unclear how much the proposed score actually contributes beyond simpler alternatives.

Questions for authors
  1. How can the likelihood of recorded actions reliably indicate whether a skill is helpful or harmful? (You could measure this by LLM-Judge)
  2. Could you compare with simpler attribution rules, such as using a fixed baseline instead of the historical advantage?
  3. Could you provide direct evaluation of attribution quality and more feedback ablations on WebShop and AppWorld?
Reviewer_ARfWOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

SkillSeg updates textual skill libraries while keeping the language models frozen. It combines action-likelihood differences with episode outcomes to assign segment-level feedback, then screens proposed edits using a gate adopted from Skill-Pro. Experiments cover ALFWorld, WebShop, and AppWorld.

Critical strengths

The matched ALFWorld ablation shows an improvement over assigning episode labels to individual segments. The AppWorld results also suggest some value.

Critical weaknesses
  1. Limited novelty over Skill-Pro. Skill-Pro already analyzes skill-controlled trajectory segments, generates textual revisions through hindsight attribution, and screens candidates with a PPO-style gate. SkillSeg mainly adds a with/without-skill likelihood contrast combined with episode outcomes to organize positive and negative reflection. I find this an incremental contribution rather than a substantial methodological advance.

  2. Insufficient experimental validation. The controlled feedback ablations are limited to ALFWorld; equivalent comparisons are missing on WebShop and AppWorld. The additional-model experiments report only SkillSeg results, without corresponding baselines. While the ALFWorld ablations provide useful evidence, the experiments do not sufficiently establish whether the proposed feedback mechanism offers consistent benefits across environments and models.

Questions for authors

see above

Reviewer_3mUmOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces SkillSEG to propose a framework for skill-level attribution and targeted optimization, based on an agent’s execution trajectories and task feedback. The problem addressed is of significant practical importance, as the quality of external skills directly impacts an agent’s long-term execution capabilities. While the paper introduces a plausible feedback-attribution mechanism for a meaningful problem and achieves some performance gains, it lacks a thorough discussion regarding its underlying attribution assumptions, the stability of generalization during continuous evolution, and its scalability within large-scale skill libraries.

Critical strengths

S1. SkillSeg addresses a well-chosen problem, the technical contributions are highly reproducible, and the algorithm presented in the paper is clear and concise.

S2. The experimental details provided are quite comprehensive. Several representative baselines were selected for analysis, and multiple experimental runs were conducted to calculate the variance.

Critical weaknesses

c1. The contribution of the core methodology is limited and lacks empirical validation. The central claim involves an attribution mechanism based on skill-level segments, the actual evaluation relies solely on global task outcomes. In particular, skill attribution and skill optimization lack both direct comparisons and an explanation of the fundamental differences.

C2. the paper fails to discuss the optimization of the skills themselves—specifically, whether such optimizations might compromise a skill’s inherent generalizability, particularly when applied to previously unseen tasks.

C3. Although skill optimization occurs at the segment-level, modifications are applied at the library level. This may mask localized, detrimental updates, and consequently, there is insufficient granular discussion regarding the evolution of skills. For example, based on SkillSeg, It is possible for a Skill to correctly execute its assigned Segment, yet still receive negative feedback because the overall task fails due to subsequent actions by other skills.

C4. The current skill library is quite small. The primary experiments begin with a library of only 6 general-purpose skills, which is insufficient to demonstrate the method’s effectiveness in the context of a large-scale skill library. Furthermore, most of the experiments assessing generalizability are limited to one-way cross-task evaluations.

C5. The diagrams illustrating the methodology show clear signs of having been generated by AI, making it difficult to intuitively grasp the proposed methodology.

Questions for authors

Please mainly respond to C1-C5.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.