- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SkillSeg, a framework for improving agent skills through segment-level feedback. It compares action likelihoods with and without the active skill, and combines this with the episode outcome to decide whether a skill should be reinforced or revised. Experiments on ALFWorld, WebShop, and AppWorld show improvements over several skill optimization baselines.
Critical strengths
- Good empirical results. The method has strong results on ALFWorld and AppWorld compared with several existing baselines.
- Useful feedback ablations. The paper compares different feedback strategies, including trajectory-level labels, segment-level labels, and likelihood-based feedback. These comparisons are helpful.
Critical weaknesses
-
The feedback attribution mechanism is not very convincing.The likelihood contrast only measures whether a skill makes the recorded actions more likely, not whether those actions are actually useful. For example, a helpful skill in a failed episode can still receive negative feedback. I don't think this really solves the credit assignment problem
-
The main scoring rule is quite simple and not sufficiently justified. For binary rewards, the sign of the historical advantage is mostly determined by the episode success or failure, so the historical baseline does not really change the feedback polarity. Also, the improvement over the C-only baseline is relatively small (85.07% vs. 83.08% on ALFWorld). It is unclear how much the proposed score actually contributes beyond simpler alternatives.
Questions for authors
- How can the likelihood of recorded actions reliably indicate whether a skill is helpful or harmful? (You could measure this by LLM-Judge)
- Could you compare with simpler attribution rules, such as using a fixed baseline instead of the historical advantage?
- Could you provide direct evaluation of attribution quality and more feedback ablations on WebShop and AppWorld?