- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper identifies two limitations of SkillRL: continued training can plateau, and automatically distilled skills can become overfitted, conflicting, or incorrect. It proposes Skill2Task, which organizes skills into a typed relation graph and uses skill properties and relations to synthesize new tasks for continued RL. Experiments with Qwen3-4B and Qwen3-8B on ALFWorld and search-augmented QA report improvements over SkillRL on both seen and unseen evaluations.
Critical strengths
- The paper identifies an important and underexplored failure mode of skill-augmented agents: automatically accumulated skills can encode overgeneralized, conflicting, or invalid guidance. Experiments also show that such skills can directly impair task execution, offering a clear motivation for improving how learned skills are evaluated and reused.
Critical weaknesses
-
Skill2Task appears to rely heavily on environment-specific grounding, including permitted edits, legal goals, transition rules, source evidence, and a planner or blind solver for solvability verification. This assumption is well matched to ALFWorld and QA but may substantially limit applicability to open-ended environments where valid task modifications and automatic success criteria are unavailable. Moreover, the task-synthesis teacher receives rich, potentially privileged environment information, but the paper does not evaluate whether directly providing the same information to the acting agent would already yield comparable performance. Without this baseline, it is difficult to separate the benefit of task synthesis from that of introducing stronger environment-specific prior knowledge.
-
The motivation for grounding skill revision in executable tasks is reasonable, since environment feedback can provide evidence beyond teacher self-assessment. However, the paper does not compare against a direct skill-revision baseline in which the same teacher revises flagged skills without synthesizing and executing additional tasks. It is therefore unclear whether task synthesis is necessary, or whether most gains could be obtained through direct skill-bank refinement at substantially lower cost.
-
Graph construction relies on skill-text similarity and co-retrieval on the original training tasks to propose candidate skill pairs, but this crucial retrieval stage is neither specified nor evaluated. This may miss semantically dissimilar skills that are not retrieved together.
Questions for authors
- Are the baselines matched for total compute? Please report the additional RL training cost and the Teacher/solver cost for graph construction, task generation, and validation.
2 comments
Dear reviewer, the discussion period is about to close. Please read the authors' response and let us know briefly which of your concerns remain. If you have any further questions for the authors, please post them soon so the authors have time to respond.
We thank the reviewer for the thoughtful questions on environment grounding, direct skill revision, and retrieval coverage. We clarify the shared synthesis framework and the complementary contributions of policy learning and library refinement below.
Q1. Does grounding restrict applicability or introduce privileged information?
Thank you for the question. Skill2Task synthesizes training tasks in both ALFWorld and Web/Search, yielding 1,017 and 738 accepted tasks, respectively. ALFWorld instantiates executable goals in existing scenes; Web/Search constructs questions grounded in source material and validates answers through independent retrieval. These settings cover both environment interaction and information retrieval. The shared framework turns skill conditions and interactions into task requirements, which each environment instantiates and verifies. Its operation therefore extends beyond a simulator with a planner.
We would also like to clarify the role of grounding information. Permitted edits and legal-goal constraints define valid training instances; plans, reference answers, and source evidence support offline verification. These are task-construction and checking inputs, rather than additional solution instructions supplied to the acting policy. The policy learns through its own interactions and task rewards, and uses the standard observations and tools at evaluation. Giving an actor a description of the construction constraints does not supply the execution experience generated by practicing those tasks. This distinction explains why grounding is part of the synthesis interface, while executable practice drives policy learning.
Q2. Is task synthesis necessary, or would direct skill revision be enough?
Thank you for this suggestion. Figure 5(b) separates policy learning from library refinement through checkpoint/library swaps. With the original library fixed, our policy improves unseen performance over the SkillRL policy on both ALFWorld and Search. With our policy fixed, the final library further improves performance. Conversely, pairing the SkillRL policy with our final library still falls below the full Skill2Task agent. These comparisons show that both components contribute: supplying better guidance alone does not recover the gains from their joint evolution.
For a direct skill-editing comparison, we use the same strong teacher, GLM-5.2, to revise the SkillRL library by correcting recommendations and adding conditions or exceptions. The SkillRL policy checkpoint is held fixed; no synthetic tasks or additional RL are introduced. This isolates the effect of revising the library itself.
Direct library revision improves overall success from 80.5% to 81.4%, while full Skill2Task reaches 83.8%. Together with Figure 5(b), this comparison supports the complementary contribution of policy learning: Skill2Task updates the policy through executable practice and refines the library from the resulting experience.
Q3. Can pair retrieval miss useful skill relations?
Thank you for this question. Candidate pairs are proposed through both skill-text neighborhoods and task-level co-retrieval. The latter can introduce skills with different descriptions that are relevant to a shared task, so candidate construction is not restricted to semantically similar pairs. The teacher subsequently evaluates their conditions, behaviors, and effects to identify the relation. This combines semantic and task-contextual relevance to identify useful skill interactions for task synthesis.
Q4. Are total compute budgets matched?
Thank you for asking about total compute. The SkillRL-continuation and Skill2Task-continuation comparisons match the policy-training setup: the same starting checkpoint, eight GPUs, learning rate 1e-6, rollout counts, episode limits, and optimizer steps. Skill2Task additionally incurs offline teacher and solver work. Across ALFWorld and Search, the measured totals are 2,457 teacher API calls and 1,948 solver API calls. We report this additional API usage separately from policy RL; the matched budget refers to policy training, rather than total end-to-end compute.