acceptodds
Withdrawn from ICLR 2027

SkillRise: Agentic Reinforcement Learning for Cross-Task Skill Evolution

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Large language model agents often encounter related yet distinct tasks that share reusable solution patterns. Yet standard agentic reinforcement learning treats tasks as independent episodes, while existing approaches to skill learning either focus on repeated attempts of one task or use pipelines with multiple stages that entangle extraction, retrieval, and execution. We introduce SkillRise, a unified reinforcement learning framework for learning skills across tasks. SkillRise organizes related instances into progressively challenging sequences and uses a single policy to alternate between task solving and curating an evolving skill document passed directly to the next task. Decoupled credit assignment across tasks supervises solving with the current task outcome and curation with discounted downstream outcomes. Experiments on ALFWorld, WebShop, and ScienceWorld show that SkillRise achieves the strongest Pass\@1 performance among the compared methods, with gains over the strongest baseline ranging from 2.3 to 8.5 percentage points. Although trained across distinct tasks, its learned curation policy remains effective for repeated attempts on the same task. Further analysis reveals scaling at test time across tasks: performance improves with longer sequences of related tasks even when each task is attempted only once. This trend suggests that SkillRise reuses transferable skills across tasks rather than benefiting from repeated sampling of the same task. SkillRise further retains strong performance while substantially reducing the runtime overhead of skill learning pipelines with multiple stages. Together, these results provide a simple and efficient training paradigm for LLM agents to extract, refine, and reuse transferable skills across tasks.

Primary area:
reinforcement learning
Keywords:
Reinforcement Learning, LLM Agent,Agent Skill

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_RRtyOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes SkillRise, an end-to-end reinforcement learning framework for cross-task skill learning. A shared policy alternates between solving related tasks and updating an evolving skill document, with current-task rewards supervising task solving and discounted downstream rewards supervising skill curation.

Critical strengths
  1. The paper presents a clear and meaningful formulation for enabling agents to accumulate and reuse transferable skills across related but distinct tasks.
  2. SkillRise provides a simple and unified framework that uses a single policy for task solving and skill curation.
Critical weaknesses
  1. The related-work analysis is incomplete, making the novelty difficult to assess. Since the main contribution concerns transferable skill learning, the paper should position itself more carefully against the most relevant literature. SkillOS is only briefly discussed, while closely related works such as D2Skill, Skill1 are omitted. Without a detailed comparison of their problem settings, skill representations, training objectives, and transfer mechanisms, the paper’s contribution and technical advancement remain unclear.
  2. The baseline selection is not well aligned with the paper’s central claim. GiGPO and LaMer mainly represent general agentic RL or Meta-RL approaches, whereas the proposed method focuses on skill learning and transfer. The absence of strong skill-learning baselines weakens the main empirical comparison. Moreover, the reported results of 85.9% on ALFWorld and 84.4% on WebShop appear below recently reported state-of-the-art results, which further limits the evidence for the method’s competitiveness.
  3. The within-task adaptation experiment does not isolate the source of the reported generalization. Because most baselines do not explicitly learn or maintain transferable skills, the experiment cannot determine whether the improvement comes from the proposed cross-task skill-training mechanism or simply from having an explicit skill document. Comparisons with stronger skill-based methods or appropriate skill-memory controls are needed.
  4. The ablation study does not convincingly establish the importance of skill curation. The training curves of SkillRise and the no-curation variant are largely similar, with only a small difference that may plausibly result from random-seed variance. This result appears weaker than the paper’s claim that skill curation is central to the framework.
  5. The efficiency comparison may be unfair because RetroAgent and SkillRL are inherently multi-stage systems, whereas SkillRise contains only two main processes. The resulting runtime advantage is therefore largely expected, and the comparison excludes GiGPO and LaMer. A more informative evaluation would report both training time and success rate for all major baselines, for example in a performance-versus-compute scatter plot, to demonstrate whether SkillRise offers a genuine Pareto advantage.
Questions for authors

Please refer to the weakness

Reviewer_pW1fOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents an reinforcement learning method that incorporates both skill induction from similar tasks and acting on future tasks based on skills inducted from previous tasks into the same model policy. Specifically, the same LM policy is used to generate / revise an accumulated skill document SiS_i from Si−1S_{i-1} and task ii, which is then used to solve task i+1i+1. The reward for the task solving prompt is standard task reward, and the that for the skill inducing prompt is a discounted future reward for subsequent tasks.

Critical strengths
  • The paper demonstrates experiments over three datasets, ALFWorld, WebShop, and Science World to show gains on Pass@1-3 over baseline methods, including skill-free prompting baselines, skill-free RL baselines, and one meta-RL baseline.
  • Sensitivity analysis shows that the method is robust to some hyperparameters introduced (e.g., the discount factor in the skill authoring reward).
Critical weaknesses
  • The main quantitative results lack statistical testing. The evaluation set used in all experiments for all datasets in this paper contain only 128 examples, and the absolute gains against the second best system ranges from 0.8-8.5% in most cases, which requires only 1-11 examples flipping predictions at this scale. The same issue plagues not only the main results, but also the ablative studies. A paired test would likely reveal that the lower end of these gaps are not statistically significant, thus the claimed gains potentially not sound.
  • The paper claims that prior skill induction work require complex pipelines, but then does not compare against them at all in empirical studies. Furthermore, the paper's experiments are largely set in very small scales where such pipeline complexity would not have been well-motivated (K=3K=3 training examples per skill accumulation run, and up to K=6K=6 tasks for skill evaluation). Some prior work use entire datasets of 10s to 100s of examples, where skill pipelines would become useful [1-3]. It is unclear how the proposed approach generalizes to these scenarios.
  • Ablative studies leave some open questions. For instance,
    • What's the performance of the system if one froze the skill curation policy? Is the curation policy actually learning anything meaningful, or is the presence of the skills itself doing most of the heavylifting?
    • How well do policies generalize when the tasks groups are not curated? How closely related do the tasks need to be in each group for this method to work at test time?
    • L163 "greedily group ... within each family" How is this done specifically? How much does the effectiveness observed for the algorithm rely on these task groups being catered, compared to randomly assigned?
    • What is the implementation difference between "GRPO" and "w/o curation" in the ablations? What's the advantage of "w/o Curation" over "GRPO" here?

[1] Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, Graham Neubig. Agent Workflow Memory. 2024. [2] Zora Zhiruo Wang, Apurva Gandhi, Graham Neubig, Daniel Fried. Inducing Programmatic Skills for Agentic Tasks. 2025. [3] Simon Yu, Gang Li, Weiyan Shi, Peng Qi. PolySkill: Learning Generalizable Skills Through Polymorphic Abstraction. 2026.

Questions for authors
  • How well do policies generalize when the tasks groups are not curated? How closely related do the tasks need to be in each group for this method to work at test time?
  • L163 "greedily group ... within each family" How is this done specifically? How much does the effectiveness observed for the algorithm rely on these task groups being catered, compared to randomly assigned?
  • What is the implementation difference between "GRPO" and "w/o curation" in the ablations? What's the advantage of "w/o Curation" over "GRPO" here?
Reviewer_6ykTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SkillRise aims to help language-model agents learn from past tasks and use that experience to solve new, related ones. It does this by having the same model solve tasks and update a document of useful skills, which is trained based on how well it helps with later tasks. The results show better performance across several benchmarks, improvements from learning across tasks, and lower runtime than some existing methods.

Critical strengths

One thing I like about SkillRise is that it offers a fairly simple way to learn and reuse skills across tasks. Instead of having separate components for extracting, storing, and retrieving skills, it uses the same model to solve tasks and update a skill document based on what it has learned. I also think using different rewards for task solving and skill curation makes sense, since the usefulness of a skill is better judged by how much it helps with later tasks. Another nice aspect is that the skill document makes it easier to see what the agent has learned and how that knowledge is passed on.

The experimental results are also encouraging, with SkillRise performing well across different benchmarks and model sizes. I particularly like that the paper looks at learning from related but different tasks, rather than just improving through repeated attempts at the same problem. The test-time scaling experiment is interesting because the agent improves as it encounters more related tasks, even though it only gets to attempt each task once. The authors also show that the learned skills can help when retrying the same task. Overall, the experiments provide promising evidence that the approach can help agents make better use of their past experience.

Critical weaknesses

I have some concerns about how novel the approach is compared to closely related work. Evolving-RL already trains a shared policy to extract and use skills based on downstream task performance, while SkillOS learns to update skills across sequences of related tasks. SkillRise makes some interesting design choices, such as maintaining a single evolving document and using different rewards for solving and curation, but the paper does not compare against these methods. It also does not directly test whether the proposed credit assignment is better than simpler alternatives. This makes it difficult to understand what is actually responsible for the improvements.

I also think the experiments could do more to show that the agent is learning skills that transfer across tasks. The ablation without curation only reports training performance, so it is unclear whether the same benefit holds on unseen tasks. It would also be useful to compare against a model that simply updates the skill document through prompting, without training the curation process. The test-time scaling results are encouraging, but they are limited to ALFWorld and do not include uncertainty estimates. Additional controls, such as using empty or shuffled skill documents, would help establish how much the learned skills actually contribute.

Finally, I have some concerns about the efficiency comparisons. Although the methods are matched in terms of task-solving attempts, SkillRise requires additional generations to update its skill document, which are not fully accounted for. The methods may also differ in how many distinct training tasks they encounter. The reported runtime improvements are promising, but the paper does not provide enough detail about the total computational costs, including the use of teacher models in some baselines. A clearer breakdown of training and inference costs would make these comparisons more convincing.

Questions for authors

.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.