acceptodds
Under review as a conference paper at ICLR 2027

AlphaSkill: Agentic Reinforcement Learning with Self-evolving Skills

Abstract

Agent skills extract interaction experience into high-level, reusable knowledge that is essential for reliably solving complex tasks. Nevertheless, the acquisition of suitable skills remains challenging: manually designing them introduces a critical scalability bottleneck, while static skills may lead to a progressive misalignment between accumulated skills and the agent’s evolving capabilities. In this work, we propose **AlphaSkill**, a reinforcement learning framework that enables self-curated skills to co-evolve alongside the agent's improving capabilities. In this paradigm, a single policy network functions as two roles: (1) a *Reasoner* that interacts with the environment to solve tasks, and (2) a *Skill Curator* that maintains a skills library by creating or refining skills based on the frontier of the Reasoner’s capability. The Reasoner is rewarded for solving challenging tasks in environments, while the Skill Curator is optimized according to whether its curated skills contribute to the Reasoner's successes, thereby creating a virtuous loop of co-evolution between task solving and skills curation. Extensive experiments on various agentic environments and reasoning tasks demonstrate that AlphaSkill achieves consistent gains compared to its counterparts. Moreover, these self-curated skills can further improve test-time performance when used as in-prompt hints.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Agent, Reinforcement Learning, Skills

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 2 reviews, above 68% of the others.
Reviewer_29ZBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces AlphaSkill, a reinforcement learning framework which jointly trains LLMs to develop self-evolving skills whilst simultaneously learning to solve a reasoning task. The core contribution of AlphaSkill is the self-evolving skill mechanism used to improve task performance. AlphaSkill is evaluated across virtual world, multi-hop and mathematical environments which suggest the advantage of the AlphaSkill over GRPO, skill-augmented RL methods and domain-specific RLVR methods. The effect of individual components is analysed through several ablation studies.

Critical strengths
  • The evaluation suite is comprehensive. The main experiments cover nine benchmarks across three domains and two models, with comparisons against GRPO, skill-augmented RL and domain-specific baselines. This provides evidence of the efficacy of AgentSkill over other approaches.
  • The approach is well-motivated and has a simple design. Hand-written rules do not scale and skills extracted by a fixed model can fall out of step with the policy. Training a Reasoner and SkillsCreator jointly addresses these problems with a simple approach.
Critical weaknesses
  • My main concern with the training approach is that there is no robust mechanism to stop the curator outputting skills that leak answer-specific information in the prompt rather than teaching generalisable strategies. The only safeguard seems to be the instruction in the curator prompt. A skill which leaks answers information would very likely lead to G1 < G2 . This behaviour would be reinforced in the curator via SFT. The curator is therefore optimised for whatever passes same-task validation rather than for generality. Whilst there is some evidence in Figure 4 that some skills do transfer, it is an inherent weakness when G2 is evaluated on a question the curator has already seen.
  • Whilst the evaluation suite is comprehensive, it is difficult to establish the advantage of AlphaSkill over competing methods without confidence intervals. This is made especially important as many of the evaluation benchmarks used are small (AIME2025 30, Bamboogle 125, ALFWorld 140) and many reported differences are small and look like they could be within statistical noise.
Questions for authors
  1. Can you provide confidence intervals in Figure 2 and Figure 3?
  2. Can the authors train a variant where G2 uses a different question from the same task to G1, and evaluate its performance against the original method?
3 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We sincerely appreciate the reviewer's time and effort in reviewing our manuscript. Our responses to your concerns are as follows.

[W1] My main concern with the training approach is that there is no robust mechanism to stop the curator outputting skills that leak answer-specific information in the prompt rather than teaching generalisable strategies.

Thanks for the insightful comments. We address it through an audit of the accumulated skills, discussion of their role during training, and a cross-task training experiment.

(1) Auditing answer-specific information in the skill library after training. We inspected each skill in the final libraries using DeepSeek-V4.1-flash to judge whether it explicitly contained the answer of a specific question. As shown in Table R1, over 90% of skills are classified as containing no direct answer information, indicating that the curated skill library remains general and is not dominated by explicit answer information to specific questions for passing same-task validation. We also observed a lower incidence of explicit answer information for a stronger backbone model with stronger instruction-following capability. We further visualized the high-frequency terms appearing in the skill library in Figure 8 of the paper.

Table R1. The percentage (%) of skills without answer-specific information in the content.

ModelsALFWorldWebshopMulti-hop QAMathematics
Qwen2.5-7B-Instruct94.491.890.194.9
Qwen3-4B-Instruct-250797.695.393.095.8

(2) Deeper discussion on how skills take effect during training. We would like to discuss the implications when a skill may contain answer-specific information. First, the evaluation is conducted on held-out tasks, so any answer-specific information of a training instance contained in the skills does not reveal answers to the evaluation instances. Second, we believe that the training gains primarily arise from learning higher-quality trajectories under skill guidance (i.e., reasoning) and the model’s improved understanding/knowledge of the environment (i.e., skills internalization). More broadly, many agentic tasks like ALFWorld and Webshop are verified through changes in the environment state, rather than by matching a reference answer as in mathematics. Solving these tasks requires interacting with the environment and changing its state through successive actions. Even when skills contain a task-specific procedure, the agent must ground that procedure in its observations and execute a sequence of valid actions to reach the expected final state. Such task-specific guidance can therefore serve as scaffolding that reduces redundant exploration to help the policy sample efficient reasoning and acting trajectories for more effectively learning helpful behavioral patterns.

(3) Training with skills from a different task. To more thoroughly rule out the potential influence of answer-specific information in the skills, we train a variant using Qwen3-4B-Instruct-2507 in which each guided rollout group G2G_2 receives skills curated from another unguided rollout group G1G_1 with different tasks. For ALFWorld, we exchange skills between distinct tasks of the same task type. For the other environments, we randomly pair training tasks and exchange their curated skills for guided rollouts. As shown in Table R2, this cross-task variant remains effective and even achieves higher performance than AlphaSkill on some benchmarks. These results provide empirical evidence that the training benefits largely do not stem from potential answer-specific guidance on the tasks that produced it.

Table R2.

MethodALFWorldWebshopNQHotpotQABamboogleMATH500OlympiadMinervaAIME25
AlphaSkill98.679.251.144.246.488.860.146.043.3
AlphaSkill w/ skills exchange97.980.450.244.947.289.860.645.243.3

Overall, our empirical results indicate that explicit answer-specific information is rare, while the cross-task experiment provides further evidence that skill guidance remains useful beyond its source tasks. When task-specific information does occur, its effect on policy learning may not be negative, since it can still contribute useful training experience. Moreover, the main evaluations use held-out tasks without skill injection, so this information does not directly supply answers during evaluation. The additional results and discussion have been updated in Appendix B of our revised manuscript.

AuthorsOpenReview ↗

Rebuttal (2/2)

[W2 & Q1] Can you provide confidence intervals?

Thanks for the question. We have additionally conducted three independent evaluation runs with sampling seeds 7, 42, and 123 for each result and report the corresponding standard deviations in Table R3 and Table R4. The evaluation results show minimal fluctuation across different seeds, confirming that AlphaSkill's improvements are not attributable to statistical noise. The additional results have been updated in Appendix B of our revised manuscript.

Table R3. Extended version of Table 1 in the main paper, with standard deviations added.

ModelMethodALFWorldWebshopNQHotpotQABamboogleMATH500OlympiadMinervaAIME25
Qwen2.5-7B-InstructInitial13.3 ± 0.37.5 ± 0.312.7 ± 0.915.8 ± 0.415.2 ± 0.776.5 ± 0.439.5 ± 0.328.7 ± 0.35.6 ± 1.6
GRPO85.2 ± 0.768.9 ± 0.240.3 ± 0.237.8 ± 0.339.5 ± 0.479.3 ± 0.241.1 ± 0.231.4 ± 0.210.0 ± 2.7
AlphaSkill96.7 ± 0.376.2 ± 0.749.2 ± 0.745.0 ± 0.644.3 ± 0.882.5 ± 0.243.6 ± 0.134.7 ± 0.310.0 ± 5.4
Qwen3-4B-Instruct-2507Initial25.0 ± 0.618.9 ± 0.419.3 ± 0.319.3 ± 0.414.7 ± 1.088.5 ± 0.356.7 ± 0.342.2 ± 0.237.8 ± 3.1
GRPO93.3 ± 0.774.5 ± 0.242.4 ± 1.040.1 ± 0.340.3 ± 0.889.4 ± 0.558.2 ± 0.143.4 ± 0.640.0 ± 5.4
AlphaSkill98.8 ± 0.979.1 ± 0.251.0 ± 0.644.5 ± 0.246.4 ± 1.389.1 ± 0.460.2 ± 0.145.5 ± 0.341.1 ± 1.6

Table R4. Extended version of the baseline comparison in the main paper, with standard deviations added.

MethodALFWorldWebshopMethodNQHotpotQABamboogleMethodMATH500MinervaOlympiadAIME25
Initial13.3 ± 0.37.5 ± 0.3Initial12.7 ± 0.915.8 ± 0.415.2 ± 0.7Initial76.5 ± 0.428.7 ± 0.339.5 ± 0.35.6 ± 1.6
Reflexion43.1 ± 0.328.9 ± 0.5ZeroSearch43.7 ± 0.134.5 ± 0.728.0 ± 0.0Fewshot77.1 ± 0.429.2 ± 0.240.0 ± 0.25.6 ± 4.2
SkillRL90.2 ± 0.972.9 ± 0.6SkillRL46.3 ± 0.542.6 ± 0.573.1 ± 0.4AbsoluteZero78.3 ± 0.133.9 ± 0.242.0 ± 0.38.9 ± 1.6
Skill089.5 ± 0.773.2 ± 0.7Skill042.3 ± 0.639.5 ± 0.466.9 ± 0.4Critique-GRPO80.7 ± 0.333.2 ± 0.542.5 ± 0.312.2 ± 3.1
AlphaSkill96.7 ± 0.376.2 ± 0.7Ours49.2 ± 0.745.0 ± 0.644.3 ± 0.8AlphaSkill82.5 ± 0.234.7 ± 0.343.6 ± 0.110.0 ± 5.4

[Q2] Can the authors train a variant where G2 uses a different question from the same task to G1, and evaluate its performance against the original method?

Thanks for the valuable suggestion. As suggested, we have provided the results of this variant in Table R2 in response to W1. As shown in Table R2, this variant remains competitive and even achieves higher scores on some benchmarks. We greatly appreciate the reviewer for giving us a new perspective, which helped us strengthen the empirical validation of our method. The additional results have been updated in Appendix B of our revised manuscript.

Dear Reviewer 29ZB,

Thanks again for your insightful review. As the discussion period is coming to a close (less than one day), we would like to briefly summarize the additional experiments addressing your main concerns:

  • Skills content audit. Most skills are classified as containing no answer-specific information, indicating that the curated skills remain general.
  • Training with skills from different tasks. We train a variant in which each guided rollout group receives skills curated from another unguided rollout group with different tasks. This cross-task variant remains effective, demonstrating that the training benefits do not stem from potential answer-specific guidance on the tasks that produced it.

These results have been updated in our revised manuscript, with full experimental details provided in the rebuttal above. We would be very grateful if you could revisit our rebuttal and let us know whether any points need further clarification.

Best regards,

Authors of AlphaSkill

Reviewer_NxtjOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AlphaSkill, an agentic reinforcement learning framework that jointly improves task-solving capabilities and skill curation through a self-evolving training process. Specifically, a shared policy serves as both a Reasoner, which learns to solve downstream tasks through reinforcement learning, and a Skill Curator, which extracts reusable skills from successful and failed trajectories. Experiments across diverse agentic environments and reasoning benchmarks demonstrate promising improvements over existing baselines. Overall, the idea is interesting, but I suggest the authors provide a more comprehensive analysis.

Critical strengths
  1. The overall idea is interesting and meaningful. Rather than treating skills as static external knowledge, the paper integrates skill curation with downstream task optimization, allowing the two capabilities to co-evolve.
  2. The proposed approach is conceptually clear and relatively straightforward to implement. I
  3. The experiments cover multiple domains, including interactive environments, multi-hop question answering, and mathematical reasoning. The improvements over standard GRPO and several skill-based baselines.
Critical weaknesses
  1. Although the authors mention that AlphaSkill introduces approximately 1.2× training overhead, it remains unclear whether the performance improvements come from the proposed co-evolving mechanism or simply from the additional computation. I would encourage the authors to include compute-matched baselines, particularly by comparing AlphaSkill against standard GRPO and relevant skill-based methods under the same training FLOPs or computational budget.

  2. The paper would benefit from a more detailed discussion of the paper: Skill1 (Unified Evolution of Skill-Augmented Agents via Reinforcement Learning), which explores a closely related idea of jointly optimizing skill-related capabilities and downstream task execution.

  3. While the current ablation studies demonstrate the effectiveness of skill evolution and the SFT decay mechanism, several important design choices remain insufficiently explored. Please see the question part. Do you have any insights on this problem?

Questions for authors
  • How sensitive is the method to the number of rollouts used for skill generation and validation?

  • How does the relative weight of the Skill Curator's SFT objective affect training dynamics and final performance?

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We thank the reviewer for the valuable feedback and positive support! We address the concerns point by point as follows.


[W1] Although the authors mention that AlphaSkill introduces approximately 1.2× training overhead, it remains unclear whether the performance improvements come from the proposed co-evolving mechanism or simply from the additional computation.

Thanks for the insightful comment.

(1) AlphaSkill introduces about 1.2× in training time due to additional rollouts and optimization of the Skill Curator. We report the average per-stage time during training in comparison with standard GRPO in Table R1, which shows that AlphaSkill achieves promising improvements over GRPO without significantly increasing training time. The additional comparison have been updated in Section 4.3 of our revised manuscript.

Table R1: The average time (s) of each stage during ALFWorld training.

RolloutSkills CreationLog probTraining
GRPO96.6/2.38.3
AlphaSkill98.920.82.510.6

(2) To ensure a fairer comparison under the same computational budget, we further compare the performance changes of GRPO and AlphaSkill during training with Qwen2.5-7B-Instruct on ALFWorld under identical training tokens. As shown in Table R2, AlphaSkill consistently outperforms GRPO at the same token budgets, indicating that the gains do not merely stem from additional computation.

Table R2: The performance changes of GRPO and AlphaSkill on ALFWorld under the same training token.

Token Budget18.7 M32.8 M42.3 M50.1 M
GRPO22.153.684.385.7
AlphaSkill39.382.192.995.0

[W2] The paper would benefit from a more detailed discussion of the paper Skill1.

Thanks for the valuable suggestion. The main distinctions between Skill1 and AlphaSkill are how skills are generated and supervised. Skill1 optimizes selection, utilization, and distillation using task outcomes and utility-derived credit signals; its distillation reward compares the current outcome with the historical utility of retrieved skills. AlphaSkill instead generates or refines skills from an unguided trajectory group, executes a subsequent skill-guided group on the same task, and uses the observed improvement to select skill demonstrations for internalizing. We have expanded the related work discussion in Section 5 to incorporate Skill1 accordingly.


[W3 & Q1] How sensitive is the method to the number of rollouts used for skill generation and validation?

Thanks for the question. The current rollout number is N=8, split into vanilla group ∣G1∣=4|G_1|=4 and guided group ∣G2∣=4|G_2|=4. The Curator uses G1G_1 for curating skills, and the comparison between the two groups supplies the validation signal. Thus, N controls both the evidence for curation and the precision of the empirical utility. We additionally conduct a sensitivity study with different N using Qwen2.5-7B-Instruct on Webshop, where the number of rollouts in the two subgroups is both N/2. The results in Table R3 show that the final performance consistently improves as N increases. This trend is consistent with the roles of the rollout budget: More rollouts provide richer evidence about successful decisions and failure modes, while more rollouts of two sub-groups enable a more reliable validation of skill utility. The additional comparisons have been updated in Section 4.2 (RQ3) of our revised manuscript.

Table R3: The sensitivity of rollout numbers.

Rollouts81216
Webshop76.278.079.6
AuthorsOpenReview ↗

Rebuttal (2/2)

[W3 & Q2] How does the relative weight of the Skill Curator's SFT objective affect training dynamics and final performance?

Thanks for the question.

(1) In current setup, the SFT weight is α=λs\alpha = \lambda^s, where λ=0.95\lambda=0.95. In Table 4 of our original manuscript, we evaluated the ablation variants w/o evolving (α=0\alpha = 0) and w/o decaying (α=0.95\alpha = 0.95), and Figure 3 discussed the dynamics of reward and Validated Skill Ratio. We further supplement these variants with their entropy dynamics in Figure 4 of the revised manuscript. When α=0\alpha = 0, the Curator is not optimized, so its guidance for policy gradually mismatched. This in turn limits the exploration space of the policy, where the entropy declines more rapidly. When α\alpha is fixed at 0.95, the policy continues to invest effort in optimizing skill curation even after the Reasoner's capability has nearly converged (i.e., the effect of skills diminishes), instead of shifting the weight to the Reasoner, resulting in fluctuating entropy in the later training stage. In contrast, AlphaSkill exhibits a favorable pattern of early exploration followed by later convergence. These observations support the purpose of the schedule: stronger Curator supervision supports learning from experience early, while its progressive decay shifts the relative optimization weight toward the Reasoner later. These dynamics are also reflected in its higher final performance compared with the two variants. These analyses have been updated in Section 4.2 (RQ3) of our revised manuscript.

(2) To further investigate the effect of λ\lambda on the final performance, we experimented with a wider range of values. As shown in Table R4, the overall performance of Qwen2.5-7B-Instruct on Webshop remains robust across different choices of λ\lambda, suggesting that AlphaSkill is insensitive to this hyperparameter in a reasonable interval.

Table R4: The sensitivity analysis of the relative weight of Skill Curator.

λ\lambda00.250.50.750.95
Webshop72.875.476.077.276.2

Thanks for the author's rebuttal. They addressed most of my concerns. I will keep my positive scores, for sure.

AuthorsOpenReview ↗

Thanks for the feedback

Thanks for reviewing our rebuttal and the positive assessment of our work! Your constructive feedback has greatly helped us improve our paper on the computational and sensitivity analyses. Have a wonderful day! :)

Comments on the paper

4 comments
AuthorsOpenReview ↗

Rebuttal (1/2)

We sincerely thank the reviewer for insightful comments and positive support! Below we address the main concerns raised in the review.


[W1.1] The w/o evolving variant removes the SFT term, but both roles still share parameters updated by RL, so it does not hold the curator fixed.

Thanks for your valuable comment. To isolate the effect of parameter changes on Curator, we conduct additional experiments using a frozen Curator (w/o updating) that is initialized from Qwen2.5-7B-Instruct. This w/o updating variant retains the same rollout budget and acceptance rule. As shown in Table R1, AlphaSkill outperforms these ablated variants on most scenarios, suggesting the effectiveness of evolving and internalizing skills in improving agentic reasoning. The additional results have been updated in Table 4 of our revised manuscript.

Table R1.

MethodALFWorldWebshopNQHotpotQABamboogleMATH500MinervaOlympiad
w/o updating88.672.044.543.343.278.432.441.5
w/o evolving90.072.845.243.543.277.631.641.9
AlphaSkill96.476.248.344.744.882.834.943.6

[W1.2] The acceptance rule compares four guided with four unguided rollouts on the same task, so acceptance depends on sampling noise and on the base success rate.

Thanks for the insightful question. To further investigate whether the accepted skills result from sampling noise, we expand the rollout number of corresponding tasks with validated skills at saved checkpoints. For each task with validated skills, we compared the mean reward of the 8 vanilla rollouts with 8 skill-guided rollouts. As shown in Table R2, the validated skills improve the success rate of guided groups by 30% relative to vanilla groups (an average of more than two successful rollouts out of eight per task), which validates the utility of the validated skills in larger rollouts. The additional results have been updated in Appendix B of our revised manuscript.

Table R2.

ckptGroupALFWorldMulti-hop QAMath
50 stepvanilla0.2290.1460.300
guided0.542 (+0.313)0.438 (+0.292)0.625 (+0.325)
100 stepvanilla0.5210.2080.321
guided0.833 (+0.312)0.563 (+0.355)0.661 (+0.340)

[W1.3] Figure 5 holds the reasoner fixed but does not separate better curation from a larger library. A fixed-curator or equal-size random-SFT control would show what adaptation contributes.

Thanks for the valuable suggestion. We additionally provide the performance of frozen Qwen2.5-7B-Instruct Reasoner, using snapshots of the skill library collected during training with a fixed Curator (w/o updating in W1.1) to separate better curation. As shown in Table R3, snapshots of the skill library accumulated by the evolving Curator contribute to higher performance than its frozen counterpart. The additional results have been updated in Figure 7 of our revised manuscript.

Table R3.

SnapshotALFWorldWebshop
50 step20.7 / 18.613.2 / 12.6
100 step28.6 / 25.719.6 / 17.0
150 step27.1 / 27.925.8 / 23.4
200 step32.1 / 28.525.0 / 23.8
250 step31.4 / 27.926.2 / 24.2

[W2.1 & Q1] The gains over GRPO remain, but Table 3 does not support the sentence about uniform superiority.

Thanks for raising this concern. In multihop-QA, the retrieval, scoring, and decoding ways are the same. For Bamboogle, AlphaSkill achieves improvements over GRPO on Qwen2.5-7B-Instruct (44.8 vs. 39.8), but it is weaker than SkillRL. We note that SkillRL will identify failure patterns on the test dataset and propose new skills to cover these gaps during RL training. These skills may be fitted to the test data and internalized into the model, creating potential data leakage. In contrast, we curate skills only on the training dataset while the test dataset is just for evaluation. We revise the statement and provide additional explanation in Section 4.2 of our revised manuscript.


[W2.2 & Q2] The test-time experiment also lacks a stated budget.

Thanks for this question. In the test-time pass@k with skills evolving setting, we organize sampling into rounds of eight rollouts per task. All tasks share an initially empty skill library. After each round, the Curator creates or modifies relevant skills in the skills library based on the sampled rollouts. The updated skills then guide subsequent exploration through prompt injection and will not be reset per question. Unlike during training, we do not use ground-truth labels to filter or select skills at test time. This also evaluates the trained policy’s ability to curate helpful skills from online experience without ground-truth-based skill fitting. These clarifications have been updated in Section 4.2 (RQ4) of our revised manuscript.

AuthorsOpenReview ↗

Rebuttal (2/2)

[Q3] How many seeds and how much variability?

Thanks for this question. In all training, we use the default settings of verl framework. The seed of dataloader is 42, and the training backend (FSDP engine) is stochastic by design (temperature=1.0). To quantify robustness, we additionally conducted repeated evaluations and training runs with different random seeds.

(1) Evaluation variability. We have additionally conducted three independent evaluation runs for each result with sampling seeds 7, 42, and 123, respectively. The corresponding standard deviations are provided in Tables R4 and R5.

Table R4. Extended version of Table 1 in the main paper, with standard deviations added.

ModelMethodALFWorldWebshopNQHotpotQABamboogleMATH500OlympiadMinervaAIME25
Qwen2.5-7B-InstructInitial13.3 ± 0.37.5 ± 0.312.7 ± 0.915.8 ± 0.415.2 ± 0.776.5 ± 0.439.5 ± 0.328.7 ± 0.35.6 ± 1.6
GRPO85.2 ± 0.768.9 ± 0.240.3 ± 0.237.8 ± 0.339.5 ± 0.479.3 ± 0.241.1 ± 0.231.4 ± 0.210.0 ± 2.7
AlphaSkill96.7 ± 0.376.2 ± 0.749.2 ± 0.745.0 ± 0.644.3 ± 0.882.5 ± 0.243.6 ± 0.134.7 ± 0.310.0 ± 5.4
Qwen3-4B-Instruct-2507Initial25.0 ± 0.618.9 ± 0.419.3 ± 0.319.3 ± 0.414.7 ± 1.088.5 ± 0.356.7 ± 0.342.2 ± 0.237.8 ± 3.1
GRPO93.3 ± 0.774.5 ± 0.242.4 ± 1.040.1 ± 0.340.3 ± 0.889.4 ± 0.558.2 ± 0.143.4 ± 0.640.0 ± 5.4
AlphaSkill98.8 ± 0.979.1 ± 0.251.0 ± 0.644.5 ± 0.246.4 ± 1.389.1 ± 0.460.2 ± 0.145.5 ± 0.341.1 ± 1.6

Table R5. Extended version of the baseline comparison in the main paper, with standard deviations added.

MethodALFWorldWebshopMethodNQHotpotQABamboogleMethodMATH500MinervaOlympiadAIME25
Initial13.3 ± 0.37.5 ± 0.3Initial12.7 ± 0.915.8 ± 0.415.2 ± 0.7Initial76.5 ± 0.428.7 ± 0.339.5 ± 0.35.6 ± 1.6
Reflexion43.1 ± 0.328.9 ± 0.5ZeroSearch43.7 ± 0.134.5 ± 0.728.0 ± 0.0Fewshot77.1 ± 0.429.2 ± 0.240.0 ± 0.25.6 ± 4.2
SkillRL90.2 ± 0.972.9 ± 0.6SkillRL46.3 ± 0.542.6 ± 0.573.1 ± 0.4AbsoluteZero78.3 ± 0.133.9 ± 0.242.0 ± 0.38.9 ± 1.6
Skill089.5 ± 0.773.2 ± 0.7Skill042.3 ± 0.639.5 ± 0.466.9 ± 0.4Critique-GRPO80.7 ± 0.333.2 ± 0.542.5 ± 0.312.2 ± 3.1
AlphaSkill96.7 ± 0.376.2 ± 0.7Ours49.2 ± 0.745.0 ± 0.644.3 ± 0.8AlphaSkill82.5 ± 0.234.7 ± 0.343.6 ± 0.110.0 ± 5.4

(2) Training stability across random seeds. We have additionally conducted experiments in which we repeat the training of AlphaSkill using Qwen3-4B-Instruct-2507 with three random seeds in Table R6.

Table R6.

MethodALFWorldWebshopNQHotpotQABamboogleMATH500OlympiadMinervaAIME25
AlphaSkill97.9 ± 0.679.0 ± 0.350.6 ± 0.444.2 ± 0.146.9 ± 0.888.7 ± 0.660.4 ± 0.246.0 ± 0.340.0 ± 2.7

The resulting evaluation and training show high consistency across seeds. The additional results have been updated in Appendix B of our revised manuscript.


[Q4] Table 6 lists 140 ALFWorld test environments while Appendix B gives 134 seen plus 140 unseen, which set does the All column use?

Sorry for the confusion. We report the seen environments in Table 1. To test broader generalization, we provide results on the unseen set in Table R7. The evaluation results further demonstrate the effectiveness of AlphaSkill compared to baselines. The additional results have been updated in Appendix B of our revised manuscript.

Table R7.

InitialReflexionGRPOSkillRLSkill0AlphaSkill
ALFWorld (unseen)9.0 ± 1.240.0 ± 0.484.8 ± 0.787.6 ± 0.486.3 ± 1.394.0 ± 1.1
Reviewer_eGwFOpenReview ↗

Thanks for the additional experiment.

The frozen-curator control in Table R1, the repeated evaluation in Tables R4 and R5, and the unseen ALFWorld results in Table R7 address most of what I raised, and I accept the clarification that Table 1 reports the seen split.

AuthorsOpenReview ↗

Thanks for the feedback

We sincerely thank the reviewer for taking the time to review our rebuttal, and are glad that our clarifications and additional results helped address concerns. Your initial questions inspired us to dive deeper into the roles of Skill Curator and the utility of skills, which significantly strengthens our manuscript. Have a nice day :)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.