- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper identifies skill blindness as a credit-assignment failure in skill-based RL. Then it proposes Skill-Conditioned Policy Optimization (SCPO), which address skill credit blindness. Besides, SCPO aggregates skill utility during training to update the skill library. Experimental results show superior performance compared to GRPO.
Critical strengths
- This paper identifies skill credit blindness and proposes SCPO to address it.
- Experimental results show superior performance compared to GRPO.
Critical weaknesses
- The baseline comparison is too limited. The main experiments compare SCPO only against vanilla GRPO, despite this paper being positioned in the broader literature on skill-based RL and self-evolving skill libraries. Comparisons with stronger skill-learning or alternative credit-assignment baselines are necessary to determine whether the gains specifically come from SCPO's proposed credit decomposition.
- The statistical evidence on several benchmarks is relatively weak. AIME 2024/2025/2026 and HMMT 2025 contain only 30 test problems, and although the authors report results over 10 seeds, no statistical significance tests or confidence intervals are provided.
- The construction of the initial skill library is unclear. SCPO critically relies on an external skill library, but this paper does not explain how this library is initialized before training, including its source, size, and quality-control procedure. The appendix describes how new skills are extracted and evolved during training, but this does not specify the initial library on which the method depends.
- The training reward comparison in Figure 2 is potentially confounded by different reward definitions. SCPO introduces additional reward shaping and also defines an optional diversity-aware reward before advantage estimation, so a higher mean training reward than GRPO is not necessarily evidence of a better learned policy unless exactly the same reward is used for both methods. In contrast, the batch pass rate is substantially lower for SCPO early in training and only becomes roughly comparable to GRPO later, which provides limited evidence for the claimed optimization advantage.
- It is unclear whether SCPO and GRPO are compared under a matched sampling budget. SCPO requires both skill-conditioned and no-skill rollouts for the same problem to estimate paired skill gain, introducing additional rollout generation beyond standard GRPO. This paper does not report the total number of generated tokens or demonstrate that GRPO receives an equivalent rollout budget, so part of the improvement may come from additional sampling rather than the proposed credit-assignment mechanism.
- The choice of mathematical reasoning as the sole evaluation domain limits the empirical validation of SCPO. Skill-based RL is particularly relevant in agentic settings where skills must be repeatedly retrieved, composed, and transferred across heterogeneous tasks, whereas mathematical reasoning provides a relatively narrow and static testbed for these capabilities. A more convincing evaluation would include agentic or interactive environments where skill reuse and transfer are directly exercised.
Questions for authors
Please refer to the Weaknesses.