- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a self-improvement method for LLM agents. The method establishes a teacher–student distillation framework and employs an LLM-based skill curator to evolve and compose skills. Based on task outcomes, it dynamically determines the direction of distillation between a skill-free Student and skill-augmented Teachers. Finally, the policy is jointly trained using GRPO and correctness-gated distillation.
Critical strengths
Major Strengths
The proposed method is very interesting.
Minor Strengths
The paper cites recent literature and builds upon recent research.
Critical weaknesses
The paper contains significant errors in its references. The cited work "AlphaEvolve Team. Alphaevolve: A coding agent for scientific and mathematical discovery" cannot be found, while a closely related work exists [1]. Additionally, the authors of "Training Language Models to Self-Correct via Reinforcement Learning" are incorrectly listed. These errors make me suspect that the authors used AI generative tools to produce the references without verifying them. Furthermore, the Introduction contains no references, making it difficult to verify the accuracy of some statements.
[1] Novikov, Alexander, et al. "Alphaevolve: A coding agent for scientific and algorithmic discovery." arXiv preprint arXiv:2506.13131 (2025).
The novelty of the paper's core contributions is insufficiently established. Previous works such as SkillRL and UCOB have already explored combining external behavioral discovery with policy internalization, while UCOB has also employed bidirectional distillation based on behavioral differences between Skill and No-Skill configurations. Moreover, the experimental comparisons lack closely related methods such as UCOB.
Questions for authors
-
Can the authors clarify the differences between the proposed method and existing methods such as SkillRL and UCOB? If the main difference is simply introducing Teachers with different Skill configurations, how does this constitute a substantial methodological contribution?
-
Can the authors provide additional experimental comparisons with closely related methods, particularly UCOB, to better demonstrate the effectiveness of the proposed method?
-
Can the authors correct the reference errors and provide appropriate citations in the Introduction, particularly for statements such as "Existing on-policy distillation and self-distillation approaches largely emphasize the second process"?
-
Can the authors further analyze the effect of using multiple Teachers, for example, by comparing with a strategy that performs multiple runs and selects the best one?
-
Can the authors provide additional ablation studies or discussion to clarify the contribution of correctness gating and the roles of different gating mechanisms?