- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper uses a teacher model to generate hints, thereby constructing hint prompts to train the student model, enabling the student model to still achieve a certain performance improvement when all rollouts are incorrect. The experiment results show its performance outperforms RL methods such as GRPO, as well as knowledge distillation methods such as OPD.
Critical strengths
- The paper conducts experiments across multiple model scales and provides extensive ablation studies.
- The paper starts from a meaningful research problem and investigates training optimization in the all wrong setting. The core idea is somewhat analogous to self-distillation, where additional privileged information is incorporated into the prompt to improve model optimization.
Critical weaknesses
- The paper does not sufficiently explain why capabilities learned with privileged prompts can transfer to prompts without privileged information. The underlying mechanism is unclear.
- The novelty appears limited and is closer to an engineering improvement. The method is similar to prompt-space curriculum learning methods [1][2][3], which improve training efficiency by incorporating additional hints into the prompt. However, no direct experimental comparisons with these methods are provided.
[1] Nudging the Boundaries of LLM Reasoning, ICLR 2026
[2] Self-Hinting Language Models Enhance Reinforcement Learning, arxiv 2026
[3] Learning to Hint for Reinforcement Learning, arxiv 2026
Questions for authors
Q1. Why can capabilities learned with privileged prompts transfer to standard prompts without privileged information? What drives this transfer, and how efficient is it?
Q2. Qwen3.5 base models of different sizes belong to the same model family and have similar policy distributions. Therefore, directly performing OPD within the same base model family may provide only limited incremental signals from the teacher, which can constrain the gains of OPD [1]. A more comprehensive comparison would be to first construct a teacher with a clear capability gap from the student through RL post-training, and then compare OPD and ZPPO under the same teacher–student setting. This would provide a more meaningful comparison in knowledge transfer.
[1] Rethinking On-Policy Distillation of Large Language Models: Phenomenology, Mechanism, and Recipe, arxiv 2026