- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper looks at on-policy self-distillation (OPSD), where the frozen teacher is conditioned on the verified solution but the student only sees the question. The authors call this gap "privilege mismatch". Their fix is to replace the full solution in the teacher's context with a compact structural skeleton: an ordered sequence of tagged reasoning operations with intermediate and final results stripped out, enforced by a validator that rejects answer strings, result-bearing equations, and any numbers not already in the question. Teacher and student start from the same checkpoint and only the student's LoRA parameters are updated, using forward KL at student-generated prefixes. On Qwen3-1.7B/4B/8B over AIME24, AIME25 and HMMT25, SB-OPSD gets the best three-benchmark mean Average@12, +5.70/+3.31/+3.22 over base, ahead of full-solution OPSD. A bandwidth sweep from functional tags up to full solutions peaks at the short skeleton, and ordering and cross-sample controls suggest both operation order and question-skeleton matching matter.
Critical strengths
-
The bandwidth hierarchy (B1 to B4) plus the ordering and cross-sample controls form a well-designed experimental spine. "More privileged information is not better" is a clean and non-obvious result. The fact that shuffled and reversed skeletons land below base makes the claim that structure matters (rather than just the removal of results) something the experiments actually test, not just assert.
-
The intervention is small and deployable. It changes only what goes into the teacher's context, keeps the student question-only, and adds nothing at inference time. That makes it easy for others to adopt or to refute.
Critical weaknesses
-
The training and evaluation lengths do not match, which undercuts the on-policy premise. Max training completion is 1,024 tokens (Table 5) while evaluation generates up to 38,912, with median outputs around 19.7K. Over 90% of each deployed trajectory is therefore never distilled on. For competition math with Qwen3 thinking models this means the method only ever shapes the opening of the reasoning, and I cannot tell whether the gains reflect better long-chain reasoning or something specific to the truncated training regime.
-
The baselines look undertrained, which makes the headline deltas hard to read. At 4B, OPSD reaches 61.70 against a base of 61.76, so zero gain, and SFT lands at 61.61, below base. In Table 3 majority-vote SFT drops from 72.22 to 52.22 on AIME24. Most of the claimed +3.37 over OPSD at 4B comes from OPSD not working at all here. With LoRA only, 100 steps and 1 rollout per prompt, my worry is that every method is undertrained and the ranking between them is not informative.
-
The skeleton extractor is never identified. Appendix B.3 says only "a fixed offline extractor" and names no model. If it is a stronger external LLM then this is not self-distillation any more, and the comparison against OPSD (which uses the dataset solution as-is) is confounded by another model's reasoning entering the teacher's context. It also blocks reproduction.
Questions for authors
- Which model generates the skeletons, and what fraction of examples pass the validator inside the three-attempt budget?
- In Table 2, shuffled (36.17) and reversed (34.01) fall below base (36.67). What do the training curves look like there? Is the student actively degrading or just not improving?
- When the 1,024-token cap truncates a rollout mid-reasoning, is the KL still computed over the truncated tail, and did you apply any length correction?