- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies data selection for on-policy distillation when the teacher is itself trained. It asks whether choosing the teacher's training data can beat choosing the student's prompts at matched training budgets. Prompts are scored by the pass-rate gap between an existing reference model and the initial student, and the top prompts are used to train the teacher with GRPO before the student is distilled. A prompt-local Bernoulli model gives a sufficient condition for teacher-data selection to beat any student-prompt allocation. On SmolLM3-3B, the method improves the final student over uniform teacher data by 6.9% in instruction following and 4.1% in math.
Critical strengths
-
The question is well posed and new for OPD. When OPD teachers are trained for the purpose with multi-teacher domain specialists, the teacher's data is a real selection lever. Comparing it directly with student-prompt selection at matched budgets is a useful framing
-
The core in-domain result is well supported. The gain over uniform teacher data reproduces across five independent fresh-probe selections. The teacher's gain also transfers almost fully to the student.
Critical weaknesses
-
The gain holds at matched training presentations, not at matched total cost, and the paper's own cheaper alternatives do as well or better. Probing costs about 16x a uniform teacher's rollout tokens and needs a reference that already solves the task, while distilling directly from that reference ties on the primary score and is ahead under the five-word floor. Training a uniform teacher 3x longer, still cheaper than probing, gives 51.1% versus 43.5% for selected data, and selection adds nothing at that budget. The experiments never show a setting where this method is the right practical choice.
-
Teacher and student start from the same SmolLM3-3B checkpoint, so the student pass rate in the score is also the teacher's own initial pass rate. In that setup, the method reduces to reference-guided RL data selection for the teacher, which is close to RHO-LOSS-style reducible-loss selection. Targeting the student's weaknesses is never actually tested. The student-prompt selectors also give essentially no gain over uniform in every comparison, so beating them mostly restates beating uniform.
Questions for authors
-
Can you run a setting where teacher and student differ at initialization, for example a larger teacher, and compare scoring with the student's pass rate against scoring with the teacher's own initial pass rate? This directly tests the "target the student's weaknesses" claim.
-
Clarification: The in-domain gains come with shorter responses and shrink or reverse under relative-length thresholds, while the single-teacher IFEval/IFBench gains are unresolved. Beyond the five-word floor, what evidence shows that the students follow instructions better rather than answer more tersely?
Response to Reviewer Shuu
W1. Total cost and cheaper alternatives
We thank the reviewer for distinguishing matched training budgets from total cost. Our comparison does not establish that selection is cheaper end to end. We match training presentations at each stage, not the cost of constructing the teacher's dataset. The probe takes about 16 times the rollout tokens of a 40-step uniform teacher, or 1.3–2.1 such runs in recorded GPU-hours. Direct reference distillation was unresolved on IF Primary and ahead under Raw-5. A 120-step uniform teacher's student scored 51.1% on IF Primary, against 43.5% for 40-step selected data, but its median answer was 21 words rather than 61. The gain from longer uniform training reversed at both relative-length cutoffs against the 40-step uniform control. That is a qualification on how to read the longer-training result, not a matched-cost win for selection. See Section 4.4 and Appendices D.1.13 and E.5.
W2 / Q1. Shared initialization and student targeting
We thank the reviewer for pointing out that the shared checkpoint makes the two gap scores identical in the main experiments. We compared both scoring rules using different SmolLM3-3B starting checkpoints. The teacher started from a checkpoint trained on uniform data, and the student started from the original supervised checkpoint. With student prompts and training budgets fixed, six paired runs gave 49.76% using the student's initial pass rate and 50.58% using the teacher's. We did not find an advantage from using the student's pass rate. As with RHO-LOSS, reference performance guides selection. Here we measure its effect on the distilled student. See Section 2 and Appendix E.4.
The student-prompt rules were close to uniform: 39.5% for the gap rule and 40.6% for the learnability filter, versus 39.9% for uniform prompts. The contrast is informative because we also applied the gap rule to teacher data and observed a gain. This does not rule out other ways of selecting student prompts. See Appendix E.2.
Q2. Beyond a five-word minimum
We thank the reviewer for asking whether the gains reflect better instruction following or just shorter answers. The placement advantage survives both a stricter length check and a separate requirement that answers complete the task. We applied the same short-answer penalty to every teacher and compared six teacher–student pairs per arm: gap-selected teacher data with uniform student prompts versus uniform teacher data with gap-selected student prompts. We fixed the comparison before scoring 750 new prompts outside the Open-MOPD training corpus. The IF Primary gain was an absolute 6.17%. Requiring each answer to reach 25% or 50% of the initial student's median length for that same prompt left gains of 5.94% and 3.98%, respectively. Unlike a five-word minimum, this check uses a different threshold for each prompt. See Section 4.5 and Table 5A.
Length by itself does not establish completion. In a separate audit, Claude Opus 5.5 judged full answers without seeing which model wrote them. On the 183 prompts in the main analysis, counting only verifier passes rated as complete left a gain of 6.2%. An exploratory check of these same models also found gains over uniform of 3.1% on IFEval and 3.9% on IFBench. These results address the brevity concern for the follow-up with the short-answer penalty. The original runs remain sensitive to length, and their public benchmark differences in Table 1 remain unresolved. See Appendix E.2.