- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The authors study the problem in on-policy self-distillation, where the teacher model is a copy of a student model with the additional privileged information. The authors study how different forms of the information access given to the teacher affects the distillation performance on the student. The authors evacuate performance of the Qwen models of sizes 1.7B, 4B, and 8B parameters. They show that different models benefit differently based on levels of information, from coarser to finer information.
Critical strengths
The topic the authors address is important for understanding how much privileged information affects the performance of the on-policy distillation.
The authors provide clear information to reproduce the results.
The results are comprehensive for the evaluated models.
Critical weaknesses
The experiments are limited, which makes it difficult to get any general conclusion. The authors only study one model family of students using a single training dataset.
The authors study the mathematics domain in the paper, the domain which benefits from reasoning. However, the student model used does not have thinking mode enabled.
Questions for authors
Why using a larger model offline compiler to create abstract packages?
Have you checked if the teacher answer is correct (as ground truths are not available)? How was that evaluated?
Why would the student not use thinking template? What is the performance when thinking is enabled explicitly?
Have you tried training the teacher model on the reference answers, or strategies? Given this setup, the teacher input is already different from the student input, so the distribution mismatch might be caused by different reasons.
What is the performance when SFT the student on the provided dataset with reference answers (L1)?
I am curious what is the behavior of the stronger models. Have you tried 32B models?
What is the reason for the brevity (explicitly asking for sub 90 tokens in L2-L4?) The context seem oversimplistic.
Why the authors focused on LORA training and not full SFT? The learning rate for LORA seems to be small.
The experiments are only studied using a single dataset on a specific domain.
What is the reason of having the two compilers for the context modification? Which one was actually used in the main paper? It seems that there are not much performance differences across compilers.
Have you tried going beyond L1 and providing more context than the original answer to enrich the context?
In the OPSD dataset used for reference answer, what teacher has been used for the reference answer?
How meaningful are such abstractions to the model? Would a student not already know such abstractions to the problem ahead? While L2-L4 responses are more fitted towards the Qwen models, L1 is not fitted at all.
5 comments
Response to Reviewer mK4c (Part 2)
4. The two compilers are a control, and the small difference between them is the result (Q1, Q10).
The primary compiler is Qwen3.5-397B-A17B throughout the main tables; Qwen3-8B is a control, reported separately in Table 12. We use the larger model for the primary condition so that the abstractions are as well written as we can make them, which makes the comparison against L1 conservative rather than favourable to us.
Your observation that there is not much difference across compilers is exactly what the control was built to test. Recompiled by Qwen3-8B under the same contracts, all six abstractions still beat L1, so the effect does not need a strong writer:
| Student | L1 | L2 | L3 | L4 |
|---|---|---|---|---|
| 4B | 63.43 | 64.07 | 63.71 | 64.35 |
| 8B | 66.67 | 67.96 | 67.04 | 67.13 |
ID peak mean, with L2--L4 recompiled by Qwen3-8B. L1 is from the original dataset, needs no compiler and is shared. These are the L1 column and the Qwen3-8B block of Table 12.
At 8B that compiler is the student's own base model, so one model writes the abstractions, teaches from them, and learns from that teaching, with no external model at any stage. The result can then be applied without access to a frontier model, which is a stronger claim than the primary result alone.
5. Teacher-side training, and whether the discrepancy has another source (Q4).
Updating the teacher is a reasonable thing to try, by refreshing it every so many steps or by keeping an EMA of the student. SDFT (Shenfeld et al., ICML 2026, cited in our related work) does the latter, so the teacher's parameterization is plainly a live design axis rather than something self-distillation fixes. We keep ours frozen because that is OPSD's setting: Zhao et al. fix the teacher to the initial policy rather than the currently updating one, and report that this stabilizes training and acts as implicit regularization against drifting too far from the initial policy. Our L1 condition is OPSD, so changing the update rule would move the baseline we are intervening on.
It is also orthogonal to what we ask. SDFT's teacher is an EMA and is still conditioned on a demonstration; OPSD's is frozen and is still conditioned on a reference solution. However the teacher's weights are maintained, it has to be shown some context, and which context that should be is the variable we vary.
On the second half: the teacher's input differs from the student's by construction, and that is where the supervision comes from. Teacher and student are the same frozen network, so if the teacher read the student's prompt the two distributions would coincide and the loss would have no gradient. The asymmetry is the mechanism rather than a side effect of it, and it is how this family of methods is defined: SDFT writes the teacher as and the student as , the same network under different conditioning, and notes that learning from a teacher with privileged information requires on-policy sampling in the first place.
What does need separating is which part of the asymmetry does the work: the interface, meaning the teacher's thinking-enabled template, or the privileged content that the bridge introduces. Each has its own control. The no-hint reference scores the same fixed base-model rollouts through the thinking-enabled teacher interface with no privileged content at all, which isolates the template. Separately, the shared-bridge runs of Table 20 hold the introductory wording fixed across L1--L5, which isolates the bridge. Section 4.3 makes the qualitative point; the submission plots the means in Figure 2(a) without tabulating them, so we give them here:
| Teacher context | 1.7B | 4B | 8B |
|---|---|---|---|
| No hint, interface only | 0.1175 | 0.1315 | 0.1035 |
| L4, problem category | 0.1618 | 0.1757 | 0.1557 |
| L1, full solution | 0.2265 | 0.2240 | 0.1975 |
Mean teacher-to-student KL at initialization, answer-aligned, unclipped and over the full vocabulary. These are the quantities plotted in Figure 2(a).
The interface on its own accounts for a KL of 0.10 to 0.13, and the full solution adds a further 0.092 to 0.109 on top of that, so the privileged content is not a small perturbation on a format effect. The interface is also identical across L1--L5, and so is the student prompt, so whatever it contributes is constant across the conditions we compare. Every difference we report is between two contexts that differ only in what the hint says, and the whole L1 to L4 ordering sits inside that 0.092--0.109 span, taking up 44--59 percent of it.
Zhao et al. is the OPSD paper, ICML 2026; the table numbers we quote follow arXiv:2601.18734v3. SDFT is Shenfeld et al., "Self-Distillation Enables Continual Learning", ICML 2026; its EMA teacher and its formulation are in Section 3.
Response to Reviewer mK4c (Part 3)
6. SFT on L1 is already reported, on this dataset and these scales (Q5).
Zhao et al. Table 2 gives 1.7B averages of 37.1 for the base model, 35.8 for SFT on the same solutions, 37.7 for GRPO and 43.4 for OPSD, and attributes the SFT regression to the concise reasoning style of the OpenThoughts traces. We did not re-run it because our L1 condition is OPSD itself, so our baseline is already the stronger of the two.
7. Whether the student already knows these abstractions, and whether L2--L4 are fitted to Qwen while L1 is not (Q13).
This is the sharpest question in the review and it has two halves.
On the first: the student never sees the hint, at rollout time or at evaluation. The category is privileged teacher context. So the question is not whether the student could name the category but whether telling the teacher changes its scoring, and it measurably does. Reading the L4 and no-hint rows of the table in Part 2, the 17.7-token category raises teacher displacement to 1.38, 1.34 and 1.50 times the no-hint interface at 1.7B, 4B and 8B. If the information were already available to the model in that position, the distribution would not move.
On the second: two conditions in the study are not compiler-written at all. L5 is the final answer extracted verbatim from the reference, and prefix-10% is a literal prefix of the reference. Both match or beat L1 at 4B (+0.19 and +0.46 on the ID peak mean), so the effect cannot be an artifact of the teacher finding compiler-written prose more familiar. What remains is the residual by which the abstractions exceed prefix-10%, up to 0.93 points, and we accept that a style-fit account could still live there. The test is a compiler outside the Qwen family, and we have not run it.
8. 32B students, other model families, other training domains, and contexts richer than L1 (Q6, Q9, Q11).
These four questions all concern generalization beyond our current training setup. Our experiments use one model family and one training pool: Qwen3 at 1.7B, 4B, and 8B, trained on OPSD’s mathematics pool of 29,434 problems. We therefore limit our conclusions to this setting and leave broader validation across model families and training domains to future work.
Two points qualify the single-domain interpretation. First, the evaluation covers multiple domains, including multilingual mathematics on MT-AIME in seven languages, graduate-level science questions on GPQA-Diamond, and two families of logic puzzles, ZebraLogic and AutoLogi. Second, the effect is not uniform across model scales. Within the model sizes evaluated, the results indicate a transition between 1.7B and 4B, with the pattern persisting at 8B.
Our analysis focuses on L1 because it already provides both the derivation and the final answer, making it the most informative condition considered in our study. The existing results show that adding privileged information does not consistently improve utility. L1 produces the largest displacement, but not the highest utility, while adding the answer to the prefix condition still performs worse than all three answer-free abstractions. These findings suggest that providing more privileged information is unlikely to change our main conclusion.
What we will change. We will clarify in the data section that “verified” refers to the labels provided by the source pool and will describe the pool’s provenance and original labels. We will state in the evaluation section, rather than only in Appendix A.1, that thinking mode is enabled and include the Zhao et al. ablation citation there. We will also clarify in Appendix B that the typical lengths are used for calibration rather than as constraints, and move the no-hint results to the main text so that the interface control is presented alongside the KL analysis. Finally, to test whether the advantage of L2–L4 depends on their being generated by Qwen, we will regenerate them with a model outside the Qwen family and repeat the comparison.
Thank you again for these thoughtful comments, which will help us improve the clarity and scope of the paper.
Response to Reviewer mK4c (Part 1)
Thank you for the detailed list. Several of these questions we can answer directly from the submission, and where your reading of the setup differs from what we ran, that tells us the paper did not make the setup visible enough. We take the soundness rating seriously, so we start with the four points it appears to rest on.
1. The student does use thinking mode at evaluation (Q3).
This is stated in the submission in two places. Appendix A.1: the training rollout uses Qwen3's non-thinking chat template, teacher scoring uses the thinking template, and evaluation enables thinking mode. Section 3.3 gives the attribution in the main text: "Following Zhao et al., the teacher scores with Qwen3's thinking template while student rollouts use the non-thinking template, an interface difference common to L1--L5." Every Avg@12 number in the paper is therefore a thinking-mode score, sampled at temperature 1.0 with a 38,912-token budget.
The non-thinking rollout template is not a choice of ours and not an oversight. It is the ablated setting of the method we intervene on: Zhao et al. (Appendix B, Table 5) measure per-token KL across all four student/teacher thinking combinations at all three scales and select student-off with teacher-on because it maximizes the KL signal on mathematics tokens. Changing it would change the method rather than the teacher's context, which is the single variable this study manipulates.
2. Ground-truth answers are available, and the design makes solution quality a shared constant (Q2).
The training pool is OPSD's released snapshot of the OpenThoughts mathematics subset (
siyanzhao/Openthoughts_math_30k_opsd), 29,434 problems with source labels olympiads (21,315), math (5,351), aops_forum (2,291) and amc_aime (477). Every problem carries a reference answer. Those are the answers OPSD's own GRPO baseline verifies against, the answers our L5 condition extracts verbatim, and the answers the routing control in Appendix E checks boxed outputs against.Two clarifications that we think bear on soundness more than the provenance does. First, the teacher never produces an answer. It is frozen at the initial checkpoint and only assigns probabilities to tokens the student generated, so there is no teacher output whose correctness could be checked. Second, and more important: reference quality cannot confound this study. All five conditions are built from the same 29,434 problem--solution pairs, L1 is the trace itself and L2--L5 are derived from that same trace, and every comparison we report is paired on problem identity. A defect in the pool is shared by all five arms.
You are right about one thing here, and we will fix it: the paper says "verified reference solution" in its own voice. That property is inherited from the source pool and described by Zhao et al., not established by us, and we will attribute it accordingly.
3. There is no 90-token instruction (Q7).
The contracts in Appendix B give per-level typical lengths as calibration, 10--30 tokens for L4, 20--60 for L3, 30--90 for L2. The binding constraints are semantic, not metric: a ban on solution-action verbs plus a verb test for L4, a "via X"-removal test that deletes a prepositional clause and checks whether the sentence still classifies the problem, a method-name test, a verb-count test that flags an operation sequence in disguise, and a transferability self-check requiring the same L3 sentence to apply unchanged to a different problem in the same L4 category. Realized lengths come out well under the guidance (38.9, 25.2 and 17.7 tokens for L2, L3 and L4; Table 6).
On "the context seems oversimplistic": brevity here is the independent variable, not a limitation of the implementation. The finding is that a 17.7-token description of the problem trains a better 4B and 8B student than the 657.3-token derivation it was written from. A context that was not this simple would not test the question we are asking.