- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Paper mentions that privileged distillation can make a student better at predicting the final answer while simultaneously making it worse at actually performing the reasoning needed to reach that answer. Therefore, not necessarily better answer prediction means better reasoning, because the privileged teacher may have a high probability of producing the final answer immediately and when the student does reach the end, the privileged teacher provides a better answer signal. Also there are less training of later reasoning, as later reasoning steps receive training only if the student reaches them. Paper also compared different ways of assigning teacher targets (RETAIN, RELABEL, ON, and PROCESS).
- RETAIN (Keep the teacher identity that generated the trajectory and target will be generated by the same teacher condition)
- RELABEL(Keep the same prefix, but independently sample which teacher condition provides the target)
- ON (Prefix is generated by the student, while the teacher target is generated under an independently sampled privilege condition)
- PROCESS (Prefix is generated by the student; intermediate tokens use the unprivileged teacher as the target, while the terminal position uses a mixture of privileged and unprivileged teacher targets) An interesting result is that there is no universally best protocol. The effectiveness of each protocol depends on how the teacher condition, trajectory generation, and target assignment interact.
Critical strengths
- A clear and useful prediction–execution decomposition: The paper identifies an important failure mode of privileged OPD: privileged supervision can improve the student's final-answer prediction while simultaneously reducing its probability of completing the reasoning process. Separating forced accuracy, free accuracy, and completion makes this phenomenon directly measurable and provides a substantially more informative lens than final-answer accuracy alone. This is consequential because it offers a concrete explanation for otherwise puzzling cases where distillation improves conditional answer quality but harms end-to-end reasoning performance.
- A systematic comparison of different supervision and teacher-assignment protocols. The paper does more than evaluate a single distillation strategy: it compares multiple protocols, including ON vs. PROCESS and RETAIN vs. RELABEL, to study how the choice of teacher targets and the handling of teacher identity across trajectories affect learning. This is useful because it helps disentangle whether the observed behavior arises simply from privileged answer information or from how that information is propagated and assigned during the student's reasoning trajectory.
Critical weaknesses
- Limited evaluation on challenging reasoning tasks and larger models: The evaluation is primarily conducted with a Qwen3-1.7B model and relatively simple tasks such as sequential lookup and GSM8K. It remains unclear whether the observed prediction–execution tradeoff persists for substantially harder reasoning benchmarks, where longer and more careful reasoning may be genuinely necessary (e.g., AIME), or for larger reasoning models. Evaluating the method on challenging mathematical reasoning datasets such as AIME, would substantially strengthen the claim that the observed execution effects generalize beyond the controlled experimental setting.
- The paper does not sufficiently distinguish harmful premature termination from beneficial shorter reasoning: The theoretical setup has a fixed number of required intermediate computations, making "completion" well-defined, but natural reasoning tasks generally do not have a known required reasoning length. Consequently, reduced thinking-token counts on GSM8K cannot by themselves establish degraded execution; shorter trajectories may reflect more efficient reasoning rather than premature termination.
Questions for authors
- EARLY-PRIV deliberately gives the teacher an 80% probability of continuing and 20% probability of producing the correct answer early at intermediate decisions. The purpose is to isolate the execution mechanism. I understand that if privileged supervision changes the teacher's stopping behavior, that behavior can transfer to the student. But does naturally occurring privileged distillation in real OPSD systems actually cause this problem? Is EARLY-PRIV realistic?
- Is shorter reasoning necessarily bad? There are some previous work mentioning shorter reasoning can be beneficial. I agree that premature answers can be harmful, but early answer does not mean bad execution.
- Lookup task has a known required computation depth, but a model could legitimately answer(in your case solve a math problem) without a long chain.
- Suggest having evaluation on more challenging reasoning tasks, and larger models.
3 comments
Thanks for the detailed response and the additional experiments. The EARLY–ZERO comparison is helpful and provides stronger evidence for the proposed mechanism in the controlled setting. The AIME and MATH500 results also improve the evaluation.
However, my main concern remains partially unresolved. The larger-model experiments use checkpoints that were not trained under matched conditions, so it is difficult to attribute the observed accuracy differences to privileged distillation. Also, the continuation experiments do not show a clear accuracy improvement from additional reasoning.
I appreciate your efforts to address my concerns. I find the idea interesting, but I think further evidence is needed. I therefore maintain my Weak Reject rating, although I acknowledge that the additional experiments have strengthened the paper.
Thank you for your thoughtful follow-up and for recognizing that the additional experiments strengthen our paper. We appreciate your acknowledgment of the EARLY–ZERO comparison and the expanded evaluation on AIME and MATH500.
We agree that the unmatched 8B checkpoints do not establish a causal effect of privileged distillation, and that our continuation experiments do not demonstrate a clear accuracy improvement from additional reasoning. These are important limitations that we will make more explicit in the revised manuscript.
We will also sharpen the distinction between the mechanisms established in our theoretical and controlled settings and the observations on natural reasoning tasks, ensuring that our conclusions accurately reflect the available evidence.
Thank you again for your careful evaluation and constructive feedback, which have helped us improve both the experimental validation and the presentation of our work.
Thank you for emphasizing the distinction between harmful premature termination and efficient, concise reasoning, and for suggesting harder tasks and larger models. We agree that a short successful solution can be preferable to a longer one.
Our goal is to characterize a setting in which the computation required by the task is observable, and then examine which aspects of that mechanism are supported by language-model experiments. Below, we clarify this distinction and report the expanded evaluation and matched control.
Our completion measure is tied to the specified computational task. In the staged-reveal construction, future lookups are revealed through the task interface, and completion requires five correct intermediate states before answering. This gives an observable criterion for execution that a natural-language token count does not provide.
For natural mathematics, a student may legitimately skip a verbose derivation or find a shorter valid route. The paper's natural-language length results do not distinguish such efficiency from omission of necessary computation. We will make this limitation explicit in the abstract and experimental interpretation.
We evaluated fixed public Qwen3-8B BASE, math-SDPO, and math-GRPO checkpoints on both complete AIME test sets:
On all 500 MATH500 problems, BASE and SDPO score 488/500 and 473/500, respectively. The paired SDPO-minus-BASE difference is −3.0 pp (95% CI: [−5.0, −1.2]), while mean generated length decreases from 5,474 to 2,653 tokens.
These evaluations broaden the observed coexistence of shorter output and lower accuracy to harder mathematics and 8B models. They do not separately measure terminal prediction and necessary-computation completion. The public checkpoints were not retrained under matched conditions, and SDPO uses JSD/TopK100/EMA0 rather than the paper's forward-KL protocol. Each problem uses one fixed sampling stream. We will present these results as expanded empirical coverage with these qualifications.
EARLY's 20% early-answer mass is an experimental intervention, not an estimate of a universal rate in natural distillation. Its purpose is to test the consequences of early-answer supervision in a task where required computation can be measured.
In a separately recalibrated reconstruction, we added ZERO with the same legal-action normalization and trie construction, setting early-answer mass to zero in the adapted privileged intermediate-action branch. The natural unreferenced branch and off-support fallback remain. At the primary update-8 checkpoint, EARLY minus ZERO reduces completion by 26.48 pp (95% CI: [−31.17, −21.95]) and free accuracy by 22.19 pp (95% CI: [−26.49, −17.81]).
We also classified the first failure of completion across five fitted models evaluated on the same 256 questions:
Within this controlled task, completion failures are predominantly early answers. These counts are repeated evaluations of 256 shared questions, rather than 1,280 independent problems. The confidence intervals likewise condition on the common calibration and fixed fits. Forced accuracy also decreases under EARLY, so this reconstruction does not reproduce the original three-way signature of improved forced accuracy with worse completion and free accuracy.
To examine the distinction on natural tasks, we conducted two exploratory AIME continuation interventions. Neither established an accuracy recovery attributable to additional thinking. The distinction between concise reasoning and harmful premature termination therefore remains unresolved for the observed natural-task accuracy losses.
A subsequent audit found that all 107 AIME native outputs containing a closing marker already contained a complete boxed integer candidate before that marker; the remaining 13 outputs had no closing marker. This locates the intervention after an explicit candidate has appeared. It does not establish that necessary computation was complete, explain the unsuccessful outcome, or show that an earlier intervention would succeed.
Thank you again for this distinction and for the suggestions on evaluation. We will incorporate the harder-task results and matched control while making clear that the natural-task observations do not establish that shorter reasoning itself caused the accuracy losses.