- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper addresses the distribution mismatch problem between teacher and student models in on-policy distillation, particularly the distributional differences encountered in cross-source distillation scenarios. The authors systematically compare three supervision strategies:Online SFT, Teacher Top-K, and Hybrid Teacher Top-K. Experimental results demonstrate that, under the settings examined, Teacher Top-K consistently outperforms the other two strategies.
Critical strengths
-
The paper is well-written, clearly structured, and properly formatted, offering good readability.
-
The authors conducted comprehensive experiments comparing various distillation strategies and further analyzed why Teacher Top-K performed better under the examined settings, providing empirical evidence regarding the impact of different supervision strategies on distillation performance.
Critical weaknesses
1 The authors present Online SFT and Hybrid OPD as two methods proposed in this paper, but provide insufficient explanation of their relationship to prior work and their incremental contributions. Specifically, the paper does not discuss the relationship between Online SFT and ImitKD, nor does it clearly explain the substantive differences between Hybrid OPD and existing mixed-distribution distillation objectives.
2 The novelty of the paper is relatively limited. Both Online SFT and Hybrid OPD, which the authors claim to propose, have fairly direct precedents in prior work, but their substantive differences from and incremental contributions over prior work have not been sufficiently clarified. Furthermore, in the experiments reported in this paper, neither method outperforms the existing Teacher Top-K OPD approach, failing to demonstrate a performance advantage over the existing approach.
Questions for authors
1 What specific hypotheses did the authors aim to test with Online SFT and Hybrid Teacher Top-K? Since comparisons with Teacher Top-K involve variations in supervision forms, loss functions, and target distributions, could controlled experiments be conducted to isolate the impact of these individual factors on performance?
2 Although the paper covers Qwen3 student models of various sizes, the experiments are limited to a single model family. Have the authors validated their main conclusions using student models other than Qwen3?
3 Section 4 defines the support set for Hybrid Top-K as the Top-K set selected by the student; however, Section 4.1 states that a teacher weight of 100% corresponds to Teacher Top-K. Based on the aforementioned definition, this scenario should yield a teacher distribution match over the student's Top-K support set. The authors are requested to clarify which support set was actually used in the experiments and whether the method of constructing the support set remained consistent across different mixing ratios.
4 Section 5.3 finds that K=32 yields the best performance under the settings examined. Does this choice remain optimal across different teacher and student sizes, as well as different tasks?
We thank all reviewers for their careful reading and constructive feedback. We are glad that the reviewers appreciated our investigation of how to effectively distill large, strong teachers into smaller students. In particular, the distribution-mismatch analysis was found convincing (dc8P), and the analysis of renormalization and objective implementation was recognized as informative and practically useful (gj54, dc8P). The reviewers also appreciated the empirical comparisons of supervision strategies (LeKj), the consistent gains over student-preferred OPD across the evaluated settings (gj54, dc8P), and the investigation of teacher strength, student capacity, and training budget (dc8P).
The main concerns were the positioning of our contribution relative to prior work, the breadth of the teacher–student settings, the absence of alternative distillation baselines at 14B, comparisons with other objectives, the definition of Hybrid OPD, the robustness of the choice of K, and the computational cost of teacher Top-K supervision. We have added experiments and provided clarifications in our individual responses, as summarized below.
Our individual responses provide the point-by-point explanations and full experimental tables behind this summary. The additional experiments broaden the evidence for our findings, while our conclusions remain scoped to the configurations and objectives evaluated.
We are sincerely grateful to the reviewers for helping us make the paper more precise and better supported, and to the Area Chair for their time and consideration. We look forward to continuing the discussion.
Best regards,
The Authors