- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies on-policy distillation (OPD) with privileged information, such as step-wise hints for LLMs or bounding boxes for VLMs, given to the teacher, the student, or both. The authors argue that this can create a “privilege illusion”. A preliminary experiment (Fig. 2) supports this, showing that adding privileged information does not necessarily improve distillation and can lead to late-stage degradation and entropy collapse. To address this, the authors propose DOPD. Both the teacher and the student receive the privileged input, and each token is routed to one of four loss terms based on the teacher-student log-probability gap and the two models’ predicted probabilities. Experiments are conducted on multiple benchmarks, plus additional analyses on teacher-student size ratios, continual learning, out-of-distribution transfer, and training stability. DOPD achieves substantial gains over vanilla OPD plus some gains over other strong baselines, although all experiments are limited to the Qwen model families.
Critical strengths
- the paper studies a relevant and timely problem: what is the best practice for using privileged information to improve supervision in on-policy distillation. The paper provides more insights to the problem and a strong empirical baseline for future research.
- The proposed method (DOPD) is simple. Although i have not checked the source code, the method seems easy to implement and relevant details are provided in the paper.
- The experiments are multi-faceted, making the main empirical results convincing. However, it does not outweigh the lack of theoretical justifications.
Critical weaknesses
- The proposal is purely intuitively driven. There is no theoretical justification for why the routing in the given way works. Besides, although I appreciate that the authors provide the examples in Figure 7, i find the classification of the four types of tokens are superficially random, where some deeper theoretical reasons might be hidden.
- The intuitive justification itself seems handwavy. The motivation is to tackle the “privilege illusion” and to separate capability from information for effective distillation. However, the routing strategy essentially just measures teacher-student disagreement and confidence levels. The paper never shows how exactly such strategy separates capability from information and how this resolves the "illusion". It is possible that the teacher and student are subjected to the illusion at the same time while they can agree or disagree on some tokens. Moreover, the concept of "capability" and "information" are vague and never been properly defined in the first place. Therefore, the authors should either reframe the contribution, or provide better groundings for the intuitive claims.
- Although the experiments cover a large set of benchmarks, the models are limited to the Qwen3 and Qwen3-VL families, with the teacher-student pairs differing only in size. It is unclear whether the gains hold for other families (e.g., Llama, Gemma) or thinking models. so the claims of “robustness, scalability, and generalization” are stronger than the evidence.
- The ablation experiments in Table 6 are insufficient. First, The non-adaptive baseline is only defined in one sentence (Appendix D.3) and it is unclear how each non-adaptive baselines are implemented. Second, adaptive and non-adaptive rows differ in many factors at once, making it hard to make fair comparisons and to isolate the contribution of each type of tokens.
Questions for authors
- How does the routing-based token selection criterion of DOPD differ from the criteria of VA-OPD, TIP, and EOPD in principle besides the empirical results?
- Why are not other related methods mentioned in related work compared in the experiments? e.g., TA-OPD, REOPOLD, TSD-KD, given that they are all based on some adaptive selection rules?
- As mentioned in the weaknesses section, can the author provide evidence of the claimed generality using more model families?