- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper studies the instability of on-policy self-distillation (OPSD), which uses predictions from the same model conditioned on privileged reasoning instructions as teacher signals. However, these instructions may introduce unwanted changes in response style alongside useful reasoning corrections, potentially destabilizing training. To address this issue, the authors propose Locality-Guided Self-Distillation (LGSD), which constructs a training target by interpolating between the student and teacher distributions. The interpolation coefficient is adaptively selected for each response to keep the average KL divergence between the target and student within a predefined threshold. Experiments on mathematical reasoning benchmarks show that LGSD improves accuracy by 10.49 percentage points over OPSD on Qwen3-8B while reducing style drift.
Critical strengths
- The proposed method is straightforward to implement and introduces little additional computational overhead, making it easy to integrate into existing self-distillation pipelines.
- The method achieves substantial performance gains over OPSD on mathematical reasoning benchmarks while mitigating performance degradation during prolonged training. Although the results are encouraging, stronger evidence across more random seeds, additional models, and longer training runs is needed to establish their robustness. The underlying mechanism also remains insufficiently explained.
Critical weaknesses
-
KL regularization and geometric interpolation between student and teacher distributions are well-established techniques in post-training. Closely related methods, particularly -OPSD [1] and Veto [2], already employ similar target constructions. The paper does not sufficiently clarify the methodological novelty of its response-adaptive KL projection beyond these existing approaches.
-
KL projection uniformly scales teacher–student log-probability differences without distinguishing useful reasoning corrections from undesirable stylistic changes. It remains unclear why partial distillation should outperform full distillation, or whether the latter could achieve comparable performance with better-tuned training. Moreover, Table 9 shows higher training-seed standard deviation for LGSD than OPSD (2.46 versus 1.61), despite higher mean accuracy. Thus, the mechanism behind the improvements and the claimed training stability remain insufficiently explained.
-
Although the paper includes several ablations, more targeted comparisons are needed to isolate the contribution of response-adaptive projection. In particular, comparisons with properly tuned fixed or scheduled interpolation coefficients under matched target KL would be valuable. Evaluations with privileged teachers of varying quality and additional training seeds would further strengthen the evidence for generality and robustness. A comparison with -OPSD would also help clarify the relationship to concurrent work.
[1] Xu, Jiawei, et al. "$\beta $-OPSD: Deriving with Policy Optimization, Training with Self-Distillation." arXiv preprint arXiv:2607.28582 (2026).
[2] Jang, Ijun, et al. "Stable on-policy distillation through adaptive target reformulation." Findings of the Association for Computational Linguistics: ACL 2026. 2026.
Questions for authors
- Could the authors explain more clearly the mechanisms behind the accuracy and stability improvements, particularly given the higher training-seed variance in Table 9?
- Could the authors provide additional ablations, as suggested in the weaknesses, to verify the robustness of the improvements and isolate the contribution of the proposed method?
2 comments
Dear Reviewers,
This is a friendly reminder that the rebuttal discussion period is approaching its deadline.
Please take some time to review the authors' rebuttal and actively participate in the discussion. If you have any remaining questions or concerns, please raise them and share your feedback with the authors and other reviewers.
As the discussion period is about to close, we kindly ask you to complete your discussion and provide your updated assessment before the deadline.
Thank you for your time and effort.
Best regards, Area Chair
We sincerely thank the reviewer for their valuable feedback. We would greatly appreciate hearing if there are any remaining questions or concerns regarding our rebuttal or submission, and we would be happy to provide further clarification or additional evidence