- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper addresses the cost of keeping a teacher model running throughout online distillation. Lightning OPD collects student responses and caches teacher scores once, then trains the student offline. The authors advocate using the same teacher for SFT and distillation to reduce distribution shift when reusing fixed trajectories, supported by theoretical analysis and teacher pairing experiments. On math and coding tasks, the method achieves performance close to online OPD in the evaluated settings while reducing the distillation GPU cost for Qwen3-8B to one quarter.
Critical strengths
-
The method is simple and practical, removing the overhead of keeping a teacher running during student training. In Table 2, the distillation cost for the 8B model drops from 120 to 30 GPU hours, including offline preprocessing.
-
The experiments cover multiple models and both math and coding tasks, with overall performance close to online OPD. The teacher pairing experiments also offer a useful observation that teacher choices during SFT and distillation affect offline training performance.
Critical weaknesses
-
Table 3 uses different SFT checkpoints and distillation durations, making it difficult to attribute the performance differences to teacher consistency. Figure 3(b) controls the budget but reports only score fitting loss, without the corresponding task performance.
-
Table 1 lacks comparisons with existing offline distillation methods under the same settings, while results for GKD and AOPD come from other papers. It remains unclear how much additional performance or cost benefit the objective based on a single sampled token score provides over existing offline approaches.
Questions for authors
-
With matched SFT steps and distillation budgets in Table 3, does teacher consistency still reduce the task performance gap between offline and online methods?
-
With the same initialization, teacher, and data, how does the proposed method compare with existing offline KD objectives in accuracy and GPU cost?
3 comments
I appreciate the authors' efforts in addressing my concerns through the additional results. After considering the rebuttal, my final rating is 2.
Dear Reviewer XaKx,
Thanks for your response. We sincerely hope you will take the opportunity to reconsider our work. We are also open and eager to engage in further discussions with you if you have any further concerns. Thank you once again for your time and consideration.
Rebuttal for Reviewer XaKx
W1/Q1. Teacher consistency with matched SFT steps and distillation budgets.
Thank you for this question, and we apologize that the caption of Table 3 was unclear. The SFT steps and distillation budgets in Table 3 are already matched.
Within each row, the OPD teacher is the only factor that varies. We have rewritten the caption to state this.
Teacher consistency reduces the offline–online task gap. Lightning OPD minus standard OPD on AIME 2024 (points), from Table 3:
With the consistent teacher, Lightning OPD matches or exceeds standard OPD in every row (+0.3 to +2.7). With a switched teacher, it is level in one row and behind in the other three (−1.7 to −2.9).
Link to Figure 3(b). Figure 3(b) is a short budget-matched diagnostic (20 updates) on the same two 4B initializations and two OPD teachers as the 4B block of Table 3. The two agree: a teacher change raises the score-loss gap G=Loff−Lon by 0.0067 and 0.53, and lowers Lightning OPD's AIME 2024 margin over standard OPD by 2.6 and 2.2 points.
We have added the offline–online differences above to Appendix C.3, which details Figure 3(b).
W2/Q2. Comparison with existing offline KD objectives under the same settings.
Thank you for this suggestion. We ran the requested comparison. All methods start from the same 4B SFT student and use the same Qwen3-8B teacher, 17,398 prompts, 150 updates (batch 256, learning rate 2×10⁻⁶) and evaluation. The top-10 objectives train on the same cached student trajectories as Lightning OPD.
Accuracy.
Cost. H100 GPU hours for the distillation stage, with the same accounting as Table 2.
We have added these controlled baselines and their costs to a new Appendix C.5.