- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Flash-OPD, an efficient on-policy distillation framework that separates the update horizon from the full response. Instead of waiting for a complete rollout, Flash-OPD trains each newly generated suffixes, updates and synchronizes the student, and then resumes unfinished trajectories. It further overlaps teacher evaluation with ongoing rollout generation and adopts a memory-efficient full-vocabulary loss. The paper reports substantial wall-clock speedups over vanilla OPD while maintaining similar downstream performance.
Critical strengths
- The consume-before-continution design is intuitive and practically motivated. It allows the student model to learn from a newly generated suffix immediately, while still preserving and continuing the unfinished trajectory after the update.
- The paper reports promising wall-clock improvements over vanilla OPD. Flash-OPD reduces training time from 8.5 to 4.2 hours on math, 6.3 to 3.4 hours on code, and 12.7 to 7.9 hours on agentic training. The reported downstream score remains largely unchanged relative to vanilla OPD.
Critical weaknesses
- Table 1 only compares the proposed Flash-OPD with vanilla OPD. However, several recent approaches are closely related and already target the same rollout-efficiency bottleneck. More specifically, [1] early terminates student rollouts and report large reductions in training FLOPs. [2] explicitly controls the rollout horizon and reports up to 3 times improved training efficiency. [3] decouples rollout generation and learner updates and reports similar improved throughput. This work discusses these baseline work but no matched experiment is provided to show whether the proposed scheme is actually better in wall-clock efficiency or downstream performance. While the proposed scheduling idea is interesting, it is unclear whether it provides a meaningful advantage over existing fast OPD methods. The current experimental evaluation is below the acceptance bar. 2 The paper lacks an end-to-end ablation of its proposed two main scheduling components, i.e. partial updates and teacher overlap. Table 1 only compares vanilla OPD with the whole Flash-OPD system. Appendix D explicitly states that its component measurements do not attribute the end-to-end gains to individual components. Therefore, the source of the reported training speedup remains unclear.
References: [1] Zhang et al. Fast and Effective On-Policy Distillation from Reasoning Prefixes. 2026. [2] Zhang et al. Are Full Rollouts Necessary for On-Policy Distillation? 2026. [3] Kang et al. AsyncOPD: How Stale On-Policy Distillation Be? 2026. [4] Rang et al. Near-Policy: Accelerating On-Policy Distillation via Asynchronous Generation and Selective Packing. 2026
Questions for authors
Questions:
- Could the authors provide matched performance comparisons against recent efficient OPD methods (particularly those included in references) under the same compute budget and downstream evaluation? Such comparisons are necessary to demonstrate the practical benefit of the proposed design.
- Could the authors provide an end-to-end ablation separating partial updates from teacher overlap? This would help clarify how much of the reported speedup is from the proposed consume-before-continuation design.