- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Multi-Draft On-Policy Distillation (MDraft) for training parallel diffusion drafters. It identifies conflicting teacher supervision caused by the drafter predicting tokens independently while the teacher conditions on sampled prefixes. MDraft aggregates teacher predictions across multiple drafts and blends them with individual-prefix predictions, improving acceptance length without changing the drafter architecture or inference procedure.
Critical strengths
The paper identifies a supervision mismatch in parallel drafter training
The paper provides some theoretical analysis explaining its effect on standard distillation objectives.
The evaluation shows some improved acceptance lengths on different benchmarks
Critical weaknesses
Narrow methodological scope. The paper focuses specifically on refining OPD for parallel diffusion drafters by aggregating teacher predictions across sampled prefixes, which represents a relatively narrow contribution.
Experimental attribution. The original DFlash checkpoint receives no additional training, and no continued-SFT baseline is provided. Furthermore, applying common optimization settings across different objectives does not establish that each baseline is comparably well tuned. Could you provide more ablations and hyperparameter sweeps for the competing objectives to isolate the benefit of the proposed method?
Limited evidence of general applicability. All experiments use a single Qwen3-4B target and the same DFlash drafter, with thinking disabled and a fixed block length. The evaluation does not establish whether the proposed improvement holds across model families, larger targets, reasoning configurations, or drafters that explicitly model dependencies between proposed tokens.
Modest gains over the strongest training baselines. In Table 1, the mean acceptance-length improvement over the strongest baseline is approximately 0.21%, 2.48%, and 3.09% at temperatures 0, 0.6, and 1, respectively. Moreover, Appendix D shows that AdaFlash with survival weighting slightly outperforms MDraft at temperature 0. These results indicate a relatively small incremental benefit over existing training objectives
Questions for authors
see weakness
3 comments
-
3. General applicability (W3).
We have expanded the evaluation to Qwen3-8B, training MDraft and AdaFlash starting from the published DFlash drafter for this model. The full results are included in Section 4.2 and Table 2 of our new revision. MDraft achieves the highest acceptance length on all six benchmarks at and . At , its mean acceptance length is close to AdaFlash's (6.631 versus 6.655). The gains established in the 4B experiment broadly transfer to the larger model: at , MDraft’s mean relative acceptance-length improvement over pretrained DFlash reaches 13.4%, compared with 11.9% for Qwen3-4B.
We acknowledge that experiments across verifier families, reasoning configurations, block lengths, and different drafter architectures would be useful. Within our constrained compute budget, these additional experiments establish that MDraft’s practical improvements extend to a larger verifier.
-
4. Experimental attribution and baseline tuning (W2).
Continued DFlash training: to isolate the benefit of on-policy training, we provide an off-policy baseline by continuing the training of the Qwen3-4B DFlash drafter with its original position-weighted cross-entropy objective [5], for the same number of steps used by the on-policy methods. We evaluate the resulting checkpoint using the same protocol and include the results in the main experiment table in Section 4.1, in the row labeled SFT. Relative to the pretrained drafter, continued SFT improves mean acceptance length by 1.9% at and 0.7% at , but lowers it by 2.6% at . MDraft improves it by 3.9%, 7.3%, and 11.9%, respectively. MDraft’s substantially larger stochastic-decoding gains therefore cannot be explained simply by continuing training with the original objective.
Baseline configurations: while we cannot apply full hyperparameter tuning for all competing baselines, we use the objective-specific hyperparameters specified in the original Draft-OPD [3] and AdaFlash [4] papers, including the coefficient in AdaFlash, and the combination of losses on accepted and rejected tokens in Draft-OPD. The objective configurations and optimization setup are described in more detail in Appendix E.
Stronger AdaFlash baseline: we also evaluate AdaFlash with the same survival weighting used by MDraft (Table 6 in Appendix D). As detailed above, MDraft retains its advantage at and , showing that survival weighting alone does not account for the gains.
References
- Rishabh Agarwal et al. On-Policy Distillation of Language Models: Learning from Self-Generated Mistakes. 2024.
- Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. MiniLLM: Knowledge Distillation of Large Language Models. ICLR 2024.
- Haodi Lei et al. Draft-OPD: On-Policy Distillation for Speculative Draft Models. 2026.
- Yu-Yang Qian et al. AdaFlash: Adaptive Speculative Decoding via On-Policy Distilled Diffusion Drafters. 2026.
- Jian Chen, Yesheng Liang, and Zhijian Liu. DFlash: Block Diffusion for Flash Speculative Decoding. 2026. Official code.
Dear reviewer 7wuC,
Thank you for your careful review and helpful feedback. As the discussion period draws to a close, we would appreciate your thoughts on whether our additional experiments and clarifications address your concerns. We would be happy to answer any remaining questions.
We thank the reviewer for their careful consideration of our work and for recognizing the conditioning mismatch that we identify, the accompanying analysis, and the improved acceptance lengths. We address the reviewer’s concerns below:
1. Magnitude of the improvements and survival weighting (W4).
MDraft delivers consistent gains under stochastic decoding and measurable inference speedups through a drop-in drafter checkpoint. In the main comparison in Table 1, MDraft ranks first in 15 of the 18 benchmark–temperature settings and second in the remaining three. It leads on all six benchmarks at both T=0.6 and T=1, improving mean acceptance length over pretrained DFlash by 7.3% and 11.9%, respectively. This consistency across tasks is a central practical benefit of the method.
The gains persist with matched weighting: Table 6 in Appendix D directly addresses whether survival weighting accounts for the improvement. At T=1, MDraft outperforms AdaFlash by 6.0% with survival weights for both methods (5.903 versus 5.568), and by 5.2% with uniform weights for both (5.836 versus 5.546). At T=0.6, the corresponding gains are 1.6% and 1.5%. MDraft's advantage therefore holds under either matched weighting scheme. With uniform weighting throughout, its improvement over pretrained DFlash is 1.33× that of the strongest baseline at T=0.6 (AdaFlash), and 1.24× at T=1 (RKL). These ratios compare the improvements over the same pretrained drafter.
At T=0, survival-weighted AdaFlash's lead is only 0.006 accepted tokens (6.691 versus 6.685). With uniform weights for both methods, AdaFlash leads by 1.6% (6.671 versus 6.568). We acknowledge this greedy-decoding result; MDraft's strongest benefits occur under sampling, where it consistently leads in acceptance length under both weighting schemes.
Measured inference benefits: Figure 2 shows that these improvements translate into higher throughput. At T=1, MDraft is 6.9% faster than DFlash on GSM8K and 10.5% faster on MBPP, and 3.0% and 7.4% faster than AdaFlash, respectively. We measure tokens per second using the officially released DFlash testing codebase [5], with Transformers-based inference on a single NVIDIA H200 GPU and 128 samples per dataset, using the same checkpoints as in the acceptance-length evaluation. At T=0, MDraft and AdaFlash achieve similar throughput. These speedups are obtained by replacing the drafter checkpoint, with no changes to the inference architecture or verification procedure.
2. Scope and significance (W1).
We respectfully disagree with the description of our contribution as too narrow. On-policy distillation is an established approach to language-model training [1, 2], and recent work has demonstrated its value for speculative drafters [3, 4]. This establishes drafter on-policy distillation of drafters as a valuable and promising research direction for LLM inference acceleration. Our paper identifies a structural obstacle to applying this approach to parallel drafters: the drafter must learn a shared prediction from conflicting, prefix-dependent verifier targets. We explain the consequences of this mismatch and introduce a simple method that uses cross-prefix information to improve distillation. This addresses an important training problem while preserving the architecture and inference efficiency that make parallel drafters attractive.
The contribution also includes an explanation of what standard distillation objectives learn in this setting. Section 4.5 characterizes the optimal shared token distribution for a fixed context and position, a fixed draft-prefix distribution, and uniform position weights. Per-prefix forward KL yields the arithmetic average of verifier distributions, while reverse KL yields their normalized geometric average. We express reverse KL as matching average verifier support plus a penalty for disagreement across prefixes, and show how MDraft reduces that penalty. This explains why per-prefix reverse KL can suppress useful branch-specific tokens and provides a principled basis for the target construction.
MDraft changes only the training target, so the resulting checkpoint can replace the original drafter without changing its architecture or inference procedure.