acceptodds
Under review as a conference paper at ICLR 2027

Diffusion Drafters Can Predict Beyond Their Training Blocks

Abstract

Diffusion draft models predict fixed-length blocks of candidate tokens for parallel verification by a target model, and extending these blocks offers an opportunity to accept more tokens per round. However, naive extension can disrupt original predictions through bidirectional attention, mixing gains from added predictions with losses at the original positions. To assess the benefit of added predictions, we study whether frozen draft models can use their beyond-block predictive capability while preserving accuracy at the original positions. We introduce Native-Preserving Extrapolation (NPE), which prevents original positions from attending to added positions while allowing added positions to use representations from the original block. Across four frozen DSpark and DFlash models, NPE preserves original-position accuracy and provides useful additional predictions that increase the number of tokens accepted per round. With the original predictions preserved, we further analyze what supports or disrupts the added predictions, when these predictions accelerate generation, and what further training adds to prediction quality. Our findings show that a draft model’s useful prediction range can extend beyond its training block without sacrificing accuracy at the original positions.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Speculative Decoding, Diffusion Language Models, Block-Length Extrapolation, Training-Free Inference, Inference Acceleration

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_aa8hOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

Parallel drafters like DFlash and DSpark uses bidirectional attention with a fixed block size. A naive way to extend the block size would be by doing the bidirectional attention on the whole extended block. The paper raises the problem that top-1 accuracy of original block positions drops significantly because the model was not trained for longer blocks. To mitigate that, the paper suggests to mask the tokens beyond the original block, thus keeping the original predictions for the earlier tokens, while possibly benefiting from additional accepted tokens. The experiments show speedup using that simple method, compared to the naive extension.

Critical strengths
  1. The paper is well written, and the motivation is clear. The figures are informative and relevant.
  2. The method is simple to understand and implement, and is training-free.
  3. Experiments show speedup compared to using the naive extension.
Critical weaknesses
  1. There is limited novelty and the method is quite predictable in retrospect. This could also be seen as a strength.
  2. I think there is another ablation that is missing: training the drafter with causal attention for all positions. If bidirectional attention is not necessary, causal attention could work better for both shorter and longer block sizes.
  3. The experiments were conducted only with batch size 1, and not in a high performance serving framework like vLLM or SGLang. It could be useful to see the speedup on these.
Questions for authors

Do you think it's better to train DFlash/DSpark with causal attention to avoid such complications?

Reviewer_EZj2OpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether diffusion-based draft models can generate useful predictions beyond their training block lengths without degrading accuracy at the original positions. Motivated by the observation that naive block extension introduces interference through bidirectional attention, this paper proposes Native-Preserving Extrapolation (NPE), which preserves the original predictions while allowing additional positions to leverage existing representations. Through experiments on frozen DSpark and DFlash drafters, the paper analyzes the effects of attention interference, verification reach, and additional training on prediction quality and decoding efficiency. The results demonstrate improved acceptance lengths across all four evaluated drafters, with throughput gains of up to 18.3% in the tested configurations.

Critical strengths

Simple and effective training-free Design. The paper presents a clear motivation and a simple yet effective approach to extending the prediction length of diffusion drafters without additional training. NPE improves acceptance length across the evaluated models while largely preserving the original predictions, demonstrating its practical value.

Critical weaknesses

The proposed NPE mechanism is relatively straightforward, relying mainly on attention masking to preserve original predictions. Although NPE improves acceptance length, its additional throughput gains over naive extension are relatively modest on DSpark-4B and DSpark-8B (3.1% and 1.5%, respectively), and it still underperforms the original block on DFlash-b16. These results raise concerns about the practical significance of the proposed method. More convincing throughput improvements over optimized training-free baselines, or a direct throughput comparison with drafters trained on longer blocks, would help demonstrate its value. Stronger evidence in these directions could positively affect my overall assessment.

Questions for authors

Table 4 compares acceptance length between NPE and drafters trained on longer blocks. Could the authors also provide a direct throughput comparison under matched inference settings? This would help establish the practical value of NPE as a training-free alternative.

Reviewer_jchROpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether a draft model trained to predict a fixed number of tokens can make useful predictions further ahead without retraining. Simply adding more masked positions can change predictions at the original positions. Native-Preserving Extrapolation (NPE) changes the attention mask in every draft layer: new positions can read the original ones, but the original positions cannot read the new ones. The authors test four frozen DSpark and DFlash drafters paired with Qwen3. They measure accuracy on fixed prefixes, then run decoding on 2,320 prompts from five benchmarks, one request at a time. Tokens per round count the accepted draft tokens plus one from the target. The target accepts some predictions beyond the training block. This speeds up both DSpark models, but not the longer DFlash model. Separate training experiments show that direct training at longer offsets still improves prediction there.

Critical strengths
  1. The controls show that the new positions do useful work. DFlash had already shown that plain extension can hurt acceptance [1]. Wu separated losses at early positions from gains at new ones [2]. Jin et al. also tested a similar one-way mask in a standalone diffusion generator [3]. What this paper adds is a direct test in a frozen drafter whose proposals a separate target checks. On DSpark-4B, NPE keeps the original top prediction at 99.4% of the tested native positions, versus 74.0% for plain extension (Table 6, 7 to 10 slots). On DFlash-b7, reusing the last original state at all three new depths gives 6.326 tokens per round; computing the new states with NPE gives 7.026 with the same tree shape (Table 9). A deeper tree alone does not explain that difference.

  2. The training comparison shows what average acceptance can hide. The authors continue training DSpark-4B on the same data at lengths 7 and 12. With NPE and causal attention among the added positions, the shorter model comes within 0.9–1.5% of the longer model in tokens per round across two training seeds (Table 18). Yet, for the first pair, with the same text and inference mask, the models predict the token at offset 12 correctly 10.5% and 14.4% of the time, respectively (Table 20). A better prediction at offset 12 can affect the round only after the first 11 tokens have been accepted.

Critical weaknesses
  1. The abstract should name the baseline for its four-model claim. NPE beats plain extension in tokens per round on all four checkpoints. Only three have a matched comparison with the original, unextended block. For DFlash-b7, Table 9 reports plain extension and the repeated-last-state control, but neither is the original run. The choice also matters for speed: DSpark-4B and DSpark-8B are 18.3% and 16.4% faster than their original blocks, but only 3.06% and 1.45% faster than equally long plain extension (Tables 10 and 22). Report the fourth original baseline if available; otherwise distinguish the two claims in the abstract as clearly as in Section 4.3.

  2. Preserving scores does not preserve which tokens the target gets to check. DFlash-b16 makes this distinction clear. It accepts 0.446 tokens per round beyond its training block (Table 16), but its total rises by only 0.083, from 7.431 to 7.514 (Table 1). That implies about 0.36 fewer accepted tokens per round within the original depths. This is not a failure of the fixed-prefix proof: the tree width falls from 11 to 8, and later rounds can start from different text. It does matter for the selective-extension estimate. Table 3 applies the extra yield measured in extended rounds to rounds that filled the original block. These events are measured in different runs: the original block fills in 14.4% of rounds, while the extended tree reaches the first new position in 12.6% (Tables 3 and 17). The authors correctly label the estimated 5.8% speed gain as an oracle result. The evidence does not show how much of the gap from the measured 4.1% slowdown could be recovered just by skipping unused work.

References:

[1] Jian Chen, Yesheng Liang, and Zhijian Liu. DFlash: Block Diffusion for Flash Speculative Decoding

[2] Ephrem Wu. Ceiling-Clipped Acceptance Histograms Indicate Stranded Speed-up in Block-Diffusion Speculative Decoding

[3] Yijie Jin et al. Multi-Block Diffusion Language Models

Questions for authors
  1. Do you have a DFlash-b7 run at its original length on the same prompts and with 160 verified nodes? If so, what were its tokens per round and mean per-request tokens per second? If not, is the intended four-model comparison against plain extension?
  2. What is the paired 95% interval across prompts for the 0.083-token gain on DFlash-b16? Appendix A describes the resampling method, but Table 1 gives only the point estimate. The three timing repeats use the same generation seed.
  3. As a check on Table 3, what DFlash-b16 speed change does the Appendix H formula predict if you keep its assumed token yield but charge the extra cost to every round? The existing request records should allow comparison with the measured 4.1% slowdown, without building a new controller.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.