- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a training-free acceleration framework for block-wise diffusion decoding. Approximated suffix pruning keeps only a sliding window of width after the current block plus the terminal position of the whole sequence, preserving order through RoPE position IDs. Dynamic confidence aware parallel decoding lowers the acceptance threshold with the mask ratio of the query region, . An early exit skips all remaining blocks once EOS is predicted. Across five text dLLMs and one multimodal dLLM on six benchmarks, average speedups over vanilla are 9.3-19.8x, and on GSM8K at length 2048 the method reports 2.5 accuracy points over the strongest baseline with 4.7x higher throughput.
Critical strengths
Empirical coverage is broad: three main backbones at two generation lengths across four benchmarks, plus two block-diffusion models and one multimodal model, with both throughput and per-sample latency. The method is training-free and easy to drop into existing stacks. Retaining the terminal position has clear empirical support.
Critical weaknesses
Once prior work is subtracted, almost nothing new remains, least of all the speedup. Tables 5 and 7 decompose the 28.0x over vanilla into 3.28x from prefix caching, 1.84x from early exit, 3.74x from threshold parallel acceptance and 1.24x from suffix pruning; the first three multiply to 22.6x and are all prior work, leaving the schedule's 1.21x as this paper's own contribution, while the genuinely new part of ASP, the terminal anchor, retains one extra token and therefore costs rather than saves time. On the accuracy side, the largest jump is already present in the DPad row.
- Both claimed components overlap heavily with existing methods. DPad already provides a fixed-length suffix sliding window, a RoPE remapping that keeps retained tokens at their original absolute positions, and EOS early termination. Of DCAD's four ingredients, prefix KV reuse within a block and confidence-threshold acceptance with its fallback ( fixed at Fast-dLLM's default 0.9) are Fast-dLLM's and the query construction follows from ASP, leaving only the linear decrease of the threshold with mask ratio as new. Section 2.2 states that DPad "may lose positional information in the suffix region", but DPad remaps RoPE so each retained token keeps its original absolute position, so retained tokens lose no positional information. The one real difference is that DPad retains stochastically within the window (20-37.5% density) with no guarantee of keeping the final position of the whole sequence, which necessarily falls outside the window once the sequence is longer than it, whereas ASP keeps that position as a deterministic anchor. Early exit is introduced and ablated as one of three framework components without attribution to DPad, and EOS-based early stopping is common practice.
- The 3.7x over DPad cannot be explained by suffix cost. On LLaDA-1.5/GSM8K-512 the method deterministically retains 129 suffix tokens at , whereas DPad retains about 64 in expectation (window 256 at 25% density). The method keeps more yet reports 3.7x higher throughput (69.8 vs 19.0 TPS), so the speedup cannot come from a cheaper suffix. Decomposed through Table 5, the 3.67x factors into about 1.48x from fixed-threshold parallel acceptance, 1.21x from the schedule, and about 2.05x left unexplained on the DPad side, the last measured against Table 5's "prefix cache + suffix pruning + exit" configuration at 38.9 TPS versus DPad's 19.0 TPS. That 2.05x can only come from implementation overhead (non-contiguous gather, per-block resampling, per-token RoPE remapping) or from the DPad row running without prefix caching, and the paper measures and discloses neither.
- The accuracy gain is misattributed and unexplained. The appendix claims 13.8-27.0 points over Fast-dLLM on MBPP, attributed to suppressing distant high-entropy masks, but in the LLaDA-Instruct MBPP-512 column Fast-dLLM scores 13.8, DPad 39.4 and the method 40.8, so 25.6 of the 27.0 points are already in DPad and this paper adds 1.4. Vanilla's 14.8 looks like non-termination or formatting failure, yet no termination, truncation or format-error rate is reported. The appendix elsewhere states that with appropriate hyperparameter configurations the method can recover or surpass vanilla accuracy.
- No isolation study, and asymmetric tuning. The Fixed and Ours columns of Table 16 are numerically identical to the main table, so no experiment changes only the acceptance rule with everything else fixed; the sole isolated figure for the schedule is 1.21x in Figure 4(b). The paper's and are set per model, benchmark and length while DPad runs at defaults. Table 20's three values coincide with the main table's GSM8K-512 cells, so the terminal anchor is tested at one length only, even though the appendix itself notes that retaining it makes behavior depend on overall generation length.
Questions for authors
- On LLaDA-1.5/GSM8K-512 the method retains 129 suffix tokens at while DPad retains about 64 in expectation (window 256 at 25% density). The method keeps more yet is 3.7x faster, so where does that speedup come from? Please give step-level time breakdowns for both sides.
- For fairness, could you disclose your experimental configurations for Fast-dLLM and DPad?
- Tables 5 and 7 decompose the 28.0x over vanilla into 3.28x (prefix cache), 1.84x (early exit), 3.74x (threshold parallel acceptance) and 1.24x (suffix pruning), with the first three multiplying to 22.6x of prior work. Please confirm this decomposition and state what share of the total speedup comes from this paper's own additions. With prefix caching and EOS early termination given to all baselines, what do the main-table multiples become?
- Please decompose the 27.0 points on MBPP into early termination, suffix pruning and this paper's increment, report vanilla's termination, truncation and format-error rates in that cell, and repeat the trailing-position ablation separately at 512, 1024 and 2048.
7 comments
Response to Reviewer HS6a (Part 2/4)
Q3: Tables 5 and 7 decompose the 28.0x over vanilla into 3.28x (prefix cache), 1.84x (early exit), 3.74x (threshold parallel acceptance) and 1.24x (suffix pruning), with the first three multiplying to 22.6x of prior work. Please confirm this decomposition and state what share of the total speedup comes from this paper's own additions. With prefix caching and EOS early termination given to all baselines, what do the main-table multiples become?
- In Table 5 of our paper, prefix cache (PC) achieves 3.28× throughput of Vanilla, and PC + Exit achieves 1.84× throughput of PC alone. PC + DCAD + Exit achieves 3.74× throughput of PC + Exit, and PC + DCAD + Exit + ASP achieves 1.24× throughput of PC + DCAD + Exit. PC + DCAD + Exit achieves 22.6× the throughput of Vanilla. Attributing the first three gains to prior work overlooks our contribution in the third comparison. DCAD is our proposed dynamic-threshold decoding strategy, denoted “Dyn.” in Table 5 of the original manuscript. In Table 1 of our paper, Fast-dLLM uses PC and fixed-threshold parallel decoding (Par). On GSM8K (5-shot) with LLaDA-1.5, Streaming-dLLM improves accuracy by 0.8 percentage points over Fast-dLLM and achieves 2.7× its throughput. These gains reflect the combined effects of our suffix pruning, dynamic-threshold decoding, and Exit.
- Additional baseline with EOS early termination (Exit). In our response to Q1+Q2+W2, we added DPad (Par + PC + Exit). All baselines now use KV cache. We also adapted Fast-dLLM, a representative baseline, to use Exit. As shown in Table 6, Streaming-dLLM improves accuracy by 0.8 percentage points over Fast-dLLM (Exit) and achieves 2.2× its throughput. These gains reflect the contributions of our suffix pruning and dynamic-threshold decoding.
- In the revised manuscript, we have added Fast-dLLM (Exit) to Table 1 in the main text.
Table 6: Additional baseline comparisons on LLaDA-1.5 in the main results table,Each entry reports Accuracy / Throughput.
| HumanEval | GSM8K | MBPP | MATH | Average | |
|---|---|---|---|---|---|
| +Fast-dLLM (Exit) | 39.8/14.1 | 80.4/31.8 | 38.1/21.3 | 35.1/40.3 | 48.4/26.9 |
| +Streaming-dLLM | 40.2/26.7 | 81.2/69.8 | 38.4/61.4 | 35.1/62.4 | 48.7/55.1 |
Q4+W3: Please decompose the 27.0 points on MBPP into early termination, suffix pruning and this paper's increment, report vanilla's termination, truncation and format-error rates in that cell, and repeat the trailing-position ablation separately at 512, 1024 and 2048. The accuracy gain is misattributed and unexplained.
- Clarification on the role of suffix pruning in the MBPP accuracy recovery. DPad and Streaming-dLLM’s ASP use different suffix pruning strategies, and both recover accuracy on MBPP. A plausible explanation is that suffix pruning reduces interference from distant mask tokens. We further evaluated different combinations of components on MBPP. As shown in Table 7, Exit alone yields no accuracy gain, indicating that the improvement comes from suffix pruning. In each MBPP demonstration, Python source code immediately follows
[BEGIN], and[DONE]marks the end. The demonstrations thus provide the primary cue for the code-only output format. Our analysis of the generated outputs showed that when the full suffix was retained, the model generated additional material after the solution code, including explanations, tests, and repeated code, before emitting[DONE]. Under strict matching, this extra material causes compilation failures. With lenient matching, accuracy in the full-suffix setting rises from 13.8% to 36.8%. The ablations support our original attribution of the MBPP accuracy recovery to suffix pruning. The output analysis and the recovery under lenient matching are consistent with our hypothesis that distant mask tokens introduce interference and that pruning them helps the model follow the code-only format required by strict matching. - Additional experiments on the terminal anchor. We ablated the terminal anchor at different output lengths. As shown in Table 8, retaining the terminal anchor consistently improves performance across the tested lengths, providing further evidence of Streaming-dLLM’s effectiveness.
- In the revised manuscript, we added Table 25 to Appendix B.11 to demonstrate that the terminal anchor design generalizes across output lengths. We also expanded the analysis of the source of the MBPP accuracy gains in Appendix B.1.
Table 7: MBPP-512 ablation on LLaDA-Instruct.
| MBPP strict match result | MBPP flexible match result | |
|---|---|---|
| Baseline (Par+PC) | 13.8 | 36.8 |
| Baseline+Exit | 13.8 | 36.8 |
| Baseline+DPad | 39.4 | 39.4 |
| Baseline+ASP | 40.8 | 40.8 |
| Baseline+DCAD | 13.8 | 36.8 |
| Baseline+ASP+DCAD | 40.8 | 40.8 |
Table 8: Effect of retaining the trailing positional token at different generation length.
| Trailing Position | Accuracy | |
|---|---|---|
| GSM8K,len=512 | × | 79.6 |
| GSM8K,len=512 | √ | 81.2 |
| GSM8K,len=1024 | × | 78.7 |
| GSM8K,len=1024 | √ | 81.2 |
| GSM8K,len=2048 | × | 79.8 |
| GSM8K,len=2048 | √ | 82.0 |
Response to Reviewer HS6a (Part 3/4)
W1: Both claimed components overlap heavily with existing methods. DPad already provides a fixed-length suffix sliding window, a RoPE remapping that keeps retained tokens at their original absolute positions, and EOS early termination. Of DCAD's four ingredients, prefix KV reuse within a block and confidence-threshold acceptance with its fallback (fixed at Fast-dLLM's default 0.9) are Fast-dLLM's and the query construction follows from ASP, leaving only the linear decrease of the threshold with mask ratio as new. Section 2.2 states that DPad "may lose positional information in the suffix region", but DPad remaps RoPE so each retained token keeps its original absolute position, so retained tokens lose no positional information. The one real difference is that DPad retains stochastically within the window (20-37.5% density) with no guarantee of keeping the final position of the whole sequence, which necessarily falls outside the window once the sequence is longer than it, whereas ASP keeps that position as a deterministic anchor. Early exit is introduced and ablated as one of three framework components without attribution to DPad, and EOS-based early stopping is common practice.
-
We would like to clarify how DPad and Streaming-dLLM differ in the types of decoding redundancy they address and their applicability to pure block diffusion language models.
-
Streaming-dLLM reduces temporal and spatial redundancy in dLLMs through dynamic threshold decoding and approximated suffix pruning, respectively. DPad targets spatial redundancy alone. Temporally, dynamic threshold decoding uses the mask ratio as a proxy for uncertainty at the current decoding stage, allowing the threshold to adapt to the overall decoding state. This signal complements individual token confidence scores and enables higher throughput. Spatially, we observe that attention concentrates on nearby suffix positions and the end of the sequence, while intermediate suffix positions receive less attention and are still processed at every step. Based on this observation, ASP retains a contiguous window immediately after the current block and the terminal position of the sequence. DPad samples within a candidate window using a distance-decay rule, retaining some redundant intermediate positions while omitting the terminal position. Moreover, we measured how many suffix tokens each strategy actually retains on LLaDA-1.5 (Table 1). Streaming-dLLM retains 31.88 fewer suffix tokens on average in the experiments with a 512 token generation budget and 185.42 fewer on GSM8K with a 2,048 token budget.
-
DPad is limited to full-sequence dLLMs. Streaming-dLLM extends to the latest pure block-diffusion language models and achieves 2.6–5.4× speedups. Pure block diffusion is an increasingly important direction for dLLMs. These models support early exit and variable-length generation, and their masked generation region is small, typically just 32 tokens. DPad prunes the masked suffix beyond the current block in full-sequence diffusion language models. Pure block diffusion confines masking to the current block, leaving no cross-block suffix for DPad to prune. Streaming-dLLM can improve the performance of pure block diffusion models along both the spatial and temporal dimensions even with short suffixes. Spatially, we divide the current diffusion block into smaller sub-blocks, separate the sub-block being denoised from the subsequent masked sub-blocks, and apply approximated suffix pruning to those subsequent sub-blocks to reduce computation. Temporally, Streaming-dLLM adjusts the degree of decoding parallelism according to the proportion of masked tokens remaining in the block. Reducing the masked generation region further increases the benefit of the dynamic threshold. Table 3 of the original submission evaluates this extension on two pure block dLLMs. At comparable accuracy, Streaming-dLLM achieves 2.6–5.4× the throughput of the +Fast-dLLM baseline, which already uses prefix cache, fixed-threshold parallel decoding, and early stopping.
-
-
Streaming-dLLM’s contributions. As stated in the main text, we identify spatial and temporal redundancy inherent in dLLMs and design approximated suffix pruning (ASP) and dynamic confidence-aware parallel decoding (DCAD) to eliminate redundant computation along both dimensions. Early exit serves only as an auxiliary component. In this rebuttal, we also report results for additional baselines with EOS-based early exit enabled to provide a more complete comparison.
Response to Reviewer HS6a (Part 1/4)
Dear Reviewer HS6a,
We sincerely thank the reviewer for recognizing the breadth of our evaluation, the ease of integrating our training-free method into existing pipelines. Your feedback helped us make the manuscript clearer and more complete, and we have marked the corresponding changes in orange. We appreciate the constructive feedback and respond below.
Table 1: Suffix retention statistics for the DPad and Streaming-dLLM pruning strategies on LLaDA-1.5. The parameters k, a, and window size are DPad hyperparameters.
Table 2: DPad Tuning Comparison.
Table 3: Additional baseline comparisons on LLaDA-1.5, Each entry reports Accuracy/Throughput.
Table 4: Additional baseline comparisons on LLaDA-1.5 for long-sequence tasks.
Table 5: Component ablations on LLaDA-1.5. All rows enable Par+PC+Exit.