- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an early-bird (EB) decoding framework, motivated by the observation that tokens with similarly low entropy tend to cluster and can be jointly decoded earlier, before reaching the confidence threshold. The EB-Decode framework integrates two key enablers called LBS and LPS. Both components are developed without modifying pretrained dLLM weights and can therefore be directly deployed as plug-ins during serving. Extensive experiments across models and benchmarks consistently validate the effectiveness of EB-Decode.
Critical strengths
-
The paper's observation and motivation are well-reasoned, and the proposed method aligns perfectly with the motivation.
-
Experimental results demonstrate that the proposed method exhibits superior performance on multiple DLLMs and benchmarks.
-
The paper is written in a very clear and easy-to-understand style.
-
The paper provides comprehensive ablation experiments to demonstrate the effectiveness of the proposed method.
Critical weaknesses
-
The experimental section of the paper does not include the Accuracy-Speed Pareto Curve. Comparing the Accuracy-Speed Pareto Curve with the baseline is a crucial experiment to demonstrate that a better trade-off has indeed been achieved.
-
Can the proposed method also be applied to uniform diffusion language models, such as DMax-16B [1]?
-
EB-Decode seems to have limited performance on Block-Diffusion LLMs.
[1] DMax: Aggressive Parallel Decoding for dLLMs
Questions for authors
Please see the weaknesses. My main concerns are with the experiment section.
4 comments
Response to the Reviewer P4VJ (Part 2 of 2)
Experimental setup. To evaluate the adapted EB-Decode, we follow the official decoding setting of DMax (block size 32 and generation length 2048) and evaluate it on GSM8K. Consistent with Table 6 of the revised paper, we report TPF (tokens per forward pass) and TPS. To compare the methods at comparable TPS, we set the threshold of DMax to τ = 0.4 and the LPS threshold of EB-Decode to = 0.90.
Table R.1: Results of DMax-16B on GSM8K.
| Method | Acc. (%) | TPF | TPS |
|---|---|---|---|
| DMax | 89.23 | 5.14 | 135.4 |
| LBS | 89.92 | 5.06 | 127.6 |
| EB-Decode | 89.92 | 5.54 | 136.0 |
Results and conclusion. As shown in Table R.1, at comparable TPS, EB-Decode achieves 1.08× the TPF of DMax and 0.7 points higher accuracy. This shows that EB-Decode reduces the number of forward passes more effectively than simply lowering the threshold, while achieving higher accuracy. Therefore, even on DMax, whose upper bound for acceleration is already low, EB-Decode further improves the accuracy–speed trade-off with only small adjustments.
[W3] Performance on Block-Diffusion dLLMs
We thank the reviewer for this comment. SDAR-1.7B-Chat-b32 and LLaDA2.0-mini are block-diffusion dLLMs, which adopt block-causal attention and compute logits only for the current window at each step. Table 4 of the original paper (now Table 6 in the revised paper) evaluates only LBS, to show that LBS applies to such models with only small adjustments (using only the logits within the window and tightening from 5 to 1). LBS improves the decoding order and raises accuracy in three of the four settings in Table R.2, by up to 1.6 points. However, LBS itself is not designed to accelerate decoding directly, and acceleration requires LPS.
We therefore add LPS on top of LBS and evaluate the full EB-Decode. On each model, we regenerate the training data of LPS from the decoding trajectories of LBS and retrain LPS with the same recipe as on LLaDA-8B-Instruct. The results are shown in Table R.2 and added to Table 6 in Section 5.3 of the revised paper. These experiments are run on a new server, so the TPS and accuracy values differ slightly from those in Table 4 of the original paper.
Table R.2: Results of LBS and EB-Decode on block-diffusion dLLMs. LLaDA2.0-mini and SDAR-1.7B-Chat-b32 use the thresholds τ = 0.95 and τ = 0.9 from their official settings, respectively. TPF is the number of tokens decoded per forward pass.
| Model | Method | GSM8K Acc. (%) | GSM8K TPF | GSM8K TPS | HumanEval Acc. (%) | HumanEval TPF | HumanEval TPS |
|---|---|---|---|---|---|---|---|
| LLaDA2.0-mini | Default | 90.90 | 1.90 | 9.12 | 80.28 | 3.43 | 14.90 |
| LLaDA2.0-mini | LBS | 90.42 | 1.79 | 8.70 | 81.91 | 3.17 | 13.95 |
| LLaDA2.0-mini | EB-Decode | 90.22 | 2.13 | 13.39 | 80.49 | 3.74 | 21.56 |
| SDAR-1.7B-Chat-b32 | Default | 74.98 | 2.18 | 38.0 | 45.73 | 1.86 | 28.6 |
| SDAR-1.7B-Chat-b32 | LBS | 76.04 | 2.10 | 30.4 | 46.95 | 1.79 | 26.8 |
| SDAR-1.7B-Chat-b32 | EB-Decode | 74.68 | 2.48 | 40.8 | 46.34 | 2.00 | 33.8 |
As shown in Table R.2, EB-Decode improves TPF and TPS on both models while maintaining accuracy. On the larger LLaDA2.0-mini, EB-Decode increases TPF by 9%–12% and reaches 1.45×–1.47× the TPS of default decoding. On SDAR-1.7B-Chat-b32, the TPS gain is smaller, at 1.07×–1.18×. This is because this model is small and each forward pass takes little time, so the overhead of the routers accounts for a larger share of the total time. Nevertheless, EB-Decode still increases TPF by 8%–14% on this model, decoding more tokens per forward pass. Adding LPS further increases TPF by 12%–19% and TPS by 26%–55% over LBS alone. This shows that EB-Decode maintains accuracy and accelerates decoding on block-diffusion dLLMs without modifying the model weights.
Thanks for the rebuttal and most of my concerns are resolved. I would raise my score to 4.
Thank You for Your Reply
Dear Reviewer P4VJ,
Thank you for your prompt reply and for raising your score. We are very grateful for your feedback and sincerely appreciate your recognition of our work. We truly appreciate the time and effort you have devoted to reviewing our manuscript.
Best regards,
Authors of Submission #9370
Response to the Reviewer P4VJ (Part 1 of 2)
We sincerely thank the reviewer for the constructive review, and for recognizing the consistency between our motivation and method, the results across multiple dLLMs and benchmarks, the clear writing, and the comprehensive ablations. Regarding the reviewer’s concerns, we would like to provide further experimental results and clarifications as follows.
[W1] Accuracy–Speed Pareto Curves
We thank the reviewer for this suggestion. Since figures cannot be included in the comments under the ICLR policy, we add the accuracy–speed curves to Figure 5 in Section 5.2 of the revised paper. Figure 5 shows the curves of LLaDA-8B-Instruct and LLaDA-1.5 on GSM8K, where each curve is obtained by sweeping the decoding threshold of each method. Besides the baselines in Table 1, we also include Swordsman and a tuned entropy-threshold block selector. Since the official implementations of Learn2PD and Swordsman do not provide configurations for LLaDA-1.5, these two methods are compared only on LLaDA-8B-Instruct. The absolute TPS may differ from Table 1 due to server differences.
As shown in Figure 5, EB-Decode achieves the best trade-off on both models. With similar accuracy (within 1 point of vanilla), EB-Decode reaches about 180 TPS on both models, which is about 7× that of vanilla and higher than all baselines. On LLaDA-1.5, the operating points of EB-Decode form the Pareto frontier. This shows that EB-Decode consistently improves the accuracy–speed trade-off across decoding thresholds, rather than only at a single operating point.
[W2] Applicability to Revisable dLLMs such as DMax
We thank the reviewer for this question. Although DMax-16B decodes differently from standard masked dLLMs, we still verify the effectiveness of EB-Decode on it with only small adjustments, and the results are shown in Table R.1 and added to Table 17 in Appendix M of the revised paper.
The special setting of DMax. DMax-16B is built on LLaDA2.0-mini and can revise its own decoded tokens, which makes it fundamentally different from standard masked dLLMs. In DMax, decoded tokens are only tentative. DMax decodes from left to right in blocks of 32 tokens, and at each step it promotes the longest contiguous prefix with a very low threshold. The promoted tokens can still be rewritten in later steps, and they are finalized only when all positions in the block reach a confidence of 0.9.
Why the original design does not directly apply. Due to this special setting, the original design of EB-Decode cannot be directly applied to DMax. EB-Decode targets standard masked dLLMs and accelerates decoding by finalizing tokens earlier, while DMax changes this premise in two ways. (1) In DMax, decoding a token does not finalize it. Since a decoded token may be rewritten at any later step, it is difficult to determine when the token at a position is already correct, and thus difficult to commit it early. The oracle-replay training data of LPS also become invalid, because the tokens forced by the oracle are rewritten in the next step and the replay deviates from the original trajectory. (2) Since tokens can be revised later, DMax promotes tokens aggressively with a very low threshold and corrects them in later steps. This already makes DMax very fast and leaves a low upper bound for further acceleration.
Adapting EB-Decode with small adjustments. Nevertheless, EB-Decode can be adapted to DMax with only two adjustments. (1) LBS is restricted within each block. Since DMax decodes block by block, we follow the block-causal adaptation in W3, and the router selects a variable-length block starting from the leftmost masked position within each DMax block and promotes it in one step. (2) LPS decides whether a block can be finalized. Since DMax finalizes tokens only at the block exit, LPS scores all positions in the block and we apply its decision to the block exit. Specifically, we relax the exit condition from requiring all positions to reach a confidence of 0.9 to requiring each position to either reach a confidence of 0.9 or be predicted as final by LPS. We also collect the LPS training data by directly recording the LBS decoding process on DMax, which avoids the trajectory deviation of oracle replay. All other components remain the same as in the paper, and the weights of DMax are kept unchanged.
[to be continued]