acceptodds
Under review as a conference paper at ICLR 2027

Early-Bird Decoding: Accelerating Diffusion LLMs with Learnable Block Sizes and Parallel Sampling

Abstract

Diffusion large language models (dLLMs) offer a promising parallel decoding paradigm as an alternative to autoregressive generation through iterative unmasking. However, dLLMs typically require many steps before token confidence reaches the decoding threshold, resulting in inefficient inference even with block-wise KV caching. To accelerate dLLM inference, we for the first time propose an "early-bird (EB)" decoding framework, motivated by the observation that tokens with similarly low entropy tend to cluster and can be jointly decoded earlier, before reaching the confidence threshold. In particular, our EB-Decode framework integrates two key enablers: (1) a learnable network that adaptively groups tokens with similar uncertainty into variable-length blocks, rather than relying on fixed block sizes; (2) a position-aware sampler that learns to unmask tokens in parallel using fewer decoding steps within predicted variable-length blocks. Both components are developed without modifying pretrained dLLM weights and can therefore be directly deployed as plug-ins during serving, with negligible training and inference overhead. Extensive experiments across three models and four benchmarks consistently validate our observation and the effectiveness of EB-Decode, achieving 3.53–18.76× higher throughput than vanilla decoding with comparable accuracy and up to 1.58× higher throughput than the baseline with the highest average speedup, Fast-dLLM. We will release the code upon acceptance.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Language Model, Efficient Inference

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_P4VJOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes an early-bird (EB) decoding framework, motivated by the observation that tokens with similarly low entropy tend to cluster and can be jointly decoded earlier, before reaching the confidence threshold. The EB-Decode framework integrates two key enablers called LBS and LPS. Both components are developed without modifying pretrained dLLM weights and can therefore be directly deployed as plug-ins during serving. Extensive experiments across models and benchmarks consistently validate the effectiveness of EB-Decode.

Critical strengths
  1. The paper's observation and motivation are well-reasoned, and the proposed method aligns perfectly with the motivation.

  2. Experimental results demonstrate that the proposed method exhibits superior performance on multiple DLLMs and benchmarks.

  3. The paper is written in a very clear and easy-to-understand style.

  4. The paper provides comprehensive ablation experiments to demonstrate the effectiveness of the proposed method.

Critical weaknesses
  1. The experimental section of the paper does not include the Accuracy-Speed ​​Pareto Curve. Comparing the Accuracy-Speed ​​Pareto Curve with the baseline is a crucial experiment to demonstrate that a better trade-off has indeed been achieved.

  2. Can the proposed method also be applied to uniform diffusion language models, such as DMax-16B [1]?

  3. EB-Decode seems to have limited performance on Block-Diffusion LLMs.

[1] DMax: Aggressive Parallel Decoding for dLLMs

Questions for authors

Please see the weaknesses. My main concerns are with the experiment section.

4 comments
AuthorsOpenReview ↗

Response to the Reviewer P4VJ (Part 1 of 2)

We sincerely thank the reviewer for the constructive review, and for recognizing the consistency between our motivation and method, the results across multiple dLLMs and benchmarks, the clear writing, and the comprehensive ablations. Regarding the reviewer’s concerns, we would like to provide further experimental results and clarifications as follows.

[W1] Accuracy–Speed Pareto Curves

We thank the reviewer for this suggestion. Since figures cannot be included in the comments under the ICLR policy, we add the accuracy–speed curves to Figure 5 in Section 5.2 of the revised paper. Figure 5 shows the curves of LLaDA-8B-Instruct and LLaDA-1.5 on GSM8K, where each curve is obtained by sweeping the decoding threshold of each method. Besides the baselines in Table 1, we also include Swordsman and a tuned entropy-threshold block selector. Since the official implementations of Learn2PD and Swordsman do not provide configurations for LLaDA-1.5, these two methods are compared only on LLaDA-8B-Instruct. The absolute TPS may differ from Table 1 due to server differences.

As shown in Figure 5, EB-Decode achieves the best trade-off on both models. With similar accuracy (within 1 point of vanilla), EB-Decode reaches about 180 TPS on both models, which is about 7× that of vanilla and higher than all baselines. On LLaDA-1.5, the operating points of EB-Decode form the Pareto frontier. This shows that EB-Decode consistently improves the accuracy–speed trade-off across decoding thresholds, rather than only at a single operating point.

[W2] Applicability to Revisable dLLMs such as DMax

We thank the reviewer for this question. Although DMax-16B decodes differently from standard masked dLLMs, we still verify the effectiveness of EB-Decode on it with only small adjustments, and the results are shown in Table R.1 and added to Table 17 in Appendix M of the revised paper.

The special setting of DMax. DMax-16B is built on LLaDA2.0-mini and can revise its own decoded tokens, which makes it fundamentally different from standard masked dLLMs. In DMax, decoded tokens are only tentative. DMax decodes from left to right in blocks of 32 tokens, and at each step it promotes the longest contiguous prefix with a very low threshold. The promoted tokens can still be rewritten in later steps, and they are finalized only when all positions in the block reach a confidence of 0.9.

Why the original design does not directly apply. Due to this special setting, the original design of EB-Decode cannot be directly applied to DMax. EB-Decode targets standard masked dLLMs and accelerates decoding by finalizing tokens earlier, while DMax changes this premise in two ways. (1) In DMax, decoding a token does not finalize it. Since a decoded token may be rewritten at any later step, it is difficult to determine when the token at a position is already correct, and thus difficult to commit it early. The oracle-replay training data of LPS also become invalid, because the tokens forced by the oracle are rewritten in the next step and the replay deviates from the original trajectory. (2) Since tokens can be revised later, DMax promotes tokens aggressively with a very low threshold and corrects them in later steps. This already makes DMax very fast and leaves a low upper bound for further acceleration.

Adapting EB-Decode with small adjustments. Nevertheless, EB-Decode can be adapted to DMax with only two adjustments. (1) LBS is restricted within each block. Since DMax decodes block by block, we follow the block-causal adaptation in W3, and the router selects a variable-length block starting from the leftmost masked position within each DMax block and promotes it in one step. (2) LPS decides whether a block can be finalized. Since DMax finalizes tokens only at the block exit, LPS scores all positions in the block and we apply its decision to the block exit. Specifically, we relax the exit condition from requiring all positions to reach a confidence of 0.9 to requiring each position to either reach a confidence of 0.9 or be predicted as final by LPS. We also collect the LPS training data by directly recording the LBS decoding process on DMax, which avoids the trajectory deviation of oracle replay. All other components remain the same as in the paper, and the weights of DMax are kept unchanged.

[to be continued]

AuthorsOpenReview ↗

Response to the Reviewer P4VJ (Part 2 of 2)

Experimental setup. To evaluate the adapted EB-Decode, we follow the official decoding setting of DMax (block size 32 and generation length 2048) and evaluate it on GSM8K. Consistent with Table 6 of the revised paper, we report TPF (tokens per forward pass) and TPS. To compare the methods at comparable TPS, we set the threshold of DMax to τ = 0.4 and the LPS threshold of EB-Decode to τψ\tau_\psi = 0.90.

Table R.1: Results of DMax-16B on GSM8K.

MethodAcc. (%)TPFTPS
DMax89.235.14135.4
LBS89.925.06127.6
EB-Decode89.925.54136.0

Results and conclusion. As shown in Table R.1, at comparable TPS, EB-Decode achieves 1.08× the TPF of DMax and 0.7 points higher accuracy. This shows that EB-Decode reduces the number of forward passes more effectively than simply lowering the threshold, while achieving higher accuracy. Therefore, even on DMax, whose upper bound for acceleration is already low, EB-Decode further improves the accuracy–speed trade-off with only small adjustments.

[W3] Performance on Block-Diffusion dLLMs

We thank the reviewer for this comment. SDAR-1.7B-Chat-b32 and LLaDA2.0-mini are block-diffusion dLLMs, which adopt block-causal attention and compute logits only for the current window at each step. Table 4 of the original paper (now Table 6 in the revised paper) evaluates only LBS, to show that LBS applies to such models with only small adjustments (using only the logits within the window and tightening gmax⁡g_{\max} from 5 to 1). LBS improves the decoding order and raises accuracy in three of the four settings in Table R.2, by up to 1.6 points. However, LBS itself is not designed to accelerate decoding directly, and acceleration requires LPS.

We therefore add LPS on top of LBS and evaluate the full EB-Decode. On each model, we regenerate the training data of LPS from the decoding trajectories of LBS and retrain LPS with the same recipe as on LLaDA-8B-Instruct. The results are shown in Table R.2 and added to Table 6 in Section 5.3 of the revised paper. These experiments are run on a new server, so the TPS and accuracy values differ slightly from those in Table 4 of the original paper.

Table R.2: Results of LBS and EB-Decode on block-diffusion dLLMs. LLaDA2.0-mini and SDAR-1.7B-Chat-b32 use the thresholds τ = 0.95 and τ = 0.9 from their official settings, respectively. TPF is the number of tokens decoded per forward pass.

ModelMethodGSM8K Acc. (%)GSM8K TPFGSM8K TPSHumanEval Acc. (%)HumanEval TPFHumanEval TPS
LLaDA2.0-miniDefault90.901.909.1280.283.4314.90
LLaDA2.0-miniLBS90.421.798.7081.913.1713.95
LLaDA2.0-miniEB-Decode90.222.1313.3980.493.7421.56
SDAR-1.7B-Chat-b32Default74.982.1838.045.731.8628.6
SDAR-1.7B-Chat-b32LBS76.042.1030.446.951.7926.8
SDAR-1.7B-Chat-b32EB-Decode74.682.4840.846.342.0033.8

As shown in Table R.2, EB-Decode improves TPF and TPS on both models while maintaining accuracy. On the larger LLaDA2.0-mini, EB-Decode increases TPF by 9%–12% and reaches 1.45×–1.47× the TPS of default decoding. On SDAR-1.7B-Chat-b32, the TPS gain is smaller, at 1.07×–1.18×. This is because this model is small and each forward pass takes little time, so the overhead of the routers accounts for a larger share of the total time. Nevertheless, EB-Decode still increases TPF by 8%–14% on this model, decoding more tokens per forward pass. Adding LPS further increases TPF by 12%–19% and TPS by 26%–55% over LBS alone. This shows that EB-Decode maintains accuracy and accelerates decoding on block-diffusion dLLMs without modifying the model weights.

Thanks for the rebuttal and most of my concerns are resolved. I would raise my score to 4.

AuthorsOpenReview ↗

Thank You for Your Reply

Dear Reviewer P4VJ,

Thank you for your prompt reply and for raising your score. We are very grateful for your feedback and sincerely appreciate your recognition of our work. We truly appreciate the time and effort you have devoted to reviewing our manuscript.

Best regards,

Authors of Submission #9370

Reviewer_jC9NOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

EB-Decode accelerates diffusion language model inference using two lightweight learned modules while keeping the backbone frozen. Learnable Block Size (LBS) selects variable-sized, potentially non-contiguous groups of masked positions using predictive entropy and predicted-token embeddings, followed by several filtering rules. Learnable Parallel Sampling (LPS) predicts whether a token can be committed before reaching the conventional confidence threshold, using uncertainty features and positional information. The method is evaluated on three bidirectional diffusion models and four mathematical reasoning and code-generation benchmarks. It reports 3.53–18.76× throughput improvements over vanilla decoding, with generally small accuracy changes. Additional experiments investigate component ablations, compatibility with KV caching, router transfer, and application to semi-autoregressive models.

Critical strengths

S1. A useful joint treatment of block selection and token commitment. The method addresses two distinct decoding decisions: which positions to process together and when to finalize their predictions. The learned modules are small, leave backbone weights unchanged, and can exploit information already produced by the backbone. Supporting non-contiguous groups provides an interesting extension beyond conventional contiguous block partitioning. S2. Broad and consistently favorable reported throughput results. The evaluation covers three backbones and four benchmarks, with the combined method improving throughput across all main settings. The paper also evaluates caching on both the proposed method and Fast-dLLM, so the evidence is not limited to comparisons against an uncached baseline. Component ablations and transfer from mathematical training data to code benchmarks further support practical usefulness.

Critical weaknesses

W1. The incremental contribution over closely related adaptive methods is not fully established. Learn2PD already studies premature remasking of correct predictions and learns when predictions match the final output. Swordsman already combines entropy-based adaptive partitioning with dynamic unmasking thresholds, yet it is absent from the experimental comparison and is described as using a confidence threshold without acknowledging its dynamic adjustment in Table 12. The LBS comparison against random selection with filters is also weaker than a carefully tuned entropy-based selector. These omissions limit the evidence that learning both modules provides a superior speed–quality trade-off over strong adaptive alternatives. W2. The LPS training distribution and its relationship to deployment are insufficiently specified. Section 4.2 describes replay that uses the completed sequence to replace mismatched predictions, while Appendix D.2 describes traces under the deployment configuration and Algorithm 4 refers to LBS-generated trajectories. Oracle correction changes the intermediate states and does not, by itself, eliminate distribution shift when the deployed router makes irreversible errors. The validation false-positive budget therefore needs deployment-time calibration evidence. Relatedly, the position encoding (i−b)/∣B∣(i-b)/|B| need not lie in [0,1][0,1] for non-contiguous blocks, contrary to the stated range.

Questions for authors

Q1. Can you compare against Swordsman and a tuned entropy-based block selector under the same backend, cache configuration, and validation budget? Speed–quality curves would be more informative than a single operating point. Q2. What is the exact LPS trace-generation procedure? Please reconcile default-schedule generation, LBS rollouts, and oracle-corrected replay. How does the false-positive rate change when evaluated on trajectories generated by the deployed LBS+LPS policy? Q3. Equation (2) can be written as an average of pϕi(ℓCEi−1)p_\phi^i(\ell_{\mathrm{CE}}^i-1). How does LBS compare with simpler confidence/entropy selectors using the same filters and comparable validation tuning? Q4. For non-contiguous blocks, does the positional feature use original position offsets, rank within the selected block, or normalization by the block’s positional span? Please correct the formula or its claimed range. Q5. Please clarify EOS stopping and the throughput numerator, including whether EOS/padding-tail positions count as generated tokens. Can you report uncertainty for accuracy differences and repeated latency measurements at representative settings?

4 comments
AuthorsOpenReview ↗

Response to the Reviewer jC9N (Part 1 of 4)

We sincerely thank the reviewer for the careful and constructive review, and for recognizing that EB-Decode jointly handles block selection and token commitment, supports non-contiguous blocks, and achieves consistent speedups across settings. Regarding the reviewer’s concerns, we would like to provide further experimental results and clarifications as follows.

[W1 & Q1 & Q3] Clarifying Contribution over Related Adaptive Methods

We thank the reviewer for raising this question. We first clarify how EB-Decode differs from related adaptive methods such as Learn2PD and Swordsman. We then compare EB-Decode experimentally with stronger baselines, including Swordsman and tuned heuristic block selectors.

1. Conceptual Positioning

(1) Relation to Learn2PD. Both Learn2PD and the LPS router of EB-Decode learn when to commit a token, but they differ in the form of the block, the router architecture, and the input features. First, Learn2PD decides commitment within fixed-length contiguous blocks, while LBS in EB-Decode selects variable-length blocks whose positions can be non-contiguous. Second, to handle such blocks, LPS adopts a Transformer instead of an MLP. This allows LPS to process blocks of arbitrary length and to correlate decisions across positions through the intra-block position encoding, rather than scoring each position independently. Third, besides the top-1 confidence, LPS takes the normalized entropy, the top-1/top-2 confidence gap, and the block-wise mask ratio as inputs, which help it distinguish correct-but-underconfident tokens from genuinely uncertain ones. As shown in the ablation of Table 3, under the same block partition, starting from a Learn2PD-style MLP router and successively switching to a Transformer, adding the intra-block position encoding, and adding the three features increase TPS from 87.40 to 104.55, while accuracy stays at around 78%.

(2) Relation to Swordsman. First, we thank the reviewer for pointing out the oversight in Table 12. This table is now Table 13 in Appendix H of the revised paper, where we have corrected the commit rule of Swordsman to an entropy-adjusted threshold, which is dynamically adjusted by the average entropy of each block. We have also revised the corresponding descriptions in Sections 1 and 2.

Second, we want to further clarify the relationship between the two methods. Both Swordsman and EB-Decode adapt block selection and token commitment, but Swordsman makes these decisions per block with predefined entropy rules, while EB-Decode learns them from data and makes them per token. For block selection, Swordsman places the right boundary of a contiguous block at the position with the largest entropy change between adjacent tokens, while LBS decides for each masked position whether to include it based on its entropy and top-1 token, so the block can be non-contiguous and skip hard-to-decode tokens. For token commitment, all tokens in a Swordsman block share a threshold adjusted by the average entropy of the block, while LPS decides for each token whether it has converged and can commit converged tokens before their confidence reaches the threshold (Figure 3(b)).

2. Experimental Analysis

Direct comparisons provide stronger evidence for the advantage of EB-Decode over Swordsman and tuned heuristic block selectors. We therefore conduct the following two experiments.

(1) Comparison with Swordsman (Q1). We reproduce Swordsman with the same backend and cache configuration as EB-Decode (LLaDA-8B-Instruct, GSM8K, 4×H100, no KV cache). Table R.1 compares the two methods at similar accuracy, with TPS measured per GPU. We also add all operating points of Swordsman to the accuracy–speed curves in Figure 5(a) in Section 5.2 of the revised paper.

Table R.1: Comparison between Swordsman and EB-Decode. Speedup is relative to vanilla decoding on the same server (25.62 TPS).

MethodAcc. (%)NFE per questionTPSSpeedup over vanilla
Swordsman78.4794.0135.925.31×
EB-Decode (ours)78.6269.0175.856.86×

As shown in Figure 5(a), each operating point of Swordsman is outperformed by an operating point of EB-Decode in both accuracy and speed. This gap comes from the different ways the two methods select blocks and commit tokens. EB-Decode can form non-contiguous blocks and commit converged tokens early, and thus requires far fewer forward passes at similar accuracy.

[to be continued]

AuthorsOpenReview ↗

Response to the Reviewer jC9N (Part 2 of 4)

(2) Comparison with Tuned Heuristic Block Selectors (Q1 and Q3). The reviewer’s rewriting of Eq. (2) is correct. To address the concern that random selection is a weak baseline, we further compare LBS with tuned heuristic block selectors. To isolate the effect of block selection, we disable LPS, use the confidence-based commit rule of Fast-dLLM for all selectors, and run each selector once at τ ∈ {0.85, 0.9, 0.92}.

Specifically, Vanilla follows the setting of Table 1. The confidence selector takes masked positions with top-1 confidence of at least cc as candidates, and the entropy selector takes positions with normalized entropy of at most θ. We tune cc and θ on 300 math word problems randomly sampled from the LBS training data, and choose the value with the highest TPS among those whose accuracy is at most 1 point lower than LBS, which gives c∗=0.9c^* = 0.9 and θ∗=0.3\theta^* = 0.3. LBS uses its original setting in EB-Decode. All methods are evaluated on GSM8K under the same setting as Table 1, with slight differences due to a different server. The results are reported in Table R.2. We also add these operating points as accuracy–speed curves to Figure 6 in Section 5.3 of the revised paper, and provide the tuning details in Appendix J.

Table R.2: Comparison between LBS and tuned heuristic block selectors. Each cell reports Acc. (%) / TPS, with TPS measured per GPU.

Selector (LPS off)Ruleτ = 0.85τ = 0.9τ = 0.92
VanillaFixed blocks79.08 / 25.62 (one token per step, no τ sweep)
Confidence selectorκ ≥ cc, cc = 0.978.09 / 160.2578.39 / 141.4778.39 / 127.96
Entropy selectorH̃ ≤ θ, θ = 0.377.56 / 165.5178.85 / 143.9479.08 / 135.25
LBS (ours)Learned score78.70 / 154.6179.68 / 134.4979.76 / 127.28

As shown in Figure 6, the curve of LBS lies above those of both heuristic selectors. In their shared TPS range (about 128 to 155 TPS), LBS consistently achieves higher accuracy at the same speed, outperforming the entropy and confidence selectors by about 0.4 and 1.0 points on average, respectively. This shows that the gain comes from LBS estimating, based on context, whether each position will be consistent with the final output.

[W2 & Q2] LPS Trace Generation and Its Relation to Deployment

We thank the reviewer for carefully cross-checking Section 4.2, Appendix D.2, and Algorithm 4. We would like to clarify that the three places describe the same two-stage procedure, and that some abbreviated and ambiguous descriptions in the original paper caused the misunderstanding. Below, we first explain this procedure in detail, then clarify how the three descriptions correspond to it along with our revisions, and finally report the FP rate on deployment trajectories.

1. Explanation of the LPS Trace Generation Procedure

(1) Trace generation procedure. The training data of LPS are generated in two stages that share the same LBS block schedule. In the first stage, LBS selects blocks and tokens are committed with the confidence threshold τ = 0.9, which yields the output x0x_0 together with the positions of each block. Both the default-schedule generation and the LBS rollouts mentioned by the reviewer refer to this stage.

The second stage, oracle replay, generates the training samples of LPS. Starting from the fully masked sequence, the replay decodes block by block with the same blocks and order as the first stage, while the decoding state at each step is advanced by the replay itself. At each step, for each masked position in the current block, we record the features ff, the intra-block position pos, and the label y=1[x^=x0]y = 1[\hat{x} = x_0], where x^\hat{x} is the current top-1 prediction at that position. Positions with x^=x0\hat{x} = x_0 are positive samples (y=1y = 1), and the others are negative samples (y=0y = 0). We then commit only the positive positions, while the negative positions remain masked for the next step. If no position is positive at a step, we set the most confident masked position in the current block to its token in x0x_0 to avoid stalling.

We design the replay in this way for two reasons. First, the label yy is defined with respect to x0x_0, so x^=x0\hat{x} = x_0 indicates convergence only when the current context agrees with x0x_0. Committing only tokens that agree with x0x_0 keeps every state consistent with x0x_0, and thus keeps the labels valid. Second, each token is committed as soon as its prediction first matches x0x_0, i.e., at its first light green cell in Figure 3(a). This corresponds to the early convergence revealed by Observation 2, from which LPS learns to identify converged tokens. Since LPS also commits converged tokens before their confidence reaches the threshold at deployment, the replay yields training states close to the deployment states and generates a large number of training samples efficiently.

[to be continued]

AuthorsOpenReview ↗

Response to the Reviewer jC9N (Part 3 of 4)

(2) Clarification and revision of the three descriptions. We thank the reviewer for pointing out this inconsistency. All three descriptions refer to the two-stage procedure above, and we revise them as follows.

Lines 315–324 of Section 4.2 now give the complete two-stage procedure. We remove the inaccurate statement “replaces mismatched positions with their corresponding tokens from x0x_0” and state that a token from x0x_0 is written only when no position matches.

Lines 968–980 of Appendix D.2 also add the two-stage procedure, replace “base block size” with the fallback block size LBL_B, and remove the inaccurate statement “To remove any train/test distribution skew”.

Line 7 of Algorithm 1 and Line 2 of Algorithm 4 now first decode with LBS to obtain x0x_0 and then perform oracle replay in the same block order.

2. FP Rate on Deployment Trajectories

We thank the reviewer for raising this point. The threshold of LPS is calibrated with a 1% FP rate budget on validation traces generated by oracle replay. All tokens committed during replay agree with x0x_0, whereas LPS may commit incorrect tokens at deployment, and these commitments are irreversible, so the decoding state can deviate from the replay trajectory. We therefore agree that whether this budget still holds at deployment should be verified directly. Therefore, we measure the FP rate of LPS directly on the decoding trajectories of the deployed EB-Decode.

Experimental logic. The FP rate reported on the validation set is computed from three quantities at each state: the label x0x_0, the current top-1 prediction of the model, and whether LPS decides to commit the position. At deployment, committed tokens come from the decisions of LPS itself, so the decoding state no longer corresponds to x0x_0. We therefore construct a new label on each deployment state and record the other two quantities directly. Specifically, at each step of a deployment trajectory, the current state is decoded by the full EB-Decode (LBS and LPS). For each masked position in the current block, we record (1) the label, i.e., the token at this position after we disable LPS only at this step and complete the current block from the current state with confidence-based decoding, which plays the same role as x0x_0 on the validation set, (2) the current top-1 token, and (3) whether LPS decides to commit this position. A position is positive if its top-1 token matches the label and negative otherwise, and a negative position committed by LPS is a false positive. The FP rate is the number of positions that ever produce a false positive divided by the number of positions that are ever negative. The definition is therefore identical to that on the validation set. The key difference is that the states come from the deployed EB-Decode rather than from oracle replay, so the measured value is the FP rate at deployment. This check only reads the decoding states and does not change the output of EB-Decode.

We run this check with LLaDA-8B-Instruct on 500 randomly sampled GSM8K test questions under the same configuration as Table 1, excluding positions where the top-1 token or the label is EOS. Table R.3 reports the FP rates on the validation set and at deployment, and we add these results to Table 15 in Appendix K of the revised paper.

Table R.3: FP rate of LPS on the validation set and at deployment. Brackets show the 95% confidence interval from bootstrapping over questions.

Validation (oracle replay)Deployment (EB-Decode trajectories)
FP rate (%)0.9981.26 [1.08, 1.46]

Conclusion. The FP rate at deployment is only about 0.26 points higher than on the validation set, so the calibration of LPS on the validation set transfers to deployment. This gap corresponds to only about 0.3 additional false positives per question in a 512-token generation. Moreover, a false positive only means that the committed token differs from the result of confidence-based decoding, often merely in wording, rather than that it is wrong. Accordingly, EB-Decode achieves 79.20% accuracy on these questions, close to 79.80% of confidence-based decoding.

[to be continued]

AuthorsOpenReview ↗

Response to the Reviewer jC9N (Part 4 of 4)

[W2 & Q4] Position Encoding for Non-Contiguous Blocks

We thank the reviewer for this careful observation. For non-contiguous blocks, the positional feature uses the rank of each token within the selected block, which represents its relative order in the block. We have rewritten the formula in Section 4.2 of the revised paper as follows, and revised the corresponding parts of the main text. Let the positions in block BB be sorted by their original indices in ascending order, and let k(i)∈{0,1,…,∣B∣−1}k(i) \in \{0, 1, \dots, |B| - 1\} denote the rank of position ii within the block. Then

posi=k(i)/∣B∣∈[0,1)\mathrm{pos}^i = k(i) / |B| \in [0, 1),

where ∣B∣|B| is the number of tokens in the current block. Hence, posi\mathrm{pos}^i lies in [0, 1) regardless of whether the block is contiguous. For a contiguous block, k(i)=i−bk(i) = i - b, which is consistent with the original formula.

[Q5] EOS Stopping, Throughput Numerator, and Statistical Uncertainty

(1) EOS and the throughput numerator. The throughput numerator of EB-Decode excludes EOS and the padding positions after it. All baselines follow the same definition, and we have added this clarification to Appendix F of the revised paper.

(2) Statistical uncertainty. We compare EB-Decode with Fast-dLLM, the baseline with the most stable accuracy and speedup across all base models and benchmarks in Table 1. We conduct the experiments on LLaDA-8B-Instruct and GSM8K under the same setting as Table 1, and report the uncertainty of both accuracy and speed. Due to server differences, the absolute accuracy and TPS may differ slightly from Table 1. The results are added to Table 16 in Appendix L of the revised paper.

Accuracy. Both methods use greedy decoding, so repeated runs under the same setting produce identical outputs, and the uncertainty of accuracy comes only from the sampling of test questions. Therefore, we resample all 1,319 GSM8K test questions with replacement 10,000 times to obtain the 95% confidence interval of the accuracy of each method. Since both methods are evaluated on the same questions, we also compute their accuracy difference in each resample, which gives the 95% confidence interval of the difference. The results are reported in Table R.4.

Table R.4: Accuracy of Fast-dLLM and EB-Decode with 95% confidence intervals.

MethodAcc. (%)95% CI
Fast-dLLM78.70[76.50, 80.89]
EB-Decode78.39[76.12, 80.59]

The accuracy difference between EB-Decode and Fast-dLLM is −0.30 points, with a 95% confidence interval of [−1.90, 1.29]. Since the interval covers 0, the two methods achieve statistically comparable accuracy.

Speed. To verify that the speedup of EB-Decode over Fast-dLLM is stable, we measure the TPS of each method in 5 independent runs on a fixed set of 300 GSM8K questions, alternating the two methods between runs. The results are reported in Table R.5.

Table R.5: TPS of Fast-dLLM and EB-Decode over 5 timed runs. TPS is measured per GPU.

MethodNFE per questionTPS of 5 runsMean ± stdCV
Fast-dLLM97.0138.26, 137.86, 138.49, 138.39, 138.17138.23 ± 0.240.17%
EB-Decode73.4166.94, 166.92, 167.19, 167.31, 166.64167.00 ± 0.260.16%

The coefficients of variation (CV) of both methods are within 0.2% over the 5 runs, while the mean TPS of EB-Decode is more than 1.2× that of Fast-dLLM. This shows that the speedup of EB-Decode is stable and reproducible. Together with Table R.4, these results confirm that EB-Decode delivers a stable speedup while maintaining accuracy.

Reviewer_zUAoOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EB-Decode, a decoding framework motivated by the observation that many diffusion-model token predictions stabilize before reaching the usual commitment-confidence threshold. With the backbone frozen, a learned block-selection router groups positions into variable-size blocks, while a position-aware parallel-sampling router learns when to commit predictions early from decoding traces. Experiments on three diffusion models and four benchmarks report throughput improvements of 3.53–18.76× over vanilla decoding, and up to approximately 1.58× over Fast-dLLM. The paper also includes component ablations, cache combinations, router-transfer studies, and runtime measurements. Quality is broadly comparable in many settings, but some tasks show nontrivial degradation.

Critical strengths
  1. The early-stabilization observation leads to a concrete, deployable intervention. The method distinguishes prediction stability from threshold crossing and learns separate policies for block selection and early commitment. Keeping the backbone frozen while exploiting information already available during decoding is a compelling way to reduce denoising passes without retraining the language model itself. The observation and the two-component design form the strongest conceptual contribution (pp. 3–7).
  2. The reported gains include comparisons with an accelerated baseline and useful transfer tests. The main experiments cover multiple backbones and tasks, and the improvement over Fast-dLLM is more informative than the larger vanilla speedup alone. Cache combinations and selected cross-model and cross-length router transfers add evidence that the approach is not confined to a single configuration. These findings make the efficiency contribution potentially substantial within the measured setting (pp. 8–9, 19–21).
Critical weaknesses
  1. Keeping the backbone frozen does not make EB-Decode training-free or necessarily inexpensive to deploy. It still requires trace generation, router training, threshold tuning, and maintenance of additional modules. The paper mainly emphasizes online speedups without systematically quantifying these costs.
  2. The experiments mainly report a single operating point. The authors should provide Accuracy–TPS curves by varying decoding thresholds or commit budgets for EB-Decode and the baselines, demonstrating whether EB-Decode achieves a consistently better Pareto frontier.
  3. The evaluation focuses on mathematical reasoning and code generation, whose structured outputs may favor early commitment. Experiments on broader tasks, particularly instruction-following benchmarks, are needed to verify the method’s generality.
Questions for authors

Please refer to weaknesses.

4 comments
AuthorsOpenReview ↗

Response to the Reviewer zUAo (Part 1 of 2)

We sincerely thank the reviewer for the constructive review, and for recognizing that EB-Decode is a deployable method motivated by early stabilization, which separately learns block selection and early commitment on a frozen backbone, as well as the value of the Fast-dLLM comparison and the router transfer experiments. Regarding the reviewer’s concerns, we would like to provide further experimental results and clarifications as follows.

[C1] Training, Tuning, and Maintenance Cost

We thank the reviewer for this comment. EB-Decode requires one-time trace generation, router training, and threshold selection. We did not claim that EB-Decode is training-free, and we think it is more accurately positioned as a lightweight-training method. Because EB-Decode trains only two lightweight routers and leaves the pretrained backbone weights unchanged, this one-time cost is far lower than that of training methods that modify the backbone.

Section 5.1 of the original paper (Lines 358–362) reports the offline cost of trace generation and router training. Trace generation takes about 2 hours on 4 H100 GPUs, while LPS training and LBS training take about 10 minutes and 2 hours on a single H100 GPU, respectively. In contrast, methods that modify the backbone weights are far more expensive, e.g., about 80 A5000 GPU hours for the distillation method dParallel [1], 405 H100 GPU hours for the VRPO post-training of LLaDA-1.5 [2], and over 1,000 A100 GPU hours for the reinforcement learning method DICE on SDAR-8B [3].

Tuning, inference, and maintenance also incur only modest overhead. For hyperparameters, gmax⁡g_{\max}, Lmin⁡L_{\min}, and τ are set once per backbone and shared across all benchmarks, where τ = 0.9 follows the baselines and τψ\tau_\psi is set based on the FP budget, so no per-task tuning is needed. For inference, Section 5.3 of the original paper reports that LBS and LPS take 3.80% and 2.41% of the per-step backbone forward time, respectively, and 6.21% in total. Although the routers introduce this marginal per-step overhead, EB-Decode reduces the number of decoding steps and therefore still yields a clear overall speedup. The maintenance cost is also low. The two routers contain only about 0.63M parameters in total, and LBS directly reuses the word embedding table of the backbone. Since the backbone weights remain unchanged, a deployed model only needs to load these two lightweight modules to switch between EB-Decode and the original decoding.

Moreover, the routers of EB-Decode are transferable, so a single training run can be reused across multiple settings, which further amortizes the training cost. The transfer experiments in Appendix G show that routers trained on LLaDA-8B-Instruct can be directly applied to LLaDA-1.5 from the same series, Dream from a different series, and different generation lengths without retraining, with accuracy and throughput comparable to those of natively trained routers. For convenience, we list the results of Appendix G again in Table R.1.

Table R.1: Router transfer without retraining on GSM8K. “Native” uses routers trained in the target setting, and “Transferred” uses the router(s) listed in the second column.

Transfer settingTransferred router(s)Native Acc. (%) / TPSTransferred Acc. (%) / TPS
LLaDA-8B-Instruct → LLaDA-1.5 (same series)LBS and LPS79.23 / 64.9579.23 / 66.37
LLaDA-8B-Instruct → Dream-v0-Instruct-7B (cross-series)LPS76.19 / 43.5576.49 / 43.07
Train at L = 256, evaluate at L = 512LPS78.17 / 69.5478.24 / 66.95
Train at L = 512, evaluate at L = 256LPS74.68 / 95.3774.15 / 94.08

Although these costs are low, we fully agree with the reviewer that they should be clarified carefully in the paper. The original paper reports these costs in Section 5.1 (trace generation and router training), Section 5.3 (inference overhead), and Appendices C, D, and G (router architectures, training procedures, threshold selection, and router transfer), but they are spread across the main text and the appendix. We therefore add a dedicated paragraph (Training and Deployment Cost) to Section 5.3 of the revised paper, which summarizes the costs of trace generation, router training, threshold selection, and the maintenance of additional modules. It shows that EB-Decode requires only one lightweight training run without modifying the backbone weights, that its total cost is far lower than that of backbone-modifying training methods, and that the trained routers can be directly reused across multiple settings.

[to be continued]

AuthorsOpenReview ↗

Response to the Reviewer zUAo (Part 2 of 2)

[C2] Accuracy–Speed Curves

Following the reviewer's suggestion, we add accuracy–speed curves. Since figures cannot be included in the comments under the ICLR policy, we add the curves as Figure 5 in Section 5.2 of the revised paper. Figure 5 shows the curves on GSM8K with LLaDA-8B-Instruct and LLaDA-1.5, obtained by sweeping the decoding threshold of each method. Besides the baselines in Table 1, we include Swordsman and a tuned entropy-threshold block selector. Since the official implementations of Learn2PD and Swordsman do not provide configurations for LLaDA-1.5, these two baselines are compared only on LLaDA-8B-Instruct. The absolute TPS may differ slightly from Table 1 due to server differences.

We observe that EB-Decode achieves the best trade-off on both models. With similar accuracy to vanilla decoding, EB-Decode reaches about 180 TPS on both models, which is about 7× that of vanilla decoding and higher than all baselines. On LLaDA-1.5, the operating points of EB-Decode form the Pareto frontier. This shows that EB-Decode consistently improves the accuracy–speed trade-off across decoding thresholds, rather than only at a single operating point.

[C3] Evaluation on Instruction Following

We agree with the reviewer that instruction-following tasks provide an effective test of the generality of EB-Decode. We therefore evaluate EB-Decode on the instruction-following benchmark IFBench [4] and add the results to Table 5 in Section 5.3 of the revised paper.

The experimental setting is the same as in Table 1. Both routers are the versions trained on math data, without retraining on instruction-following data. Following the IFBench authors, we report the prompt-level loose accuracy, together with the number of forward passes per prompt (NFE), as shown in Table R.2.

Table R.2: IFBench results on LLaDA-8B-Instruct.

MethodPrompt-level loose (%)NFE per promptTPSSpeedup over vanilla
Vanilla25.33512.021.881.00×
Fast-dLLM25.67188.959.972.74×
EB-Decode25.67148.566.733.05×

Even though both routers are trained only on math data, we see that EB-Decode generalizes well to the instruction-following task. On IFBench, it matches the accuracy of vanilla decoding with a 3.05× speedup, achieves 1.11× the throughput of Fast-dLLM, and uses 21% fewer forward passes per prompt than Fast-dLLM. This shows that EB-Decode remains effective on free-form instruction-following tasks without task-specific retraining.

References

[1] Chen, Zigeng, et al. “dParallel: Learnable Parallel Decoding for dLLMs.” ICLR (2026).

[2] Zhu, Fengqi, et al. “LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Models.” ACL (2026).

[3] Bai, Haolei, et al. “DICE: Diffusion Large Language Models Excel at Generating CUDA Kernels.” arXiv preprint arXiv:2602.11715 (2026).

[4] Pyatkin, Valentina, et al. “Generalizing Verifiable Instruction Following.” NeurIPS Datasets and Benchmarks Track (2025).

Thanks for your rebuttal. I think my concerns are addressed. I will raise my score to 3.

AuthorsOpenReview ↗

Thank You for Your Reply

Dear Reviewer zUAo,

Thank you for your reply and for raising your score. We truly value your feedback and your recognition of our work, and we appreciate the time and effort you have devoted to reviewing our manuscript.

Best regards,

Authors of Submission #9370

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global response summarizing new experiments and clarifications.

Dear AC and Reviewers,

First, we deeply appreciate the time and effort you have devoted to handling our submission and preparing the review, particularly given the substantial scale of a conference like ICLR. Your efforts are truly valued.

We are encouraged by the positive feedback and grateful for the recognition of the clear motivation of our work (zUAo, P4VJ), the design that separately learns block selection and token commitment on a frozen backbone (zUAo, jC9N), and the support for non-contiguous blocks (jC9N). We also appreciate the recognition of the consistent speedups across multiple backbones and benchmarks (jC9N, P4VJ), the comparisons with Fast-dLLM and KV caching (zUAo, jC9N), the router transfer experiments (zUAo, jC9N), the comprehensive ablations (jC9N, P4VJ), and the clear writing (P4VJ).

In addition to these commendations, the reviewers requested further experiments and clarifications. The new experiments are added to Section 5 and the appendices of the revised paper, and all revisions are highlighted in blue. Since ICLR does not allow figures in comments, all figures are provided in the revised PDF.


To summarize, the following experiments have been supplied:

  • Accuracy–speed curves: see our responses to Reviewer zUAo’s C2, Reviewer jC9N’s W1 & Q1 & Q3, and Reviewer P4VJ’s W1, added to Figure 5 in Section 5.2
  • Evaluation on the instruction-following benchmark IFBench: see our response to Reviewer zUAo’s C3, added to Table 5 in Section 5.3
  • Comparison with Swordsman: see our response to Reviewer jC9N’s W1 & Q1 & Q3, added to Figure 5(a) in Section 5.2
  • Comparison between LBS and tuned confidence and entropy selectors: see our response to Reviewer jC9N’s W1 & Q1 & Q3, added to Figure 6 in Section 5.3 and Appendix J
  • FP rate of LPS on deployment trajectories: see our response to Reviewer jC9N’s W2 & Q2, added to Table 15 in Appendix K
  • Confidence intervals of accuracy and repeated speed measurements: see our response to Reviewer jC9N’s Q5, added to Table 16 in Appendix L
  • Adaptation and results of EB-Decode on DMax-16B: see our response to Reviewer P4VJ’s W2, added to Table 17 in Appendix M
  • Results of the full EB-Decode (LBS+LPS) on block-diffusion dLLMs: see our response to Reviewer P4VJ’s W3, added to Table 6 in Section 5.3

To summarize, the following questions have been clarified:

  • Training, tuning, and maintenance costs of EB-Decode and the reuse of its routers: see our response to Reviewer zUAo’s C1, summarized in a new paragraph in Section 5.3
  • Differences between EB-Decode and Learn2PD and Swordsman, and the corrected description of Swordsman in Table 12 of the original paper (now Table 13 in Appendix H): see our response to Reviewer jC9N’s W1 & Q1 & Q3
  • The two-stage generation procedure of the LPS training traces and the corresponding revisions to Section 4.2, Appendix D.2, Algorithm 1, and Algorithm 4: see our response to Reviewer jC9N’s W2 & Q2
  • The corrected position encoding for non-contiguous blocks: see our response to Reviewer jC9N’s W2 & Q4, revised in Section 4.2
  • Exclusion of EOS and the subsequent padding positions from the throughput numerator: see our response to Reviewer jC9N’s Q5, clarified in Appendix F
  • Why Table 4 of the original paper (now Table 6 in the revised paper) evaluates only LBS, and how LBS is adapted to block-diffusion dLLMs: see our response to Reviewer P4VJ’s W3

We hope the new experiments and clarifications address your concerns, and we are happy to provide further details during the discussion period. Thank you!

Best regards,

Authors of Submission #9370

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.