acceptodds
Under review as a conference paper at ICLR 2027

Reliable Parallel Decoding in Masked Diffusion Language Models

Abstract

Masked diffusion language models (MDLMs) can generate text efficiently by predicting multiple masked tokens in parallel, but predictions from the same forward pass are not necessarily reliable when committed together. We study when parallel commitment is reliable. Our diagnostics show that confidence alone does not determine a reliable commitment order: confident predictions near the end of the sequence can fix an answer before its supporting computations are established, and downstream predictions become less reliable as the uncertainty of their upstream context grows. At the same time, a single forward pass can already resolve several masked tokens, and predictions that remain stable across the final layers are more likely to be correct. Based on these findings, we propose Reliable Parallel Decoding (RPD), a training-free method that selects candidates by layerwise prediction stability and final confidence, and commits them under a cumulative entropy budget over their preceding masked positions. RPD defers predictions with uncertain upstream context while committing the remaining candidates in parallel, without relying on a fixed block schedule. Across mathematical reasoning and code generation benchmarks on LLaDA and Dream, RPD achieves the highest decoding throughput among the evaluated methods while maintaining or improving accuracy. Code is available at https://anonymous.4open.science/r/RPD-B1EB.

Primary area:
generative models
Keywords:
masked diffusion language models, parallel decoding, efficient inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_BJBLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents a method that uses proxy information (cumulative entropy, confidence drop, and layerwise stability) to help the model determine when to defer the answer. An interesting point is that layerwise hidden states can be used to assess the reliability of token predictions. With this method, the decoding strategy encourages a roughly left-to-right generation order without fixed blocks and achieves a good speed–accuracy trade-off.

Critical strengths

The layerwise hidden states can be used to assess the reliability of token predictions, which gives some new messgae to the communities. With this method, The decoding strategy encourages a roughly left-to-right generation order without fixed blocks and achieves a good speed–accuracy trade-off.

Critical weaknesses

The improvements are minimal, and in some cases, the method performs worse than other methods. On Dream GSM8K, accuracy increases only from 81.73% to 81.80% (+0.07 percentage points), while throughput decreases from 30.03 to 24.56 TPS (an approximately 18.2% reduction), comparing with Fast-Dllm. I would be interested in seeing whether the authors could demonstrate a distinctive benefit of their method beyond existing ordering strategies. Does the cumulative entropy criterion capture task-specific dependencies, or does it primarily impose a left-to-right ordering bias?

Questions for authors
  1. Writing and presentation. I think the writing of this paper could be improved:

    • Figure 2 would be better placed in Section 2 after the concept of stability is introduced, and there is too much explanatory text in the figure.
    • I found Section 2 confusing. I understand that the authors want to explain the motivation behind the different components, but it might be better to introduce a preliminaries section that clearly defines all the concepts before moving on to the analysis.
    • Sentences such as “This follows Figure 2c: as confidence drop increases, the error rate rises from 14.2% to 46.4% for Ki=2K_i=2, but only from 1.8% to 6.9% for Ki≥9K_i\geq9” could be polished.
  2. TPS evaluation at different batch sizes. Regarding the TPS evaluation, projecting intermediate-layer hidden states through the output head may introduce substantial computational overhead. I suspect this method could perform worse at larger batch sizes. Could the authors provide results for different batch sizes?

Reviewer_RYqtOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Reliable Parallel Decoding (RPD), a training-free sampler for masked diffusion language models. It combines final confidence with layerwise prediction persistence and confidence drop to select candidate tokens, then applies a cumulative entropy budget over preceding unresolved positions. The paper evaluates both variants on LLaDA and Dream across GSM8K, MATH-500, HumanEval, and MBPP, with arithmetic diagnostics, component ablations, and runtime analysis.

Critical strengths
  1. The method separates candidate reliability from commitment order in a useful, implementable way. Intermediate-layer predictions provide additional information without retraining or additional backbone evaluations. The ablations support complementary roles for confidence-drop filtering and ordering constraints: persistence alone admits too many erroneous predictions, while cumulative entropy improves accuracy over unrestricted confidence-based decoding.
  2. The reported results demonstrate a useful improvement over the evaluated default samplers. Full RPD matches or improves default accuracy in all eight model–task settings while achieving approximately 2.2–6.1× throughput gains. Evaluating two backbones and both mathematical reasoning and code generation strengthens the empirical case, although the evidence remains limited to 256-token outputs.
Critical weaknesses
  1. The central efficiency claim is stronger than the reported comparisons support. Full RPD is fastest in only one of the eight Table 1 settings; RPD-block supplies the fastest results in the other seven, but has lower reported accuracy than default decoding in five settings. The abstract and conclusion therefore conflate distinct operating points. Figure 4 also shows an EB-Sampler configuration on LLaDA GSM8K with approximately the same accuracy as full RPD and substantially higher throughput. Furthermore, Fast-dLLM is evaluated with KV caching disabled, and LoPA’s stated NFE accounting mixes batched calls with individual branch evaluations. These issues materially limit the claimed practical advantage.
  2. The evidence does not yet isolate the proposed reliability mechanism. Jointly selected upstream predictions are excluded from the entropy budget even though downstream logits were computed while those positions were masked. Individual layerwise stability does not establish joint consistency. The entropy diagnostic additionally conditions on mixed-outcome questions and does not match target confidence or mask distance. Consequently, it remains unclear how much of the benefit comes specifically from entropy-aware dependency handling rather than a positional commitment frontier. The insufficiently specified answer-delay and operand-first interventions further weaken the mechanistic argument.
  3. Both components of RPD have closely related precedents. Stability-Weighted Decoding (SWD) [1] and TACG [2] use prediction stability or persistence to assess commitment readiness, while frontier gating [3], Dependency-Oriented Sampler (DOS) [4], and Deferred Commitment Decoding (DCD) [5] address commitment order and insufficient contextual support. RPD introduces distinct choices—within-forward-pass layerwise stability and an entropy budget over unresolved preceding positions—but the experiments do not establish their advantage over these alternatives. SWD, DOS, and frontier gating are discussed without empirical comparisons, while TACG and DCD are omitted. The component ablations establish contributions within RPD, but do not establish whether these contributions improve upon existing stability-aware and context-aware decoders.

[1] Yue Wu and Jian Huang. Stability-Weighted Decoding for Diffusion Language Models. 2026. arXiv:2604.17068.

[2] Chengcheng Wang, Tingzhang Luo, Wenhao Li, Jianyuan Guo, and Chang Xu. TACG: Trajectory-Aware Commit Gating for Diffusion Language Model Decoding. 2026. arXiv:2607.03236.

[3] Jewon Yeom, Jaewon Sok, Seonghyeon Park, Jeongjae Park, Hwiyeong Lee, and Taesup Kim. Answer First, Reason Later: When Commitment Order Costs Accuracy in Diffusion Language Models. 2026. arXiv:2608.05687v2.

[4] Xueyu Zhou, Yangrong Hu, and Jian Huang. DOS: Dependency-Oriented Sampler for Masked Diffusion Language Models. Findings of ACL, 2026. ACL Anthology.

[5] Yingte Shu, Yuchuan Tian, Chao Xu, Yunhe Wang, and Hanting Chen. Deferred Commitment Decoding for Diffusion Language Models. 2026. arXiv:2601.02076.

Questions for authors
  1. Can you separate the claims for full RPD and RPD-block, address the competitive EB-Sampler point in Figure 4, and provide paired per-example results and timing variability for the small comparative gains?
  2. Does Table 1 count one batched LoPA verification as one evaluation or as four branch evaluations? Please also clarify the rationale and measured effect of replacing LoPA’s published confidence score with negative entropy, and the scope of the speed claim when Fast-dLLM caching is disabled.
  3. Does RPD outperform a positional-frontier baseline matched for commitment rate or NFE? What happens when entropy from simultaneously selected upstream candidates is retained, or when downstream predictions are checked after inserting those candidates?
  4. What exact protocol produces Figure 1(a), including the model, dataset, sample count, and identification of answer regions and operand dependencies? Does it use oracle information? For the entropy diagnostic, what are the effects over all sampled questions and after controlling for target confidence and mask location?
  5. How does RPD compare with SWD or TACG for stability-based selection, and frontier gating, DOS, or DCD for commitment ordering, under matched evaluation conditions? Component-substitution experiments—holding the ordering rule fixed when comparing stability signals, and holding candidate selection fixed when comparing ordering rules—would help establish the benefit of RPD’s specific design choices.
Reviewer_r9NQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies when masked diffusion language models can safely commit multiple tokens in parallel. Through diagnostic experiments, the authors find that downstream predictions are less reliable when upstream context is uncertain, and that predictions stable across the final layers are more likely to be correct. Based on these findings, they propose RPD, a training-free decoding method that selects candidates using confidence and layerwise stability, and commits them under a cumulative entropy budget over preceding masked positions. Experiments cover math reasoning and code generation on LLaDA and Dream.

Critical strengths

The diagnostic study in Section 2 is the most interesting part of the paper. The authors construct controlled masked states from gold reasoning traces and analyze how upstream entropy and layerwise prediction stability relate to error rates. This is a genuine mechanistic analysis, and the two observations are valuable in their own right, independent of whether RPD itself is the best method. The method is training-free, and the idea of using layerwise prediction consistency within a single forward pass to decide which tokens to commit is novel and interesting. The ablation (Table 2) clearly shows the role of each component.

Critical weaknesses

The central claim that "RPD achieves the highest decoding throughput in all eight model–task settings ... while maintaining or improving accuracy" appears to combine two different variants. In the Dream part of Table 1, the throughput advantage comes from RPD-block and the accuracy advantage from Full RPD. Neither variant alone beats Fast-dLLM on both dimensions. Presenting the speed of one variant and the accuracy of the other under the single name "RPD" is misleading. Since this claim is the paper's main contribution, this weakness is the primary reason for my rating.

Questions for authors
  • Appendix A.5.1 states that KV caching is disabled for Fast-dLLM in all experiments, yet KV caching is the main source of Fast-dLLM's speedup in practice. Is this comparison fair? Please provide comparison results with KV caching enabled.

  • "Upstream" is defined as "to the left". Eq. 6 defines the upstream uncertainty of position i as the cumulative entropy of masked positions j < i. This is a natural assumption for autoregressive models, but the defining property of diffusion language models is bidirectional conditioning; especially in code, the information a token depends on is not necessarily to its left. Could the authors justify why this design choice is reasonable?

  • On Dream, the speed advantage comes entirely from RPD-block and the accuracy advantage entirely from Full RPD, and each variant alone loses to Fast-dLLM on at least one dimension, so RPD does not clearly outperform Fast-dLLM overall. Is there an operating point of a single variant that is no worse than Fast-dLLM in both accuracy and throughput on Dream? If not, please revise the claims in the abstract and introduction accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.