acceptodds
Under review as a conference paper at ICLR 2027

Supporting Tokens First: Dirichlet Prior-Guided Decoding for Diffusion Language Models

Abstract

Unlike auto-regressive language models, diffusion language models (DLMs) iteratively refine masked sequences and can reveal multiple positions in parallel, offering the potential for faster generation. However, unresolved positions can remain conditionally dependent, making parallel decoding potentially unreliable. Explicitly modelling these dependencies through latent-variable inference is computationally expensive, while existing DLM decoding strategies primarily rely on instance-level signals, such as model confidence or attention, which can be noisy at capturing a token's influence on subsequent predictions. To address these limitations, we hypothesise the existence of supporting tokens that, when revealed early, reduce conditional dependence among the remaining masked positions. Under an idealised Bayesian latent-variable model, we show that prioritising tokens with greater prior support can reduce residual dependence in expectation. Building on this insight, we propose Prior-Guided Diffusion Decoding (PGDD), a training-free framework that exploits stable, population-level vocabulary statistics to identify candidate supporting tokens without costly latent-variable inference. Specifically, an answer-free Dirichlet Prior Bank captures query-conditioned vocabulary priors from an external corpus, which are combined with the DLM's current predictions through a Dirichlet posterior update to guide token commitment. By complementing noisy instance-level confidence with population-level prior information, PGDD prioritises supporting tokens, reducing residual conditional dependence and enabling more reliable parallel decoding. Empirical analysis confirms that higher-prior tokens tend to yield greater confidence gains at other masked positions early in denoising, while prior-guided ordering reduces residual conditional dependence. Experiments on mathematical reasoning, code generation, and planning demonstrate improved generation quality and decoding efficiency with limited computational overhead compared with recent baselines.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language models, prior-guided decoding, parallel decoding, training-free inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HZAZOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PGDD, a training-free decoding method that uses an answer-free vocabulary prior to guide token commitment order. The prior bank groups training queries and aggregates the model's initial predictions. At inference, the retrieved prior is combined with current confidence while retaining the model's original top-1 tokens. Experiments mainly use LLaDA-8B-Instruct on reasoning, code generation, and Countdown.

Critical strengths
  1. The method is simple and easy to integrate. Prior retrieval requires no additional DLM forward pass at test time. The comparisons with frequency-based, linguistic, and unrelated-domain priors are useful for understanding what information the prior bank contributes.

  2. The empirical results show useful complementary gains. On HumanEval with a 128-step budget, adaptive decoding improves from 26.22 to 29.27 while average steps decrease from 60.0 to 47.5. The combinations with Prophet, AHD, and Fast-dLLM further support its compatibility with existing decoders.

Critical weaknesses
  1. The theory and the implemented decoder use different scoring rules. Theorem A.1 analyzes a multiplicative prior-weighted score, while PGDD uses an additive score evaluated at the model's original top-1 token. These rules can rank positions differently. The theorem provides motivation, but its connection to the practical decoding rule remains incomplete.

  2. The uniform-prior control has an unexplained discrepancy. Under fixed-step decoding, a uniform prior should preserve confidence ordering apart from numerical or tie-handling effects. However, Table 8 reports HumanEval changing from 26.83 to 26.22. This is a small difference, but it needs clarification when many reported gains are around one percentage point.

Questions for authors

N/A

1 comment

We sincerely thank the reviewer for these careful observations. Both concerns are valid, and we appreciate the opportunity to clarify them.

Q1. Relationship between Theorem A.1 and the implemented decoder.

We thank the reviewer for this observation. As discussed in the paper, Theorem A.1 is intended to provide theoretical motivation for prior-guided token prioritization, rather than a direct guarantee for the exact scoring rule used in PGDD.

As described in the abstract and the main text, the theorem studies an idealized setting with a multiplicative prior-weighted score, showing how prior information can help identify influential token positions. In practice, PGDD uses an additive combination of prior weight and model confidence, evaluated at the model's original top-1 prediction. This allows us to incorporate prior information into the commitment order without changing the model's predicted tokens.

We agree that the two scoring rules are not mathematically equivalent and may produce different rankings. The theoretical result supports the motivation for prior-guided decoding, while the effectiveness of the practical scoring rule is evaluated empirically. We will make this distinction clearer in the revised paper.

Q2. Uniform-prior control discrepancy.

We thank the reviewer for identifying this inconsistency. After reviewing our original experiment logs, we found that the initial HumanEval baseline was evaluated with batch_size=4, while the uniform-prior control and all other experiments used batch_size=1.

Different batch sizes can introduce small numerical differences during inference, potentially affecting confidence-based token commitment. This explains a possible source of the reported discrepancy. HumanEval contains 164 problems, and the reported accuracies of 26.83% and 26.22% correspond to 44 and 43 correct solutions, respectively. We further checked the per-problem results and confirmed that only one HumanEval test case differed between the two runs.

We have also rechecked the other reported results and confirmed their correctness. We will correct the HumanEval baseline comparison using consistent batch settings and clarify this detail in the revised paper.

We sincerely appreciate the reviewer's careful observation, which helped us identify this inconsistency in the experimental settings and improve the reliability of our evaluation.

Reviewer_nNLtOpenReview ↗
Rating
2(−1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Prior-Guided Diffusion Decoding (PGDD), a training-free decoding method for diffusion language models. The method aims to reduce conditional dependencies among masked positions by prioritizing informative supporting tokens. PGDD constructs a query-conditioned Dirichlet Prior Bank from external answer-free queries and combines the retrieved priors with model predictions to guide token commitment order. The authors provide theoretical motivation through a Bayesian latent-variable model and empirically evaluate the method across five benchmarks, demonstrating improvements in generation quality and decoding efficiency.

Critical strengths

The paper addresses conditional dependencies in parallel DLM decoding by prioritizing informative supporting tokens. The proposed method is well-motivated and training-free, using query-conditioned Dirichlet priors to guide token commitment without additional model forward passes.

Critical weaknesses

The paper assumes that tokens with higher vocabulary-level prior weights are more informative and should be prioritized during decoding. However, a token that frequently receives high predictive probability across related queries is not necessarily informative for resolving dependencies in a specific instance. Although the empirical analysis provides some supporting evidence, the connection between prior weight and token informativeness could be better justified.

Questions for authors

A token that frequently receives high predictive probability across related queries may not necessarily provide useful information for resolving dependencies in a specific instance. Could the authors further justify the connection between prior weight and token informativeness?

Since the posterior predictive probability is essentially a weighted average of the vocabulary prior and the model's current prediction, could the authors clarify what additional benefits the Bayesian formulation provides compared to a heuristic confidence score based on linear interpolation?

1 comment

We thank the reviewer for the constructive feedback. We address the two concerns below and will clarify these points in the revised paper.

Q1. Connection between prior weight and token informativeness. We agree that a higher prior weight does not necessarily imply greater token informativeness in every instance. Our motivation is based on empirical observations rather than a theoretical equivalence. The observed relationship between prior weight and token influence varies across benchmarks and denoising stages, but generally supports prioritizing high-prior tokens during early decoding. Furthermore, task-related priors generally outperform unrelated priors, suggesting that the relevance of prior information matters. We therefore treat prior weight as an imperfect, task-conditioned proxy for informativeness and combine it with the model's instance-specific confidence rather than relying on it alone.

Q2. Bayesian formulation versus linear interpolation. We thank the reviewer for this question. The Bayesian formulation provides a more principled derivation and interpretation of the linear combination. Specifically, Dirichlet–categorical conjugacy enables closed-form additive updates, where the Dirichlet parameters represent vocabulary-level support as pseudo-counts. This provides a probabilistic foundation for combining task-level prior information with instance-specific confidence, while λ\lambda controls their relative contributions. Thus, the value of the Bayesian formulation lies in its interpretable evidence-update framework rather than a different interpolation rule. We will make this distinction clearer in the revised paper.

Reviewer_Dqp2OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes prior-guided diffusion decoding (PGDD), a training-free diffusion language model (DLM) decoding framework which uses an answer-free Dirichlet Prior Bank (DPB) to inform commitment to specific token unmasking.

The DPB is a clustering constructed from aggregated step-0 predictions from the DLM on related answer-free queries, e.g. questions from the same benchmark. The test query is used to retrieve the nearest cluster centroid from the DPB, which is then scaled as a time-dependent prior depending on the decoding step. Combining the prior with the current DLM prediction gives the Dirichlet posterior which is then sampled.

PGDD is evaluated on math and coding tasks under fixed-step and adaptive decoding on LLaDA-8B-Instruct, including a comparison using Fast-dLLM, an accelerated decoder, and to two additional decoding methods.

Analysis on the source data for the prior shows that task-related priors are linked to generally higher performance.

Critical strengths

The proposed approach of incorporating explicitly Bayesian priors in diffusion language modeling is not well explored in the literature

Critical weaknesses

Very weak empirical evidence for the usefulness of including a prior, with some over-claiming and cherry-picking of results.

Questions for authors
  • Why are the correlations for GSM8K (Figure 7a) and MATH-500 (Figure 2) so different?
  • The results in Table 1 appear to have a very small median difference between PGDD and the baseline. Are these differences measurement noise?
  • Figure 4 highlights PGDD as better and faster than LLaDA in all cases, but Table 7 shows this is not true for τ=0.8\tau = 0.8. Why was this threshold excluded from the main text?
  • In Table 3, the performance gain from the MBPP prior on HumanEval is an outlier within the entire table. Do you have an explanation for this? Since both MBPP and HumanEval are python benchmarks and have some functional overlap in queries, did you remove functionally identical questions before creating the prior from mbpp_c8? (Does this matter?)
  • It is claimed in Section 4.4 that "larger prior banks provide additional benefits on other tasks", but is there evidence for this?
1 comment

Response to the Main Weakness.

We thank the reviewer for the constructive feedback. We would like to clarify that the effectiveness of PGDD is empirically demonstrated across a set of standard benchmarks covering mathematical reasoning (GSM8K, MATH-500), coding (HumanEval, MBPP), and planning (Countdown), under both fixed-step and adaptive decoding settings. As reported in Section 4.2, PGDD improves performance in 26 out of 32 settings, with an average absolute gain of 1.15 percentage points, including a +5.49 pp improvement on HumanEval. Furthermore, Section 4.4 shows that task-related priors generally outperform unrelated priors, supporting the usefulness of prior guidance. We also report both favorable and unfavorable results, rather than selectively highlighting improvements.

Q1. Correlation differences between GSM8K and MATH-500.

Although both benchmarks involve mathematical reasoning, they differ in difficulty and problem types. GSM8K focuses on everyday math problems (e.g., shopping calculations), while MATH-500 contains more challenging problems (e.g., algebra). This may explain the stronger correlation between prior weight and token influence on MATH-500. At 25% completion, the correlation remains clear on MATH-500 ((r=0.54)), but not on GSM8K ((r=-0.06)), suggesting that prior guidance may be more useful on MATH-500.

The main purpose of this analysis is to show that tokens with higher prior weights tend to have greater influence on other masked positions at the beginning of decoding. As more tokens are revealed, this difference gradually decreases. We use this analysis to motivate prior guidance during early decoding, rather than to claim that high prior weight always means high token informativeness.

Q2. Small performance differences and measurement noise.

To address the concern about measurement noise, we conducted additional evaluations and performed paired bootstrap resampling on per-example results. Under Early Commit, PGDD improves HumanEval accuracy from 26.22% to 29.27% (+3.05 pp, +11.6% relative), with a bootstrapped improvement of +3.05 ± 3.04 pp (mean ± std). Furthermore, a two-sided sign test across the evaluated settings yields $(p=1.9\times10^{-4}) $, providing additional evidence of a consistent positive trend.

Our conclusion is based on the broader improvements across benchmarks and decoding settings, rather than any single marginal gain.

Q3. Threshold selection in Figure 4 and Table 7.

The threshold sweep examines the quality–efficiency trade-off as τ\tau decreases from 1 (fixed-step decoding) toward commonly adopted settings, rather than selecting favorable hyperparameters. For example, Fast-dLLM (ICLR 2026) uses τ=0.9\tau=0.9, while DOS (ACL 2026) evaluates with τ=0.95\tau=0.95. Figure 4 therefore focuses on six representative thresholds from 0.850.85 to 0.9750.975.

The more aggressive τ=0.8\tau=0.8 was explicitly reported in Appendix Table 7. At this threshold, PGDD still improves HumanEval by 3.66 percentage points, while GSM8K decreases by only 0.30 points, with nearly the same number of decoding steps.

Our aim is to demonstrate a generally favorable quality–efficiency trade-off across thresholds, rather than claim that PGDD achieves the highest accuracy and lowest decoding cost at every operating point. We will clarify this in the main text.

Q4. MBPP prior transfer to HumanEval and potential overlap.

As discussed in Section 4.4, capturing task-related functional similarities is precisely the motivation behind our prior design. MBPP and HumanEval share common Python programming patterns, such as arithmetic operations, string manipulation, and list processing. These similarities allow an MBPP-derived prior to provide more relevant guidance than unrelated priors, which may explain the larger improvement.

Functional similarity is desirable for prior construction, as our goal is to capture transferable task-level token statistics rather than memorize solutions. We used the original MBPP training split without functional deduplication, and found no exact text duplicates between the 374 MBPP training queries and 164 HumanEval test problems. Moreover, PGDD is answer-free: the prior uses only training queries, without reference solutions or test cases, and adjusts only the commitment order of DLM-predicted tokens.

Q5. Effect of prior bank size. We thank the reviewer for pointing out this ambiguity. By “larger prior banks,” we refer to increasing the number of clusters KK, which enables finer-grained partitioning of query contexts and more specialized prior retrieval. As shown in Table 4, several tasks benefit from this finer granularity, whereas HumanEval already performs best with K=1K=1, suggesting that a global prior is sufficient for this task. We do not claim that larger KK consistently improves performance; rather, the benefit of finer-grained priors is task-dependent. We will clarify this wording in Section 4.4.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.