acceptodds
Withdrawn from ICLR 2027

Disentangling Diffusion Decoding: Selection, Pace, and Numerical Semantics

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Masked diffusion decoders differ in position selection, commitment pace, verification, and numerical semantics, while scoring conventions also affect reported gains. We study these choices under explicit comparison contracts across five models, from three model families, and four benchmarks. In fixed-selector sweeps, doubling commitments per step halves denoiser evaluations and lowers accuracy in all 60 adjacent transitions. Selection gains also depend on pace. Across five controlled comparisons on GSM8K, confidence and two adapted attention-based selectors, ADAS and DAPD, are each regenerated at four and eight commitments under a shared harness and scorer within every checkpoint. On four models, ADAS's advantage over confidence increases 7.7–10.2 percentage points more than DAPD's as pace rises, with 95% confidence intervals excluding zero. These results characterize the adapted policies and do not establish task-general superiority. Replica batching changes 24–89% of raw output sequences across five models. We trace these output differences through termination, scorer normalization, answer extraction, and correctness. Extracted mathematical answers differ on 3.9–28.6% of items, whereas correctness changes on 2.4–7.0%; gains and losses partly offset, and every paired aggregate-accuracy interval contains zero. Supporting experiments show that token persistence does not prove peer reinforcement, while offline prediction and reference agreement do not guarantee effective or efficient decoding. Together, these results show why decoder comparisons must specify selection, pace, numerical execution, scoring, and verification rather than treating an end-to-end gain as a single intervention.

Primary area:
foundation or frontier models, including LLMs
Keywords:
diffusion language models, parallel decoding, token selection, speed-quality tradeoff, speculative verification, numerical reproducibility

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vaLfOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper carries out an empirical study of discrete diffusion language models across four different axes: evidence, selection, pace and verification. The authors evaluate how design choices in these four different axes affect generation quality across a number of models and datasets.

Critical strengths
  • A thorough evaluation of different decoding techniques is currently lacking in the literature, to the best of my knowledge, and addressing this gap has a potentially significant impact on the community.
  • Splitting the analysis across four different axes is a neat way to approach the problem and inform the readers how to design decoders. The interaction of pace and selection is particularly interesting.
Critical weaknesses
  • The main weakness in my view is that the paper reads as a series of empirical observations, some of which are not novel. The paper could still be a valid contribution to the literature if it provided either (1) an analysis that explains the results and informs how to design better decoding strategies or (2) an evaluation protocol to standardize the evaluation of discrete LLMs in the literature. The paper seems to be closer in spirit to the latter but does not propose an actionable and structured protocol, stopping at the identification of the four axes of evidence, selection, pace and verification.
  • Some of the empirical observations have already been observed in the literature before. For instance, the fact that generation quality drops with pace is well known in the literature and is the main motivation behind many decoding strategies, including DAPD and ADAS. Reproducibility issues due to batching have also been observed before: [1, Fig. 7] already reports a difference in perplexity due to batching (1 vs 8, in their case).
  • I would not suggest the authors to expand the experiments necessarily; improving on the analysis would be much more interesting, as mentioned in my first comment. However, for the purely empirical paper we have now, the experiments are somewhat limited. There are four datasets covering only coding and reasoning, and only three selection methods (confidence, ADAS and DAPD). Other more recent and larger open-weight models, such as DiffusionGemma and LLaDA2.0, are not considered, although I appreciate that their computational requirements may make a thorough empirical evaluation impractical. A discussion of whether the findings generalize to these models would nevertheless strengthen the paper.
  • The paper is mostly easy to read, but the observations are not clearly connected to a central research question or overarching conclusion. Sections 6 and 7 are very hard to follow though.

Minor comments

  • The abstract contains many acronyms and numbers. This is perhaps a matter of personal preference, but I would avoid those in the abstract so that the main message can shine through more clearly.
  • Some of the terms used throughout the text are confusing. For instance, it was not immediately clear to me what was meant by “numerical identity”, “reference fidelity” or “offline discrimination”. Defining these notions more clearly would make the paper read better.
  • The mathematical description in the first paragraph of section 2 is never really used, unless I missed it. I would suggest dropping it from the paper entirely, as it does not add much.

References

[1] Zheng, Kaiwen, et al. "Masked diffusion models are secretly time-agnostic masked models and exploit inaccurate categorical sampling." International Conference on Learning Representations. Vol. 2025. 2025.

Questions for authors
  1. Why focus only on ADAS and DAPD? Why not also FastDLLM or EB-Sampler, for instance?
  2. The effect of batch size and numerical representation in decoding is well known in the literature and not exclusive to diffusion models, as the authors discuss in the related work. Could the authors comment on potential explanations and solutions for this phenomenon in the context of diffusion models? Are there fundamental differences between diffusion and autoregressive LLMs in that regard or could one borrow solutions proposed for autoregressive models [2]?
  3. In discrete diffusion models, the number of masks allocated for the generation has also been shown to be important [3]. How is that handled in the experiments and why not study the number of masks in this framework too?
  4. What are JUDGE and VJUDGE? Unless I missed it, these terms are introduced without context or citation.
  5. Sections 6 and 7 are hard to follow. Could the authors clarify what exactly is evaluated and discussed in those sections?

References

[2] Yuan, Jiayi, et al. "Understanding and mitigating numerical sources of nondeterminism in llm inference." Advances in Neural Information Processing Systems 38 (2026): 169819-169851.

[3] Piskorz, Julianna, et al. “Masks Can Be Distracting: On Context Comprehension in Diffusion Language Models.” ICML 2026, PMLR 306:98965-99002, 2026.

Reviewer_PN2wOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how token selection, commitment pace, numerical execution, and scoring affect masked-diffusion decoding. Experiments across five models and four benchmarks examine accuracy–cost tradeoffs, selector–pace interactions, and batching-induced output changes. Additional diagnostics distinguish local reference agreement from downstream correctness and computational efficiency.

Critical strengths
  1. The numerical and scoring audits provide concrete evidence that evaluation choices can substantially alter apparent decoder gains.

  2. The controlled selector–pace experiments estimate the interaction directly using a shared evaluation harness and paired resampling.

Critical weaknesses

The central numerical audit's execution contract is incompletely specified. The main selection–pace result does not sufficiently isolate what drives the interaction.

Questions for authors
  1. What selector, commitment schedule, attention backend, and model revisions were used for Tables 1 and 4?

  2. Can you share the retained paired outputs and reproduction scripts, and clarify which settings were verified as unchanged between batch-one and replica-batch runs?

Reviewer_oNmSOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the evaluation of masked diffusion language model decoders. The authors organize decoding into evidence, selection, pace, and verification to separate factors that can otherwise be confused and confounded in performance comparisons. They conduct controlled experiments across five models and four benchmarks, examining selection–pace interactions and sensitivity to numerical execution and scoring. The results show that selection methods respond differently to increased pace and that execution changes can alter individual answers without a detected change in aggregate accuracy.

Critical strengths

The scoring comparison shows that an apparent decoder advantage can come from the scorer alone. On SDAR's GSM8K outputs, changing the scorer reverses the ranking of adapted DAPD and confidence without changing their generated text (Tables 6–7). The numerical analysis also identifies correct-to-incorrect and incorrect-to-correct changes that largely cancel in aggregate accuracy (Table 1).

Critical weaknesses

The main weakness pertains to how valuable the contribution is. I don't think the paper adds much beyond what earlier work already establishes and provides too little new insight into diffusion decoding to justify acceptance. Section 8 acknowledges that earlier work already established selection–pace interactions and sensitivity to numerical execution. The study measures these effects more precisely through paired selector comparisons and by separating output changes from accuracy changes. However, the main conclusions remain that decoder rankings depend on the comparison conditions and aggregate accuracy can hide differences between outputs. The E/O/P/C framework organizes these checks without adding much beyond standard controlled evaluation.

Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.