acceptodds
Under review as a conference paper at ICLR 2027

Does the Denoising Trajectory Reveal the Source of Uncertainty in Diffusion Language Models?

Abstract

For a large language model, the source of uncertainty determines the appropriate response. An ambiguous prompt calls for a clarifying question, whereas missing knowledge calls for retrieval or abstention. In autoregressive generation, the model decides one token at a time. During a single generation we therefore observe only the distribution over the next position, conditioned on a prefix that is already set. Whether the uncertainty stems from many valid answers or from missing knowledge only becomes visible across repeated generations. Diffusion large language models instead generate a distribution over all positions at every denoising step from a single generation. We ask whether this denoising trajectory reveals the source of uncertainty, and make three contributions. First, we construct source-of-uncertainty labels (SOUL), a benchmark of 16,224 prompts from seven corpora and an authored set, labeled according to established definitions of different sources of uncertainty. Second, we show that the denoising trajectory contains source-specific information across four diffusion language models. Statistical signals summarizing the trajectory identify the source of a prompt well above chance, and outperform the same signals taken from tokenizer-matched autoregressive models, by 0.08–0.22 F1 on task underspecification. Third, we demonstrate that a convolutional probe applied to the full position × denoising-time trajectory performs within 0.03 F1 of statistical signals, indicating that the information relevant to the source is largely captured by simple, interpretable properties of the denoising process. Because these distinctions are available from the generation a model already produces, they offer a practical basis for deciding when a system should ask, abstain, or answer.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.