acceptodds
Under review as a conference paper at ICLR 2027

GRAFT: RESTORING THE MISSING SAMPLING INTERFACE IN CONTINUOUS DIFFUSION LANGUAGE MODELS

Abstract

Continuous diffusion language models generate a whole sequence in parallel, never drawing a single word at any intermediate step; only the final step turns embeddings into text. Models such as ELF now report autoregressive-level quality at a fraction of the sampling steps taken by discrete-token diffusion models, and this design principle—no step-wise discretization—is what their own ablations credit for the gain. We show that this same principle carries a hidden cost: a collapse of the lexical long tail. Measured against held-out human text, ELF’s output matches the human rate on common words almost exactly, yet its rarest words appear at only 0.30× the human rate, a gap that does not close from 105M to 652M parameters and that no sampler setting these models expose can reach. A same-size GPT-2, in contrast, spans zero to above the human rate by its decoding rule alone. We trace the difference to what continuous diffusion models lack: with no step that ever draws a discrete word, every step is instead driven by an average over the candidates still in play, and averaging a rare candidate together with the common word competing for its slot lands on the common word. An autoregressive model avoids this because its decoding rule samples one candidate rather than averaging over all of them, so the problem never arises there. We propose GRAFT, a training-free method that grafts this missing choice back onto a frozen model: while the model is still undecided, GRAFT commits the rare words the average would otherwise discard, then regenerates the sequence from scratch with those commitments held in place. Across three ELF scales and on LangFlow, an independently developed continuous dLLM, GRAFT restores the rare-word share of generated text from 0.42–0.77× the human rate to 0.82–1.10×, a relative gain of 34–109%, and moves the entire word-frequency profile closer to human on every model tested, by 2.7× on ELF-B and 12× on LangFlow.

Primary area:
generative models
Keywords:
diffusion language models, continuous diffusion, flow matching, decoding, sampling, lexical diversity, long-tail vocabulary, training-free inference, text generation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_sFmXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper identifies an interesting phenomenon exhibited by several representative continuous diffusion language models: the coverage of rare words in generated text is lower than that in human-written text. The paper hypothesizes that, because the denoising head is trained to predict the mean of possible clean embeddings, the embeddings predicted during generation tend to bias toward more common words, with rare words being too insignificant to be recovered by the final decoding step. The paper then quantitatively verifies the hypothesis and proposes a sampling method to mitigate the issue. The proposed sampling method detects the potentially missing rare words at an early stage of generation, commits to the most likely rare word, and then reruns the generation from scratch with the committed rare word being fixed. The results show that the proposed sampling method can improve the rare-word usage

Critical strengths
  • The findings of the paper are an interesting phenomenon of several representative continuous diffusion language models. This paper quantitatively verifies the proposed hypothesis and proposes a sampling algorithm that effectively mitigates this issue.
Critical weaknesses
  • The proposed sampling method depends on many heuristics, e.g., the read time, the threshold to determine the strongest rare candidate, the number of committed rare words, and it seems many of them can not be determined without first checking the actual predicted distribution of the model. Is there a more principled way to determine these design options? Furthermore, they may also depend on other hyperparameters such as sampling steps and sequence length, which seem to be fixed in the experiments
  • The paper only reports metrics regarding word frequencies, but people would be naturally interested in whether the early committed rare words might affect the quality of final samples. Moreover, since the re-generation inevitably incurs overhead, it would be better to more comprehensively profile the quality-efficiency trade-off
Questions for authors

Please see Weaknesses

Reviewer_9c9JOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

In this paper, the authors identify a lexical long-tail collapse in continuous diffusion language models. Observing that this lack of rare word problem persists across model scales and sampler settings, the authors conduct a systematic analysis on the reason behind this phenomenon. They discover that rare words are lost when a most common competing word accumulates sufficient probability in the denosing process, and that the rare words have already lost before the final discretizinng decoding module steps in. The authors then propose GRAFT, a training-free method that restores the missing lexical tail. Across three ELF models and one LangFlow model, GRAFT consistently raises the rare-word share to a level that is comparable with human language.

Critical strengths
  • The paper is generally well written and easy to follow.
  • The authors conducted sufficient experiments, including a systematic ablation study.
Critical weaknesses
  • GRAFT has limited practical significance. The main advantage of continuous diffusion language models is their fast inference speed. GRAFT, however, four times the total NFE in the sampling process. The computational overhead will make it prohibitive in any real-world application scenarios.
  • The numbers reported in the paper are not consistent. In Table 1, human's non-word ratio is 0.66%. In Table 2, however, the same value becomes 1.14%. The caption of Figure 1 states that GRAFT raises the long-tail ratio of ELF and LangFlow to 1.15× and 1.21× of the human rate. In Table 2, however, the same values become 1.10x and 1.25x. In the abstract and introduction sections, the authors claim that changing the model sizes for ELF does not change the long-tail ratio. However, in Table 2, this number goes from 5.01% to 7.65%. There are also many similar inconsistencies in this paper, and this makes me seriously doubt the soundness of the reported values.
  • There are no text quality metrics reported in the paper. Hence, it is unclear if GRAFT will decrease the sample quality given that rare words are deliberately injected.
  • The final metric is used in selecting words to commit in the GRAFT algorithm, and the final outcome is evaluated on the same set of metrics. This is concerning because it is almost certain that committing rare words can increase the rare word ratio. GRAFT forcefully injects 32 rare words into a 1024 token sequence, and the final rare rate increases by around 4%, which is basically the same value as 32/1024. This makes me doubt besides injecting rare words, what GRAFT brings to the final outcome additionally.
Questions for authors
  • Can the authors elaborate on the practical usefulness of GRAFT?
  • Can the authors explain why there are so many experimental result inconsistencies everywhere in this paper?
  • Can the authors report sample quality metrics?
  • Can the authors elaborate if it is reasonable to use the same metric to design and evaluate the algorithm? Can the authors explain why almost all the rare word ratio gain comes from the artificially injected words?
Reviewer_TTDmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper shows that continuous diffusion LMs (ELF, LangFlow) under-generate rare words. On OpenWebText, ELF’s rarest Zipf bands appear at 0.30-0.32× the human rate, and this does not improve from 105M to 652M parameters or with sampler tuning. The authors blame the MSE-trained denoising head. It predicts an average over candidates, which pulls rare words toward the common competitor.

Critical strengths
  • Clear, well-measured problem. The tail deficit is measured against a same-distribution human baseline (Table 1). The paper also shows that gen-PPL and unigram entropy miss it. This is a real blind spot in how continuous dLLMs are evaluated.
  • Good controls and mechanism probes. The argmax-commit control is null (tail 5.01→5.31, bootstrap interval includes zero) and the equal-budget best-of-4 baseline falls far short (0.50× vs 0.82×). The rule ablation (Table 3) separates where the gain comes from and what the repairs do. Together these make it credible that the gain comes from which word is committed.
Critical weaknesses
  • Circular and narrow evaluation. GRAFT selects words by Zipf rarity, and the main metric counts Zipf rarity, so the gain is close to true by construction. Evidence that the committed words are good is thin:
  1. Only a GPT-2-large aptness check, with no human evaluation, MAUVE, or downstream quality metric.
  2. Gen-PPL is dropped, and non-words stay at 2.6-3.8% vs the 1.14% human rate. They even rise on ELF-M (1.97→3.11) and LangFlow (3.28→3.58).
  3. LangFlow overshoots at 1.25×, and its dose was tuned on JSD using the same kind of metric.
  4. The setting is unconditional OpenWebText only, with n=200 samples. On XSum, GRAFT has no effect on unsupported entities.
  • Mechanism is correlational, and baselines are weak. Finding 3 shows only that rare words lose when a more probable competitor exists, which any model would show. There is no causal test, such as an intervention on the denoiser, a training-time fix, or a formal analysis. “No sampler knob can reach it” is tested against a limited set of knobs. Missing comparisons include logit or rare-word biasing, noise or temperature on the embeddings, and other continuous-sampling methods (e.g., the cited marginal-conditioned bridges). The only baselines are stock and best-of-4. The probes also use small subsets (96-200 documents), and several are on ELF-B only.

  • Heuristic, costly, and narrow. The method has many hand-set pieces (q_min=0.08, top-512, R=4, m=8, three repair rules, 13-language gate, λ=0.5 repetition correction) plus a per-model dose for LangFlow. It costs 2.2× wall-clock and is shown on only two closely related models. The attestation gate uses held-out human text, so the paper should show it does not leak the reference distribution.

Questions for authors
  • Clarification: Can you report a metric that does not use Zipf rarity, such as human evaluation of fluency and appropriateness, or MAUVE? Does GRAFT hurt coherence or factuality?
  • Does a simple baseline match GRAFT at equal compute (rare-word logit bias at the read time, or committing random attested rare words)?
  • Clarification: Is the attestation gate’s held-out slice disjoint from the human reference used for the metrics?
  • Can you test the “averaging” cause directly, for example with a model trained with a sampling-style loss, or with synthetic data where the conditional distribution is known?
  • Future work: Does this extend to conditional tasks and other continuous dLLMs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.