acceptodds
Under review as a conference paper at ICLR 2027

What Gets Committed First Shapes the Answer in Masked Diffusion Language Models

Abstract

Masked diffusion language models such as LLaDA and Dream generate text in parallel and in any order, and are now applied to retrieval-augmented question answering. At each step the model predicts many masked positions and the decoder selects which prediction to commit; the commitment is irreversible and becomes context for all later predictions. We ask whether the decoder's choice of which prediction to commit can redirect the final answer while the underlying model state stays fixed. Final accuracy cannot resolve this, since decodings that differ in one commitment differ in every later state. We therefore branch decodings from identical states and vary only the first non-padding commitment. This single change alters 27.3% of final answers. A dominant failure pattern arises when the decoder commits an end-of-turn (EOT) token before any answer content, while answer-supporting predictions are already present elsewhere in the canvas. Token identity governs the effect: committing end-of-sequence (EOS) yields 38.7% exact match at either tested position, and committing EOT yields 0%. Following this finding, EndFirst reweights EOS against non-EOS candidates once, at the earliest step where any uniform EOS reweighting can alter the trajectory, and returns control to the original decoder. With no extra forward pass, EndFirst at 32 passes outperforms a 64-pass retrieval-guided decoder on three QA benchmarks and improves that decoder by 5.3 exact-match points on average across eight LLaDA model-task settings. These results identify commitment selection as a distinct and directly controllable source of error in masked diffusion decoding.

Primary area:
foundation or frontier models, including LLMs
Keywords:
masked diffusion language models, discrete diffusion, decoding, retrieval-augmented generation, question answering, inference-time intervention

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Yh6BOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies early commitment selection in masked diffusion LMs (mainly LLaDA, with some Dream results). Using shared-state branching at the first non-padding commitment, it reports that changing that single action can alter a substantial fraction of final answers on a diagnostic set, and highlights frequent early EOT commitments on short RAG canvases.

The proposed EndFirst method subtracts a uniform offset from non-EOS logits at that first step only, then resumes the base decoder with no extra forward pass. Headline numbers are short-answer RAG QA EM lifts on NQ / HotpotQA / TriviaQA / T-REx under a small canvas and pass budget.

Critical strengths

Shared-state branching is a sensible way to isolate commitment choice from prediction quality, and the EOT/EOS token-identity controls give a concrete failure mode on LLaDA’s short RAG setup.

Critical weaknesses
  1. Overstated Generality and Tokenizer-Specific Scope: The empirical narrative is narrowly tied to short-answer RAG tasks evaluated on a fixed 32-token canvas using LLaDA’s distinct EOT and EOS tokenization scheme. Gains fail to transfer to models with different commitment schedules, with Dream showing negligible movement in Table 1 (approximately 0 to 2 EM points). Broader evaluation regimes such as open-ended generation, multi-sentence completion, and complex reasoning are entirely absent. Consequently, the claim of a foundational commitment bottleneck appears primarily to reflect an artifact of LLaDA’s premature terminal-token emission on short retrieval canvases.

  2. Marginal Algorithmic Novelty: The proposed method, EndFirst, reduces to a single static logit offset modulated by a scalar hyperparameter cc. While the underlying diagnostic analysis is insightful, the operational technique is incremental and overlaps significantly with existing deferred-commitment and boundary-control heuristics. As an algorithmic contribution to a top-tier venue, a tuned scalar penalty represents relatively thin methodological depth.

Questions for authors
  1. Outside short-answer RAG, does first-commitment lock-in still appear (e.g., open-ended instruction following, longer canvases)? If EOT-first rates collapse, how should EndFirst be reinterpreted?

  2. Can EndFirst be stated for models that use a single terminal/pad token (or Dream’s padding regime) without relying on EOT≠EOS?

  3. Please revise the abstract/intro to state the LLaDA-EOT/RAG scope as a primary limitation rather than implying a general commitment-selection advance.

  4. Have you tried combining EndFirst with remasking / revocable decoding, or learning cc / δ\delta instead of a fixed schedule?

1 comment

Thank you for the thorough experiments. While the intervention remains simple and strictly bounded to concise-answer tasks (as confirmed on GSM8K), the upfront scope revisions make the claims fair and well-supported, so I am raising my score by 1 point.

Good luck!

Reviewer_sQVJOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper investigates the detrimental effect of premature 'EOT' (end-of-turn) token commitments in masked diffusion language models. The authors find that when the 'EOT' token is committed before the first content token, answer accuracy almost drops to zero. Building on this, they propose EndFirst, a training-free intervention, that reweights 'EOS' against other tokens, and does not use any additional forward passes. On retrieval-augmented QA, it improves LLaDA accuracy by 12.2-26.6 points on the tested datasets.

Critical strengths

The proposed intervention is simple, cheap, and very effective on LLaDA: no training and no additional forward passes are required and yet it improves accuracy by 12.2-26.6 points on RAG. The authors provide comprehensive ablations to motivate and isolate the effect of their intervention.

Critical weaknesses
  1. Proposition 1 explains why intervention before t* is redundant. But the paper does not sufficiently motivate why intervening once at t* is sufficient. Intuitively, after the intervention, we’re in a similar situation as before t*: only EOS tokens are committed. Why does the decoder not try to commit EOT again, requiring another intervention? Since the whole method is build around this, the paper should give an intuition here.

  2. The evaluation setting is limited, restricted to short-answer QA up to 64 tokens. It remains unclear whether the intervention helps with longer-form generation like reasoning tasks. Since the improvements on Dream are very limited, it raises the question whether the intervention is specific to LLaDA models on short-form generation or a general contribution for masked diffusion language models.

Some issues with presentation:

  1. “token-only” and “position-only” (Section 5.3, Figure 5b) are only defined in Appendix F, and the main text does not refer to it.
  2. ARAM is not cited at its first appearance in Section 4. It’s later introduced and cited in Section 5.
  3. Hong et al. (2026) is cited as "Wide-in Wide-out". The actual title is "Wide-In, Narrow-Out: Revokable Decoding for Efficient and Effective DLLMs".
Questions for authors

Can you give an intuitive argument why a single intervention at t* is enough, as opposed to also intervening at later steps? After committing EOS at t*, how often does the decoder commit EOT before a content token, and how does this affect accuracy in these cases?

1 comment

Thank you for the detailed response and new experiments showing empirically that intervening once at t* is sufficient and thanks for the evaluation on a diffusion model from a different family.

While the empirical results support the fact that intervening once is sufficient, my original question was about the motivation for why. One explanation I can think of is that the model’s predictive distribution at t* is a mixture of two length modes (which could come from non-diverse post-training): Either a long reasoning-answer, or a compact few-token answer (nothing in-between). Committing EOT somewhere late in the canvas conditions the model on this long-answer mode, so it is forced to fill the remaining positions/hallucinate. Replacing this with an interior EOS pushes the model into the compact-answer mode, collapsing to a concise answer. This is motivation would also explain why the method only seems to work great on short-answer formats.

I think the EndFirst-∞ suggestion by Riyf is great, thanks for implementing it. The additional results convince me that the method is a valuable simple trick to improve concise-form answering. However, the gains seem to be limited to very-short-form generation. I have raised my score.

Reviewer_RiyfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies deterministic confidence-first decoding in masked diffusion LMs (LLaDA-8B-Instruct, LLaDA-1.5; Dream-7B as a contrast) on short-answer retrieval-augmented QA with a 32-token canvas. Branching from a saved state at t⋆t^\star, the first step whose top proposal is not EOS, the authors show that replacing only the action executed there changes 27.3% of final answers. They identify a common failure, "first-commitment lock-in": the decoder commits an end-of-turn (EOT) token near the end of the canvas before any content, and the answer that follows is long and fails exact match. The proposed fix, EndFirst, acts once at t⋆t^\star. It subtracts δ=c ∣M(t⋆)∣/L\delta = c\,|M(t^\star)|/L from all non-EOS logits, which raises the EOS odds uniformly, and then hands control back to the unchanged decoder at no extra forward-pass cost. Proposition 1 states that a uniform EOS up-weighting cannot change any action before t⋆t^\star. On LLaDA, EndFirst improves EM by 12.2–26.6 points over standard RAG at c=4c=4 and by 5.3 points on average on top of ARAM. Dream shows essentially no gain.

Critical strengths

A cheap, training-free intervention with non-negligeble gains on LLaDA for that exact task.

  • EndFirst changes one decoding step, adds no forward passes, and composes with ARAM.
  • On LLaDA it improves EM substantially: +12.2 to +26.6 at c=4c=4 over standard RAG, and +5.3 on average on top of ARAM.
  • The gains hold on a held-out confirmation split with paired intervals above zero.

This matters to my assessment because the practical effect is real and well measured, even if other aspects are in question.

Critical weaknesses

1. The fix of the problem and its parameterization are essentially not ablated, so the gain cannot be attributed to "re-selecting the first commitment." The paper finds a problem: boundary tokens that fix answer length are committed before any content. It then offers one fix: a uniform offset on all non-EOS logits, applied once at t⋆t^\star, with the schedule δ=c∣M∣/L\delta = c|M|/L. None of the obvious alternatives are evaluated or at least mentioned:

  • banning EOT until the first content token is written;
  • penalizing only EOT;
  • using a constant δ\delta instead of the ∣M∣/L|M|/L schedule or using a different schedule;
  • applying the reweighting at the first kk steps, or until content appears, or until nn EOS tokens appear;
  • the c→∞c \to \infty limit, i.e., directly forcing EOS at the argmax-EOS position.

EM keeps rising up to c=8c=8 (Tables 11–12), and c=8,16c=8, 16 are at or above c=4c=4 on calibration. This suggests the "dose" may simply approximate that limit. The only alternative studied, an every-step EOS bias tuned to the same answer length, matches or beats EndFirst at c=2c=2: +1.3, −0.2, +2.3 [1.0, 3.7], and +0.5 EM (Table 7). The fixed λ = 0.25 quoted in §5.3 is not a fair comparison, since the tuned values are 1.5–1.9.

This is consequential because, without these ablations, the evidence is equally consistent with "any reasonable early-termination or length control works for an EM-based task", which is trivial. That would reduce the method to a tuned EOS bias and greatly undercut the paper's main contribution.

2. The claims about mechanism and headline performance go beyond the evidence.

  • Mechanism. Frontier-1 decodes left to right and therefore can never commit EOT first, yet it matches the baseline: 19.4 vs 19.6 on 8B NQ (Table 11). This suggests early EOT is a symptom rather than the cause and that there are more to be investigated.
  • Position. The token-vs-position decomposition tests only two late positions (mean 28.2 and 20.2, n = 31). Moving EOT between them does shorten answers, from 16.35 to 11.74 words. So the claim that "token identity, not position, governs the effect" rests on a narrow test.
  • Length and EM. Committing EOS starts a run of padding writes, and answers fall to about 1.7 words. EM rewards this format, which is legitimate given the 1–10-word instruction. But it means the gain is better described as answer termination and formation than as a distinct commitment error.
  • Headline numbers. "Outperforms 64-pass ARAM" (abstract, §1) holds only at c=4c=4. At the stated default c=2c=2, EndFirst trails ARAM on every task.
  • Dose selection. The description is inconsistent: App. G says c=2c=2 was "not selected by any sweep" and then justifies it with a sweep criterion. It is also unclear why c=4c=4 was "selected" when c=8,16c=8, 16 were at least as good on calibration.
  • Generality. The method is effectively tested on one model family. Dream gains about zero, and at L=128 the gain is +0.2 / 0.0 (Table 14). This contradicts "gains carry across canvas lengths."

This is consequential because the title, abstract and contributions present a general mechanism for masked diffusion LMs. The supported claim is way narrower: a decoding fix that improves specifically short-answer EM for LLaDA at a 32-token canvas, and not much more.

Questions for authors

Requests whose answers could change my assessment:

  1. How does EndFirst compare with simple alternatives on the 1,000-question sets? These are: (a) a hard ban on EOT, or on terminal tokens beyond step-0 padding, before the first content token; (b) an EOT-only penalty; (c) a constant δ\delta instead of c∣M∣/Lc|M|/L; (d) forcing EOS at the argmax-EOS position at t⋆t^\star. If EndFirst clearly beats these, my concern in Weakness 1 is largely resolved. If they match it, the contribution is mainly a length/termination fix.
  2. Could you report the every-step EOS bias tuned to the answer lengths of c=4c=4 and c=8c=8, on all groups and tasks? So far this has been shown only for c=2c=2, plus HotpotQA at c=4c=4.
  3. What is EM when EOT is committed early (e.g., positions 2–6) in the position-only arm? This directly tests whether position matters.
  4. Could you report a correctness metric less tied to answer form (containment with the safeguards of App. G, or an LLM/human judge) for the baseline, EndFirst, and the tuned every-step bias?

Clarifications: 5. What exact rule selected c=4c=4 over c=8c=8 and c=16c=16 on the calibration set, and how does it square with App. G's statement that c=2c=2 was not chosen by any sweep? Was the 128-item diagnostic sample drawn from the evaluation sets? 6. How do you reconcile Frontier-1, which never commits EOT first, matching the baseline with EOT-first being the dominant failure? 7. Which subset does the T-REx statement "EndFirst at c=4c=4 leads ARAM by 3.9 and 6.4" (§5.1) refer to?

Suggestions (not affecting my score): explain why baseline EM rises sharply from L=32 to L=64/128, and evaluate on a non-LLaDA model family where the early-boundary pattern also occurs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.