acceptodds
Withdrawn from ICLR 2027

Learning When to Commit: Token Scores, Stopping Rules, and Stability in Diffusion Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Parallel diffusion decoding trades task accuracy for fewer backbone evaluations, and the accuracy retained depends on both token scoring and the rule that converts scores into commitments. We introduce a -parameter predictor of token stability, defined as agreement with a one-token-per-step decoding reference. Confidence determines candidate order, while a cumulative product of predicted stability scores controls how many tokens are committed, without additional backbone evaluations. Heads trained on at most problems are reused across tasks without retraining, with low measured inference overhead. Across three backbones and three benchmarks, the decoder has lower relative accuracy loss than its paired comparator in 71% of the measured comparisons at approximately matched mean cost. Two observations clarify what the learned score contributes. First, controlled comparisons show that a token score's observed accuracy advantage can depend on the stopping rule and decoding budget. Second, local stability and task accuracy differ. At approximately matched mean commitment counts, learned stability reduces reference disagreement relative to raw confidence and margin, but not significantly relative to a learned confidence-only control. Validation-selected evaluations and single-decision interventions do not establish consistent gains in final task accuracy. Our contribution combines lightweight, transferable stability-guided decoding with a controlled analysis of how token scores and stopping rules jointly shape the accuracy–cost trade-off.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion language models, discrete diffusion LLM, parallel decoding, inference, acceleration, token stability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nRrDOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the trade-off between efficiency and accuracy in parallel decoding for dLLMs. It trains a small MLP to predict whether each candidate token is "stable". The experiments also analyze token scoring, stopping rules, local stability and final task accuracy in dLLMs.

Critical strengths

None

Critical weaknesses

W1. The presentation quality of the paper is poor. Many of the figures look like AI-generated framework diagrams. The appendix reports a lot of content that is unrelated to the paper's topic and core contribution, so the overall structure reads like a technical report. This makes the paper difficult to read.

W2. The writing of this paper lacks logical coherence. Overall, it does not meet ICLR's style and standards: it has neither a coherent story nor a clear motivation.

W3. The novelty of this work is limited. The performance gains rely mainly on the confidence-ordered cumulative-product stopping rule, which has the same structure as EB-Sampler. Even the authors acknowledge, "We do not claim this rule as new." The prediction head requires additional training to estimate the "stability score". Yet in the experiments analyzing the score, it shows no clear advantage in task accuracy, and in some settings it even falls behind the training-free margin score (e.g., 64.67% vs. 65.13% at 30.8 NFE on Dream GSM8K; Table 13).

W4. The experimental reporting is insufficient and does not show a clear advantage. All experiments use a fixed generation length of 256 tokens (and mostly block size 32), so more settings should be explored. Training the prediction head involves training–test overlap: some of the training problems also appear in the evaluation sets (e.g., H_L is trained on HumanEval problems and evaluated on HumanEval). In addition, the head uses 13 input features, yet its local reference agreement differs from a confidence-only version by at most 0.17 pp, so the need for this feature design is not evident. Finally, EB-Sampler, the closest prior work, is not included as a baseline. The appendix only contains an in-house entropy-budget control in two LLaDA-1.5 blocks, and it is not discussed.

W5. The paper reports a large number of tables but seriously lacks qualitative experiments, interpretability analysis, case studies, failure cases and a thorough efficiency analysis.

W6. Does the interaction between token scores and stopping rules claimed in this paper actually exist? This needs more experimental evidence, and it should be explained more clearly. In addition, the main text concludes from a non-significant result that "being locally more accurate does not mean being more accurate in the end, so the two are different objectives". Both points show that the rigor of the work at its current stage is very poor.

Questions for authors

This paper shows extensive signs of AI generation, and its writing, experimental design and method design are all poor. I strongly recommend that the authors thoroughly revise and improve this work. Submitting work in this state places an unnecessary burden on the review process.

The layout of the paper is poor. Readers have to search through both the main text and the appendices themselves to find the experimental evidence that the main claims rely on.

Reviewer_5nwoOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper trains a 5,121-parameter predictor of agreement with a one-token-per-step diffusion decoding reference. Candidates are ordered by confidence, and a product of predicted stability scores controls how many are committed. Experiments on three backbones and three benchmarks show cross-task reuse and competitive accuracy–cost trade-offs. Controlled comparisons examine score–stopping interactions and find that better local reference agreement does not consistently yield higher task accuracy.

Critical strengths
  • The decoder is inexpensive and transferable. With unchanged weights, normalization, and cutoff, GSM8K-trained heads retain 96.3–98.7% of sequential accuracy on MBPP at 5.42–9.21× NFE-based speedup across the three backbones. The measured predictor overhead is only 0.4–1.4% of decoding time on LLaDA, supporting its practical appeal.

  • The separation of token scores from stopping rules produces useful controlled comparisons. The common-state diagnostics, learned confidence-only control, and single-decision rollouts help distinguish reference agreement from task correctness. The paper reports adverse results and unresolved comparisons clearly, which makes the limits of stability-based decoding easier to assess.

Critical weaknesses
  • The added value of the full stability predictor remains unclear. Its local advantage over the learned confidence-only control is at most 0.17 percentage points, with no comparison surviving correction (Table 17). In the validation-selected task evaluations, gains over raw-confidence product stopping are inconsistent; the strongest positive estimate is +4.81 points on LLaDA-1.5 MBPP, with Holm p = 0.057 (Table 22). Tables 6 and 16 provide feature ablations, but the confidence-only control is absent from the validation-selected evaluation. These results leave open whether the extra features offer a useful improvement beyond learning a transformation of confidence.

  • The headline 71% win rate gives limited guidance about when to prefer this decoder. It pools unequally covered sweeps, reuses configurations, and excludes the raw-confidence and margin product controls from that summary (Appendix C.2). Among comparisons where the comparator loses less than 5% of sequential accuracy, the decoder wins 28 of 61. The matching-window sensitivity analysis is helpful, but a comparison of the accuracy–cost frontiers within each setting would better establish the practical improvement.

Questions for authors
  • How does the full head compare with the learned confidence-only control under the validation-selected protocol of Tables 20–22, with each cutoff chosen by calibration NFE alone? LLaDA-1.5 MBPP, where the strongest positive result appears, would be a particularly informative check.

  • Can the existing runs provide per-setting accuracy–NFE frontiers including raw-confidence product and margin product, with head-training problems excluded where applicable? Which cost ranges show a clear advantage for the proposed decoder over these controls and the external comparators?

Reviewer_x43uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper speeds up DLLM decoding by training an extra lightweight stability predictor to control parallel token unmasking. It compares against six baselines and achieves lower relative accuracy loss in most cases across different backbone models and tasks at comparable NFE. The paper also studies how token scoring interacts with the stopping rule and finds that better local stability does not consistently improve end-to-end task accuracy.

Critical strengths

The method is lightweight and practical. The stability predictor has only 5,121 parameters, adds no backbone evaluations, and has low measured inference overhead. It can also be reused across tasks without retraining. This makes the approach potentially useful in practice rather than only providing a theoretical speedup.

Critical weaknesses
  1. The threshold and cutoff selection may be task-specific. The paper uses different values across tasks, models, and target NFE, and selecting them requires a separate calibration sweep. This could limit the practicality of the method for a new task.
  2. The end-to-end runtime comparison across methods is missing. The main experiments compare methods using NFE, but fewer backbone evaluations do not necessarily mean faster decoding because different methods have different computational overhead.
  3. The claimed “gap between local stability and task accuracy” is not very novel. Parallel token unmasking changes the decoding context and can lead to a different generation trajectory, so better local agreement does not necessarily produce better final task accuracy. This seems like an expected property of parallel DLLM decoding rather than a new finding, and the paper should better explain what new insight this experiment provides.
  4. The reported 5× or higher speedup in Table 1 uses one-token-per-step decoding as the reference. This is not a strong practical baseline because DLLMs are already commonly decoded with multiple tokens in parallel. Therefore, the reported speedup does not clearly show how effective the proposed method is compared with existing parallel decoding approaches. The paper should emphasize speedup over competitive parallel decoders rather than mainly over single-token decoding.
Questions for authors
  1. Does the method remain faster than all baselines when comparing end-to-end runtime rather than NFE?
  2. What is the speedup over existing or naive parallel DLLM decoders, rather than one-token-per-step decoding?

Minor suggestion:

  1. NFE is introduced in line 86, but its full name is never stated. This may be slightly confusing for readers unfamiliar with this area.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.