acceptodds
Under review as a conference paper at ICLR 2027

Token Freezing in Diffusion Language Models Needs No Convergence Test

Abstract

Masked diffusion language models recompute every position at every denoising step, including tokens that are already committed and will not change. Freezing a committed token saves this compute by serving its cached keys and values, at the cost of staleness for the positions that still read them. The prevailing rule freezes a token once a convergence statistic of its output distribution, such as stepwise KL divergence, falls below a threshold. We show that this statistic cannot see the damage freezing does. Since a committed token is never rewritten, freezing affects the output only through the keys and values that other positions read. The statistic, however, measures the token's own output distribution. At a committed position, this distribution already places nearly all of its probability on the committed token, so it barely changes even when the keys and values do. To test whether the statistic's timing matters, we randomly shuffle its freeze times across tokens without increasing total compute. On every model we test, accuracy does not measurably change. We therefore drop per-token signals and freeze every token a fixed number of steps after commitment. We show that, among schedules that use no per-token information, this uniform delay minimizes the expected motion that frozen keys and values have left. Across five open diffusion language models, the delay matches or exceeds the prevailing rule at equal compute and improves on it by two to three points on held-out MATH. Rules built on other per-token signals, including the motion of the keys and values themselves, do not beat it either. The delay's savings carry over to wall-clock time, with 1.3–1.4× speedups at batch size 8 and above.

Primary area:
generative models
Keywords:
diffusion language models, token freezing, inference optimization, masked diffusion, convergence criteria, discrete diffusion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_4Pu3OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper re-examines permanent token freezing in masked diffusion language models. A committed token's identity no longer changes, but its keys and values can continue changing and affect predictions at other positions. The authors argue that monitoring the committed position's own posterior is poorly aligned with this outgoing influence, particularly when that posterior is saturated. They propose freezing every token after a fixed delay following commitment instead of using a token-specific convergence test.

Validation includes a softmax-saturation bound, measurements of token-wise non-contraction, shuffled-wait controls, comparisons with alternative signals, and accuracy-compute evaluations across five models. Two pre-specified held-out MATH-1500 comparisons report gains of 2.80 and 2.27 percentage points over the KL criterion at approximately matched work. A separate compacted executor realizes roughly 1.4x speedups at larger batch sizes using replayed schedules. The theoretical optimality result is restricted to uninformed schedules and a convex mean-exposure condition.

Critical strengths
  1. A consequential diagnosis supported by targeted controls. Sections 4.1-4.3 distinguish the statistic being monitored from the representations that propagate freezing error. Saturation analysis, alternative signals, and shuffled waits provide complementary evidence. This supports acceptance because the paper explains a failure mode of an existing design choice rather than merely reporting a better threshold.

  2. A simple replacement with held-out empirical support. Table 3 reports positive MATH-1500 differences with paired confidence intervals excluding zero, following development on a disjoint split. The broader model and schedule evaluations show that a fixed delay is a useful competitive baseline. A substantially simpler policy with this evidence is a meaningful contribution even without general optimality.

Critical weaknesses
  1. Runtime and accuracy do not come from the same live execution path. Appendix G explicitly states that timing replays freeze schedules and that no accuracy number comes from the compacted executor because numerical differences can alter decoding. The timing demonstrates achievable execution savings for those schedules, but does not establish that the accelerated implementation realizes the reported quality. This matters to the practical acceleration claim and could lower my assessment if live decoding loses the quality advantage.

  2. Some superiority and quality-preservation language is stronger than the comparisons establish. The held-out delay arms use somewhat more computation than their criterion counterparts, and the main matching rule permits a three-point skipped-work gap. Table 15 reports no statistically separated differences under the alternative rule requiring the delay to skip at least as much work. Also, intervals containing zero after shuffling or across delay settings do not establish equivalence or non-inferiority. The evidence supports a competitive simple rule more strongly than a universal superiority or no-loss claim.

Questions for authors
  1. Clarification: Are any complete generations evaluated with the compacted executor running the policy live? If available, please provide their accuracy, realized skipped work, and speed on the same questions. This would connect the practical speed claim to task quality.

  2. Evidence needed for assessment: Can you quantify the held-out advantage at more closely matched work, and give a bound on the plausible accuracy loss for the claimed safe-delay range? The concern is whether the current intervals and budget differences support the strength of the conclusion, not a request to establish a new general theorem.

Reviewer_e46UOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper revisits token freezing in masked diffusion kanguge models, showing that existing criterion (i.e., step-wise KL) poorly reflects the KV-state fluctuations that affect other positions and that its assumed per-token contraction does not hold empirically. Motivated by these rigorous analysis, they propose a fixed-time post-commitment delay, Uniform Delay, that achieves comparable or better accuracy at similar compute across some models, with practical inference speedups.

Critical strengths

S1: Careful observation and well-motivated analysis

The paper is grounded in careful empirical observation and rigorous analysis (the best point for me); the authors examine existing convergence criteria from multiple angles. I particularly appreciate how the paper goes beyond aggregate statistics to uncover overlooked behavior and designs targeted experiments to confirm its hypotheses.

S2: Formulation backed by thorough validation

Proposed method, Uniform Delay, is a simple and well-motivated approach that follows naturally from the analysis, stated above. The authors support their proposal with a theoretical analysis of exposure, direct measurements of KV staleness, and FLOPs/wall-clock benchmarks. These provided careful empirical validations make the work a solid and practically relevant contribution.

Critical weaknesses

W1: Positioning relative to Oba et al. ICLR26

This paper cites Oba et al. as a main related work and calls into question the necessity and design of their tests. They choised the step-wise KL for a test criterion in terms of theoretical motivation since, under certain assumptions, it provides a bound on "the deviation of the terminal predictive distribution at a frozen position." In contrast, this paper focuses on a different source of error (i.e., the staleness of cached key-value states and its effect on other positions) and derives a bound on the resulting per-step logit deviation. However, neither analysis directly guarantees the quality of the final generated sequence; in particular, the proposed waiting time k comes with no such guarantee.

I therefore view the two works as offering distinct, potentially complementary theoretical contributions. The paper would benefit from a more balanced discussion that acknowledges the motivation behind the existing work's design and distinguishes the objectives, assumptions, and limitations of the two approaches.

W2: Generality of the empirical conclusions

The advantage of Uniform Delay varies across models and evaluation settings. In Table 4, only three of the ten comparisons show statistically significant improvements, while the results for LLaDA-MoE slightly favor the criterion arm. The improvements on held-out set are convincing, yet the comparisons against adaptive freezing policies are limited to a relatively small set of model–task settings.

These results establish the proposal (Uniform Delay) as a simple and competitive alternative, but do not necessarily support the broader claim that "per-token convergence tests are generally unnecessary." I would encourage the authors to distinguish more carefully between what the experiments demonstrate and the extent to which the conclusions can be generalized.

Questions for authors

Could the authors clarify how the held-out MATH-1500 set was constructed? The authors stated that it is disjoint from MATH-500, yet it is unclear how the problems were selected and from which pool. Providing the selection rules or scripts would help ensure reproducibility.

Reviewer_sEDDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the problem of freezing the Key–Value (KV) cache of already-determined tokens during inference in Masked Diffusion Language Models (MDLMs). The authors question the rationality of existing methods that use the KL divergence between output distributions at adjacent denoising steps to determine whether a token has converged. They point out that the output distributions of already-determined tokens are usually close to saturation, so their KL changes can hardly reflect changes in KV representations accurately, whereas the latter may affect predictions at other positions through attention. Based on this observation, the authors propose a Uniform Delay strategy, in which each token continues to be updated for a fixed number of k steps after being determined, and its KV representation is then permanently frozen. Experiments on five open-source dLLMs show that, under comparable computational cost, this strategy generally outperforms KL-threshold-based freezing methods.

Critical strengths
  1. The research problem is of practical value. Iterative dLLMs involve substantial redundant computation, and studying when to stop updating the KV representations of already-determined tokens is directly relevant to improving inference efficiency.

  2. The analysis of existing KL convergence criteria is relatively convincing. The paper identifies the discrepancy between saturation of output probabilities and changes in KV representations, and provides support through theoretical analysis, token-level trajectories, and experiments such as reordering freezing times.

Critical weaknesses

W1. The methodological novelty and its relationship to existing KV cache management work need to be further clarified. This paper mainly focuses on permanent KV freezing after tokens are determined. Although the focus is relatively specific, the idea of fixed waiting followed by caching is related to existing dLLM inference acceleration work. For example, dKV-Cache [1] has studied delayed caching and token representation dynamics, dLLM-Cache [2] uses feature similarity for adaptive caching, and Fast-dLLM [3] combines KV cache with parallel decoding for acceleration. Although the caching mechanisms of these methods are not exactly the same as permanent freezing, the authors are advised to further discuss their connections and, where possible, compare inference efficiency under the same accuracy and computational budget. This would help clarify the independent contribution of this paper to the field of dLLM KV cache management.

W2. The practical speedup gains and scope of applicability remain somewhat limited. The paper reports about 1.3–1.4× speedup at relatively large batch sizes, but no practical speed gain is observed at batch size 1, indicating that the efficiency of the method may depend on specific execution environments. The applicability of the method to samplers that allow remasking or token correction also remains to be studied. The authors are advised to further discuss these limitations and provide evaluations closer to real deployment conditions.

References [1] Ma, X., Yu, R., Fang, G., & Wang, X. (2025). dKV-Cache: The Cache for Diffusion Language Models. NeurIPS 2025.

[2] Liu, Z., et al. (2026). dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching. ICML 2026.

[3] Wu, C., et al. (2026). Fast-dLLM: Training-free Acceleration of Diffusion LLM by Enabling KV Cache and Parallel Decoding. ICLR 2026.

Questions for authors

See Weakness

Comments on the paper

1 comment

Dear reviewers,

The author-reviewer discussion period is ending soon. If you have not done so yet, please read the authors’ rebuttal and reply to it. Tell the authors whether it addresses your concerns, and update your score if your assessment has changed.

The ICLR 2027 Reviewer Guidelines list « discussing with the authors to clarify and improve the paper » as one of the core duties you accepted as a reviewer. With the high number of submissions this year, your input matters more than ever. Recall that reviewers that do not fulfill their duties may also face penalties on their own submissions.

Thank you for your time and help.

Best regards,


Your Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.