acceptodds
Withdrawn from ICLR 2027

HaloBlock: Faster and Better Decoding for Diffusion Large Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion large language models (dLLMs) have attracted increasing research attention due to their bidirectional attention and arbitrary order decoding. However, there are two issues in dLLMs inference: the decoding difficulty is dominated by the tokens on the right side of the block, and the block-wise decoding inherently suffers from boundary contextual discontinuity. To tackle these limitations, we propose HaloBlock, a difficulty‑aware adaptive halo mechanism for dLLM inference. We first leverage bidirectional cumulative entropy to characterize block decoding difficulty. Then, based on the bidirectional decoding difficulty, we adaptively allocate bidirectional halo regions on both sides of each block to mitigate boundary contextual discontinuity. The difficulty‑aware design of HaloBlock smooths cross‑block context propagation and lowers decoding difficulty for hard tokens, yielding better generation quality and faster convergence. Evaluated on LLaDA‑8B‑Instruct, Dream‑7B‑Instruct and LLaDA‑1.5 across mathematical reasoning and code generation benchmarks, HaloBlock demonstrates consistent accuracy improvements and higher inference efficiency. Source codes will be publicly available.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Diffusion Large Language Models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_WDamOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes HaloBlock, a training-free decoding method for diffusion large language models (dLLMs). It identifies the left-dominance effect and contextual discontinuity at block boundaries of the block-decoding mechanism of dLLMs, and adaptively adds bidirectional halo regions based on token entropy and positional coefficients. Evaluation on LLaDA and Dream models show improved generation quality and inference efficiency in different settings.

Critical strengths
  1. The proposed method is easy to adapt, and the evaluation shows that HaloBlock can provide advantage over several baselines under various scenarios and settings, while only one hyper-parameter requires tuning.
Critical weaknesses
  1. The paper's background and story line are not solid, examples:

    • Block-wise decoding was not first introduced by [1], works including [2, 3] had already introduced it before.
    • Decoding mechanisms like Fast-dLLM are not the only decoding and caching mechanisms of dLLMs; however, the paper builds its claim based on Fast-dLLM's asynchronous KV cache updates.
    • The claim that contextual discontinuity is caused by asynchronous KV cache updates should be verified. In Fig. 2, the fluctuation reduction of HaloBlock is not obvious.
  2. The implementation and evaluation scope are limited. While the paper claims to address one structural limitation of block-wise decoding in diffusion language models, the evaluation only uses models that were trained as full-sequence dLLMs. The authors should support and evaluate their method on block diffusion language models like SDAR, LLaDA2, DiffusionGemma, or even the smaller BD3LM paper they bring up at the beginning. Moreover, baselines like DCD [4], which directly tries to address the block boundary problem, should be compared.

References:

[1] Arriola et al., Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models, ICLR 2025.

[2] Han et al., SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Control, ACL 2023.

[3] Sahoo et al., Simple and Effective Masked Diffusion Language Models, NeurIPS 2024.

[4] Shu et al., Deferred Commitment Decoding for Diffusion Language Models with Confidence-Aware Sliding Windows, arXiv 2026.

Questions for authors
  1. Why are linear positional coefficients used in the bidirectional difficulty estimator? Did the authors evaluate alternative weighting functions?

  2. Can HaloBlock be implemented on top of real block diffusion language models? How would it impact the KV cache integrity of the previous block?

Reviewer_oo6kOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HaloBlock, a training-free decoding method for diffusion large language models (dLLMs) that aims to improve both generation quality and inference efficiency. The authors identify two limitations of existing block-wise decoding: the left-dominance effect, where tokens near the right boundary of a block tend to be decoded later, and boundary contextual discontinuity, attributed to asynchronous KV-cache updates across blocks. To address these issues, HaloBlock introduces adaptive halo regions on both sides of each decoding block. The halo lengths are determined by bidirectional cumulative token entropy and positional coefficients. By refreshing the KV states within these extended regions, the method aims to improve contextual information propagation and reduce decoding difficulty. Experiments on LLaDA-8B-Instruct, Dream-7B-Instruct, and LLaDA-1.5 demonstrate improvements in accuracy and inference efficiency across mathematical reasoning and code generation benchmarks.

Critical strengths
  1. Practical significance. The paper addresses an important problem in dLLM inference, namely the trade-off between generation quality and decoding efficiency. Improving the efficiency of block-wise decoding without additional training is practically valuable.

  2. Simple and intuitive design. HaloBlock introduces a relatively straightforward mechanism for extending the effective decoding region and updating KV states. The method does not require additional training and can be integrated into existing inference frameworks.

  3. Comprehensive empirical evaluation. The experiments cover three representative dLLMs and four benchmarks, including mathematical reasoning and code generation. The results show promising improvements in both accuracy and throughput, particularly on Dream-7B-Instruct.

Critical weaknesses
  1. Unclear definition of NFE. The token-wise NFE in Figure 1 appears inconsistent with its conventional definition [1], which measures the number of model forward evaluations required to generate a sequence or block, rather than the iteration at which an individual token is unmasked. This terminology may confuse readers.

  2. Limited novelty of the left-dominance effect. Similar observations have already been reported in [2, 3]. In particular, d²Cache [3] identifies a more general phenomenon that dLLMs tend to decode masked tokens near known tokens, which already explains the argument in Lines 040–042 and 177–181 that right-side tokens are harder to decode due to insufficient contextual information.

  3. Limited applicability to recent dLLM architectures. Recent developments have increasingly explored native block-diffusion architectures and advanced multi-block decoding approaches, such as Fast-dLLM v2 and d3LLM. However, HaloBlock is evaluated only on full-attention diffusion models such as LLaDA and Dream, limiting its demonstrated practical applicability to these newer paradigms.

  4. Potentially misleading interpretation of decoding iterations. Line 187 states that right-side tokens require more iterations to converge. However, if Figure 1 uses one-token-per-step decoding, the total number of iterations is fixed by the block size, and the figure merely reflects the unmasking order. If the authors intend to claim that right-side tokens require more iterations under parallel decoding, supporting experiments should be provided.

Questions for authors
  1. Consequences of boundary contextual discontinuity. Although the authors observe confidence and entropy discontinuities at block boundaries, there is no direct evidence showing their impact on generation quality or decoding efficiency. Could the authors provide further analysis?

  2. Hardware-efficient halo allocation. In my experience, aligning the number of tokens processed per forward pass to power-of-two sizes or hardware-friendly multiples can improve GPU throughput. Could the authors investigate whether such alignment of dynamically allocated halo lengths could further improve inference efficiency?

Reviewer_gF9GOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HaloBlock, a training-free decoding method for diffusion large language models (dLLMs). The authors identify two limitations of block-wise decoding: position-dependent decoding difficulty within blocks and contextual discontinuity at block boundaries. To address these issues, HaloBlock adaptively allocates bidirectional halo regions based on token entropy and positional weights, enabling additional KV-cache updates across block boundaries. Experiments on multiple dLLMs and benchmarks demonstrate improvements in generation accuracy and inference efficiency.

Critical strengths
  1. Practical and lightweight design. HaloBlock introduces a simple, training-free mechanism that can be integrated into existing block-wise dLLM decoding frameworks without modifying the underlying models. Its compatibility with existing decoding strategies makes it practically useful.

  2. Consistent empirical improvements. The experiments demonstrate improvements in generation accuracy and inference efficiency across multiple dLLMs and benchmarks. Although the gains are modest in some settings, the consistency of the results provides evidence for the practical effectiveness and general applicability of the approach.

Critical weaknesses
  1. Limited novelty. The main challenges addressed in this paper have been explored in prior work. The proposed method mainly combines existing ideas with a relatively simple entropy-based heuristic, making the contribution somewhat incremental.

  2. Insufficient validation of adaptive allocation. It remains unclear whether the improvements primarily come from the adaptive allocation strategy or simply introducing halo regions. Some static halo configurations achieve comparable or better results, suggesting that more targeted ablations are needed.

Questions for authors

None

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.