- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes HiAda, a training-free, history-aware adaptive decoding framework for accelerating diffusion large language models (dLLMs). HiAda introduces an attention-guided active set that separates token prediction from KV-cache updates, an adaptive mask window that removes fixed-block decoding constraints, and a history-aware confidence mechanism that promotes early commitment of repeatedly predicted tokens. Periodic full KV-cache refreshes are employed to mitigate representation staleness. Experiments on LLaDA-8B-Instruct and Dream-7B-Base across four reasoning and code-generation benchmarks demonstrate up to 51.1× speedup over vanilla diffusion decoding and 2.01× over Fast-dLLM, with task-dependent accuracy–efficiency trade-offs.
Critical strengths
- Significant inference acceleration. HiAda demonstrates substantial throughput improvements over vanilla diffusion decoding and Fast-dLLM across multiple benchmarks and generation lengths. These results suggest that the proposed framework can effectively reduce redundant computation in dLLM inference.
- A unified and training-free inference framework. The proposed method jointly coordinates token selection, KV-cache updates, and parallel decoding without additional model training. The integration of historical attention and prediction signals provides a practically motivated approach to adaptive computation.
- Comprehensive component-level analysis. The paper provides incremental ablation studies, hyperparameter sensitivity analyses, and computational-complexity derivations, offering useful insights into the contributions and trade-offs of the proposed components.
Critical weaknesses
-
Limited methodological novelty. The core components of HiAda closely resemble existing methods. In particular, d²Cache [1] already performs fine-grained token-level KV-cache updates, making the claimed distinction in Lines 114–117 regarding token-level granularity insufficiently justified. Similarly, the history-aware confidence mechanism resembles CreditDecoding [2], which accumulates historical prediction evidence to accelerate token commitment. The novelty appears to lie primarily in combining existing techniques.
-
Insufficient comparisons with competitive baselines. The main experiments primarily compare HiAda against vanilla decoding and Fast-dLLM, omitting competitive recent methods such as d²Cache [1] and CreditDecoding [2]. Although Dynamic-dLLM and ES-dLLM are additionally evaluated in the appendix, their task-performance scores are not reported, making it difficult to assess the accuracy–efficiency trade-off.
-
Limited applicability to emerging dLLM architectures. Recent dLLMs, including LLaDA2.0 [3] and Fast-dLLM v2 [4], adopt block-diffusion architectures. HiAda's block-free decoding strategy is not directly compatible with their block-causal constraints, potentially limiting its practical applicability to newer dLLMs.
-
Potential hardware inefficiency from non-contiguous token selection. HiAda selects scattered token positions, whereas Fast-dLLM processes contiguous blocks. Prior work such as Native Sparse Attention [5] highlights the importance of hardware-aligned computation and efficient memory access. HiAda's irregular indexing and gather/scatter operations may introduce additional overhead, potentially reducing its throughput advantage on GPUs such as the NVIDIA A100.
Minor Writing and Presentation Issues
-
Section 4.4 states that the full-refresh interval is determined as a function of k, but does not specify the actual function in the main text, requiring readers to consult the appendix.
-
Several sentences contain missing spaces after punctuation, including Lines 36 ("process.Unlike"), 75 ("quality.As"), 119 ("confidence.Fast-dLLM"), 330 ("updates.Accordingly"), and 507 ("Harness.An").
-
Line 919 contains stray quotation marks or unintended characters (```).
Reference
[1] Jiang et al. d²Cache: Accelerating Diffusion-Based LLMs via Dual Adaptive Caching. ICLR 2026. Paper
[2] Wang et al. CreditDecoding: Accelerating Parallel Decoding in Diffusion Large Language Models with Trace Credit. ACL 2026. Paper
[3] Bie et al. LLaDA2.0: Scaling Up Diffusion Language Models to 100B. arXiv 2025. Paper
[4] Wu et al. Fast-dLLM v2: Efficient Block-Diffusion LLM. ICLR 2026. Paper
[5] Yuan et al. Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention. ACL 2025. Paper
Questions for authors
-
How does HiAda scale to longer sequences? Under the proposed refresh schedule , full KV-cache refreshes become less frequent as the generation length increases, potentially aggravating cache staleness. Could the authors evaluate longer generation lengths (e.g., 1K tokens) and tasks involving substantially longer prompts, comparing both throughput and task performance with recent acceleration methods?
-
How does HiAda perform on other GPU architectures? As discussed in W4, non-contiguous token selection may introduce additional hardware overhead. Could the authors evaluate HiAda on GPUs such as the NVIDIA A100 and compare its throughput against Fast-dLLM under identical settings? A runtime breakdown of token indexing, sparse computation, and cache updates would help clarify the impact of irregular memory access.
Thank you for these constructive suggestions. We have conducted additional experiments and investigated the proposed reverse attention direction.
1. Additional Comparisons and Case Study
We have extended Figure 7 with comparisons against D²-Cache and Elastic-Cache under matched hardware and benchmark settings. HiAda achieves 93.3, 226.5, 130.0, and 67.7 TPS on HumanEval, MATH, MBPP, and GSM8K, respectively, compared with approximately 7.5, 6.0, 2.4, and 18.0 TPS for D²-Cache, corresponding to 3.8–54× higher throughput. Compared with Elastic-Cache (approximately 17, 55, 47, and 50 TPS), HiAda achieves 1.4–5.5× higher throughput. Figure 7 also reports the corresponding task accuracy, illustrating the efficiency–quality trade-off.
These improvements arise from several methodological differences:
D²-Cache does not explicitly decouple masked-token prediction from KV recomputation, while its cross-layer attention rollout introduces additional overhead. HiAda uses independent active sets for these two tasks, leveraging historical attention from a single layer to predict promising decoding positions with lower selection overhead.
dLLM-Cache requires additional Attention/FFN output caches (approximately doubling KV-cache storage under MHA). Its V-verify mechanism may also miss context-induced changes at positions whose token identities remain unchanged, requiring periodic full refreshes. HiAda avoids dense Value verification and additional intermediate-feature caches, directly selecting active positions before sparse computation.
Furthermore, HiAda's block-free adaptive window maintains decoding parallelism without waiting for fixed blocks to complete. We will also provide a qualitative decoding trace illustrating the evolution of active sets and the adaptive window.
2. Reverse Attention Direction
We appreciate this insightful suggestion. The proposed reverse attention direction is well-motivated, as newly decoded tokens transition from [MASK] to actual tokens, and attention from remaining masked positions to these newly decoded tokens may intuitively reflect their influence on subsequent predictions. This provides a meaningful alternative perspective for identifying promising decoding positions.
Motivated by this observation, we evaluated the reverse direction under identical settings.
Hit Ratio (generation length = 512, block size = 64, last layer):
The reverse direction consistently reduces the hit ratio by 3.0–6.5 percentage points, suggesting that our original direction provides more reliable predictions of future decoding positions.
Furthermore, reverse attention is less compatible with our token-level sparse computation. HiAda computes attention queries only for the active mask and cache sets, while cached keys cover the entire sequence. Our direction therefore directly provides the required attention scores, whereas reverse attention requires additional queries from unselected masked positions, introducing extra computation and undermining sparsity.
These results support our original attention direction in terms of both selection accuracy and computational efficiency.