- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
DUET is a training-free inference method specific for block diffusion language models. It combines two complementary mechanisms for reducing computation. First, at each layer, it selects a subset of historical prefix KV and reuses them within the new generated block. Second, at a relatively shallow layer, it identifies active query rows early (in new generated block) and allows only those rows to compute through the remaining layers.
The paper evaluates DUET on LLaDA2.1-mini and SDAR-8B-Chat-b32. At a 32K context length, compared with dense inference, DUET achieves speedups of 1.34×/2.83× and 2.41×/4.72×, respectively. In terms of accuracy, LLaDA’s MATH-500 score decreases from 91.80 to 87.60. For SDAR, the accuracy remains close to that of dense inference. These results suggest that the method can provide useful acceleration for long-context inference, although the impact on output quality depends on both the task and the model.
Critical strengths
-
Coherently combines two complementary mechanisms for reducing computation. Firstly, DUET selects a subset of historical prefix KV and reuses them within the new generated block. Second, at a relatively shallow layer, it identifies active query rows early (in new generated block) and allows only those rows to compute through the remaining layers.
-
Useful long-context latency gains. At a 32K context length with a budget of 256, the reported latency results support speedups of 1.34×/2.83× over dense inference for LLaDA and 2.41×/4.72× for SDAR.
Critical weaknesses
-
Ablation is insufficient The paper not reports latency–quality ablations for prefix sparsity alone, query sparsity alone, and their combination. It also does not separately evaluate the effect of reusing inactive KV entries within the current block. This makes it difficult to causally attribute the observed gains to specific components and leaves the evidence for configuration robustness across the two models somewhat limited.
-
The novelty relative to prior observations on the stability of prefix sparsity could be clarified further. Prefix sparsity stability appears to play an important role in DUET, yet related work such as SparseD has already observed that attention patterns in diffusion language models can remain stable across diffusion steps. It would therefore be helpful for the paper to more clearly distinguish which part of DUET constitutes the novel contribution beyond this prior observation.
Questions for authors
As illustrated in Weakness.
3 comments
Replies to Reviewer D7pp (2)
W2: Clarification of Novelty Beyond SparseD and the Motivation for Dual Sparsity
Thank you for pointing out this related work. We agree that SparseD has already observed the stability of attention patterns across denoising steps in DLMs and leveraged this property to reuse precomputed sparse attention patterns. We would like to clarify that DUET does not claim this observation or the general idea of reusing sparse patterns across denoising steps as its primary novelty.
There are two important distinctions between SparseD and DUET. First, SparseD focuses on DLMs, whereas DUET targets BDLMs, which employ block-wise generation and have a different inference structure. Although attention stability has been observed in DLMs, its effectiveness for reducing redundant computation in BDLMs requires separate investigation. Our experiments validate that this property can also be effectively exploited in the BDLM setting. Second, SparseD reuses precomputed head-specific sparse attention patterns, whereas DUET selects a compact set of historical KV entries for each generation block and reuses this selection across denoising steps within that block.
More importantly, DUET's main contribution is the identification and exploitation of two complementary sources of redundant computation in BDLM inference, rather than the observation of attention stability alone. Specifically, we find that (1) only a small subset of historical KV entries is important for the current generation block, motivating prefix sparsity; and (2) only a subset of positions in the current block requires full computation at each denoising step, motivating query sparsity. These two mechanisms target different dimensions of computation and are complementary to each other.
Based on these observations, we propose DUET, a training-free dual-sparse inference framework that jointly exploits historical-context sparsity and current-block query sparsity. Our ablation experiments further show that neither component alone provides the same overall quality–speed trade-off as their combination. The full method maintains comparable accuracy while achieving effective end-to-end acceleration, particularly in long-context settings.
We will clarify the relationship with SparseD in the revised paper and emphasize that DUET's novelty lies in its BDLM-oriented characterization of computational redundancy and the resulting dual-sparse inference design, rather than in claiming the first observation of attention stability across denoising steps.
Thank you for the additional ablations and the clarification. These results help clarify the contributions of the two sparsity components and their quality–latency trade-off. I maintain my rating of 3.
Replies to Reviewer D7pp (1)
Thank you for the thoughtful comments. Below we provide our detailed responses to each concern. For closely related questions and concerns, we provide a combined clarification.
W1: Ablations of Prefix/Query Sparsity and KV Reuse
Thank you for this helpful suggestion. We conducted additional ablation experiments to isolate the contributions of the two sparsity components and investigate the effect of reusing inactive KV entries across denoising steps. All experiments were conducted using the corrected implementation with the resolved-token editing issue fixed (raised by Reviewer 6hrN).
1. Ablation of prefix and query sparsity.
To isolate the contribution of each component, we evaluated Prefix-only DUET, Query-only DUET, and Full DUET in terms of both accuracy and end-to-end latency, using the same configuration as reported in the paper (
prefix_budget=256,query_ratio=0.7).Table G: LongBench accuracy.
Table H: Accuracy on short-context benchmarks.
As shown in Tables G and H, Full DUET achieves accuracy comparable to the two individual components across both benchmark groups, demonstrating that combining the two mechanisms maintains comparable accuracy.
Table I: End-to-end runtime and speedup over Dense at a 32K context length.
The latency results demonstrate the complementary contributions of the two components in long-context inference. Prefix-only achieves a 1.37× speedup, while Query-only achieves a 1.30× speedup over dense inference at 32K. By combining both mechanisms, Full DUET achieves a 1.34× speedup, approaching the performance of Prefix-only while retaining comparable accuracy to both individual components. These results show that the two sparsity mechanisms complement each other, enabling DUET to effectively reduce redundant computation in long-context inference.
2. Ablation of KV reuse across denoising steps.
To further investigate the effect of reusing inactive KV entries within the current generation block, we implemented a variant called Refresh, which recomputes and reselects the KV entries at every denoising step instead of reusing the selected KV entries across steps. We compared its accuracy and end-to-end latency with DUET.
Table J: LongBench accuracy.
Table K: End-to-end runtime and speedup over Dense at a 32K context length.
The results show that Refresh achieves a modest average accuracy improvement of 0.57 percentage points over DUET on LongBench. However, recomputing and reselecting KV entries at every denoising step introduces substantial computational overhead. At 32K, Refresh takes 53.615 s, approximately 20.6× longer than DUET (2.600 s), and is substantially slower even than dense inference. This indicates that although refreshing KV entries can yield a modest accuracy improvement, the associated overhead makes it impractical for efficient long-context inference. By reusing the selected KV entries across denoising steps, DUET avoids this redundant computation and achieves a substantially better efficiency–accuracy trade-off. We will include this ablation analysis in the revised manuscript.