- Rating
- 3(+2)
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2(+1)
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes REPRISE, a training-based framework for accelerating long-context inference in bidirectional diffusion language models. During inference, REPRISE combines sparse attention with separate KV-cache refresh schedules for prompt and generation tokens. During training, a frozen dense teacher supervises a student operating under sparse attention and reused KV caches. The proposed selective cache replay reconstructs the computational dependencies of historical KV caches, allowing subsequent prediction losses to update earlier cache construction without backpropagating through the entire generation trajectory. Experiments on RULER and LongBench demonstrate improved inference efficiency and generation quality compared with Prefix Cache and Dual Cache
Critical strengths
-
Significant inference efficiency improvements. REPRISE achieves substantial speedups over conventional KV-caching methods, particularly in long-context settings, while maintaining or improving generation quality on RULER.
-
An interesting approach to training under approximate inference. Rather than relying exclusively on training-free approximations, REPRISE explicitly adapts the model to sparse attention and stale KV caches. Its selective replay mechanism provides an efficient way to preserve gradients through historical cache construction.
-
Detailed experimental analysis. The paper includes controlled training ablations, multiple random seeds, and statistical analyses that help distinguish the effects of sparse distillation, cache reuse, and historical cache gradients.
Critical weaknesses
-
Poor organization and unclear presentation. The paper is unnecessarily difficult to follow due to its organization and uneven level of detail. Section 4.1 revisits observations already explored in prior work, such as prompt KV stability in dLLM-Cache [1], while presenting the speedup of sparse prompt refreshes before explaining the actual sparse attention strategy. Equation (2) introduces the function ℓ without explicitly defining it. Furthermore, Section 4.2 presents ablation results intended to demonstrate REPRISE's effectiveness before the method itself is introduced in Section 5. These organizational issues substantially hinder readability and obscure the paper's contributions.
-
Limited methodological novelty. The motivation and training mechanism closely resemble Self Gradient Forcing (SGF) [2], which already identifies the missing gradient path through historical KV caches and addresses it through no-gradient rollout followed by differentiable cache reconstruction. Sparse-aware fine-tuning has also been explored in MAGE v1 [3]. Moreover, the separation of prompt and generation cache updates has been investigated by dLLM-Cache [1], while the hierarchical management of historical and active-block caches resembles Fast-dLLM v2 [4], albeit under a different attention architecture. Overall, REPRISE appears primarily to combine and adapt existing techniques rather than introduce a fundamentally new methodology.
-
Limited practical relevance to emerging dLLM architectures. Recent prominent dLLMs, such as Fast-dLLM v2 [4] and LLaDA2.0 [5], increasingly adopt block-diffusion architectures, where block-causal attention enables exact KV-cache reuse across completed blocks. This largely eliminates the stale prompt-cache problem that motivates REPRISE. Consequently, the practical value of its cache-adaptation mechanism for newer dLLMs remains unclear.
-
Insufficient comparisons with long-context-specific methods. Despite targeting efficient long-context dLLM inference, the main experiments primarily compare REPRISE against conventional caching methods. More relevant baselines include Prefilling-dLLM [6], which addresses redundant long-prompt computation, Focus-dLLM [7]. Without comparisons against these methods under comparable long-context workloads, it is difficult to assess REPRISE's competitiveness in both inference efficiency and generation quality.
Reference
[1] Liu et al. dLLM-Cache: Accelerating Diffusion Large Language Models with Adaptive Caching. ICML 2026. https://arxiv.org/abs/2506.06295
[2] Zhuang et al. Self Gradient Forcing: Native Long Video Extrapolation. arXiv 2026. https://arxiv.org/abs/2607.20368
[3] Kwon et al. MAGE: All-[MASK] Block Already Knows Where to Look in Diffusion LLM. arXiv 2026, v1. https://arxiv.org/abs/2602.14209v1
[4] Wu et al. Fast-dLLM v2: Efficient Block-Diffusion LLM. ICLR 2026. https://arxiv.org/abs/2509.26328
[5] Bie et al. LLaDA2.0: Scaling Up Diffusion Language Models to 100B. arXiv 2025. https://arxiv.org/abs/2512.15745
[6] Xiong et al. Prefilling-dLLM: Predictive Prefilling for Long-Context Inference in Diffusion Language Models. arXiv 2026. https://arxiv.org/abs/2606.10537
[7] Long et al. Focus-dLLM: Accelerating Long-Context Diffusion LLM Inference via Confidence-Guided Context Focusing. ACL 2026. https://aclanthology.org/2026.acl-long.556/
Questions for authors
How does the proposed sparse attention strategy differ from SparseD? Section 5.1 introduces a pooled Q/K-based block selection strategy that appears conceptually similar to SparseD. What are the specific methodological advantages of this design, and why was SparseD not directly adopted?
11 comments
W2: Methodological contribution relative to SGF and prior work
Thank you for pointing out these connections. We have clarified the relationship between the existing components and our contribution in Sections 2 and 5.3. REPRISE uses the overwrite structure of generation refreshes in a bidirectional dLLM to identify which cache computations remain dependencies of selected predictions. This lets it skip overwritten generation-cache history while preserving the gradient path from later losses to prompt-cache construction.
1. Relationship to SGF: using refresh boundaries to identify necessary reconstruction — Sections 2 and 5.3
We acknowledge SGF's use of no-gradient trajectory collection and differentiable context reconstruction to train historical KV representations through later losses. Section 2 explicitly discusses this connection.
REPRISE uses the particular dependencies of its cache policy: at a generation-refresh boundary, every layer recomputes all generation KV entries while continuing to reuse prompt KV. On a fixed saved token trajectory, predictions after that boundary depend on prompt-cache construction, while the overwritten generation-KV computations can be skipped.
For example, a probe in the fourth generation block requires reconstruction of the initial prompt cache, the generation cache at the fourth-block boundary, and the subsequent active-block computation. The first three blocks' generation forwards need not be replayed, while the later loss still differentiates through the reconstructed prompt cache. Section 5.3 now states this overwrite relationship and the basis for retaining or skipping each computation.
2. Relationship to earlier cache policies — Sections 2 and 5.1
We explicitly cite dLLM-Cache's separate update strategies for prompt and generation states and Fast-dLLM v2's block/sub-block cache management. REPRISE retains global bidirectional attention and jointly considers sparse computation, approximate cache reuse, and training under this architecture.
The refresh policy determines both which states are reused during inference and which historical computations affect the selected training losses. Section 5.1 specifies this deployment policy, and Section 5.3 constructs differentiable reconstruction from its dependencies so that training directly matches the sparse cached states used at deployment.
3. Relationship to sparse-aware fine-tuning: testing reused-state training and cache-construction gradients — Section 6.3, Figure 4, and Appendix D
MAGE v1 studies lightweight fine-tuning for sparse attention, and is explicitly cited in Section 2. To examine REPRISE's training components, we compare fresh sparse distillation, detached prompt-cache training, and REPRISE under the same deployment policy, training data, adaptation rank, and update budget. Results on 416 RULER 64K inputs, averaged over three training seeds, are:
| Training method | Mean score | REPRISE score gain | Paired 95% confidence interval |
|---|---|---|---|
| Fresh sparse distillation | 78.24 | +1.38 | [0.40, 2.43] |
| Detached prompt-cache training | 76.67 | +2.95 | [1.74, 4.23] |
| REPRISE | 79.62 | — | — |
Both gains hold for every matched training seed. They support training under reused states and allowing later losses to differentiate through prompt-cache construction, respectively.
4. Direct gradient and resource measurements for selective replay — Section 6.3 and Appendix B.5, Table 9
We added a direct comparison with layerwise activation-checkpointed full-history reconstruction, using identical saved trajectories, token-transfer decisions, attention selections, and selected losses. Across 14 jointly completed configurations and three repetitions each, all 42 paired adapter-gradient comparisons have maximum absolute difference 0, relative L2 difference 0, and cosine similarity 1.
One configuration in Table 9 uses a 512-token prompt, a 512-token generation canvas, and probe position 96, measured on the same H100:
| Reconstruction method | Update time (s) | Peak GPU memory (GiB) |
|---|---|---|
| Full history with activation checkpointing | 42.58 | 64.34 |
| Selective replay | 3.73 | 17.90 |
For later-block probes with 512-token prompts, selective replay uses 8.7%–17.6% of the reference update time and 22.4%–46.6% of its peak memory. With 4,096-token prompts, it also completes the eight later-position configurations where the reference runs out of memory. Gradient agreement concerns identical selected losses on fixed discrete trajectories; these measurements directly establish the benefit of reducing reconstruction through refresh-boundary dependencies.
W3: Practical relevance relative to block diffusion
Thank you for pointing out how attention architecture affects cache reuse. We agree that the block-causal structures in Fast-dLLM v2 and LLaDA2.0 allow exact KV reuse across completed blocks. We have clarified REPRISE's architectural scope and added a discussion of its relationship to block diffusion in prefill optimization.
1. REPRISE targets dLLMs that retain global bidirectional attention — Sections 1, 2, and 3
Sections 1 and 2 state this scope and discuss block-causal models including Fast-dLLM v2 and LLaDA2.0. Section 3 explains that, under global bidirectional attention, changing generated content can alter deeper-layer prompt representations even when the prompt tokens themselves remain fixed. Reusing earlier prompt KV is therefore approximate.
REPRISE trains under this deployment state to adapt the model to sparse attention and cache reuse. Our objective is to reduce quality loss while accelerating inference, and to improve quality in settings where adaptation permits it.
2. Retaining cross-block bidirectional context has practical task value — Section 1
Section 1 now cites Set Diffusion and Speculative Correction. Set Diffusion demonstrates benefits from more flexible cross-block context use relative to block diffusion in text infilling. Speculative Correction's GSM8K ablation shows a benefit from global answer refinement relative to local refinement. These results support the value of cross-block context in the corresponding tasks and motivate efficient execution of models that retain this capability.
3. Sparse prefill and exact cache reuse are potentially complementary — Sections 5.1 and 5.4
We have added this discussion in Section 5.4. For a new prompt without a reusable cache, block diffusion still requires initial prefill to construct prompt KV. Exact cache reuse reduces repeated computation after construction, while sparse prefill can reduce attention computation during initial construction itself.
Section 5.1 specifies that REPRISE applies sparse prompt attention from initialization, without first running dense attention to collect a sparsity pattern. It therefore reduces attention computation in both the first cache construction and subsequent refreshes.
This sparse-prefill design offers a direction for extension to block diffusion: adapt sparse selection and training to the block-causal attention structure and cache dependencies, combining cheaper initial construction with subsequent exact reuse. Section 5.4 presents this as potential complementarity; the measured results in this paper concern UltraLLaDA-8B with global bidirectional attention.
# Response to Reviewer WsBV
Thank you for your detailed review and for recognizing the inference benefits, the training objective, and the experimental analysis. We have revised the organization, clarified the methodological contribution and architectural scope, and added the requested comparisons. We address each weakness and the question below.
W1: Organization and clarity of presentation
Thank you for identifying the specific presentation issues. We have revised Section 4 to introduce the execution mismatch and training cost before the method, placed trained-model comparisons after the method, and supplied the missing definitions.
1. Relating prompt-KV stability to prior work — Section 4.1
Section 4.1 now explicitly credits dLLM-Cache for using the temporal stability of prompt states to reduce cache refreshes. Our measurements examine this observation under the backbone and evaluation settings used in this paper and motivate different refresh frequencies for the prompt and generation regions. The revised text makes this relationship to prior work explicit.
2. Explaining the sparse strategy before its speedup — Section 4.1; full specification in Section 5.1 and Appendix A.3
Before reporting latency, Section 4.1 now explains the strategy used in the measurement: 128-token prompt blocks, pooled Q/K scores for key-block selection, and a nominal 25% prompt-key-block budget. Interactions involving generation tokens and the incomplete prompt-tail block remain dense. Readers can therefore understand the measured strategy before interpreting its speedup, with the exact rule specified in Section 5.1 and Appendix A.3.
3. Defining the loss before Equation (2) — Section 4.2
We now define ℓ(φ, C) as the distillation loss against a fresh dense teacher, computed by subsequent operations using parameters φ and the supplied prompt cache C. The later prediction loss is written as L(φ) = ℓ(φ, C(φ)). Equation (2) then explains the gradient paths through the current prediction computation and the earlier cache construction. The full KL objective appears in Section 5.2, Equation (4).
4. Placing training ablations after the method and revising the motivation — Section 4.2, Figure 2(c–d), and Section 6.3, Figure 4
The trained-model comparisons are now presented in Section 6.3 and Figure 4. Section 4.2 and Figure 2(c–d) instead contain two diagnostics that motivate the method:
These diagnostics lead into cache-aware training and selective replay in Section 5. Their effectiveness is evaluated afterward in Section 6.3.