- Rating
- 3
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper propose RISA to address the overhead of repeatedly reading historical key-value (KV) states during the denoising process in block diffusion language models. RISA decouples persistent HOT/COLD storage tiers from the transient history-read schedule utilized by individual blocks. This read schedule—shared across layers and KV heads—combines protected HOT pages, eligible selected HOT pages, retrieved COLD pages. Subsequent denoising steps reuse these historical indices while recomputing attention with fresh queries and current-block KV states.
Evaluation results indicate that performance across DreamReasoner-8B and SDAR-8B-Chat remains on par with dense attention. In DreamReasoner under a 32K prefix configuration, latency drops from 5.438 ms to 2.348 ms, and full-step latency decreases from 12.002 ms to 9.192 ms—yielding speedups of roughly 2.32× and 1.31×, respectively. At a 128K context length, the attention path achieves a 5.53× speedup.
Critical strengths
Clear Motivation: The paper clearly identifies repeated KV reads and construction as key bottlenecks in block-wise diffusion.
Practical Method: RISA combines recoverable history, bounded sparse reads, and block-level plan reuse without modifying the backbone or decoder.
Good Experimental Results: it preserves comparable accuracy while achieving strong and consistent speedups across models and context lengths.
Critical weaknesses
Limited Ablation and Hyperparameter Sensitivity Analysis:
Table 3 only evaluates No COLD recovery, Per-step selection, and No HOT cap, with accuracy measured on a limited AIME25 subset. The paper lacks a systematic quality–latency sensitivity analysis for key hyperparameters such as$q,\ \rho,\ K_{\max},\ G,$ as well as the block size.
Disconnect Between Quality and Latency Gains Evidence linking quality preservation to the sparsity levels driving latency gains remains incomplete. While the paper notes that shorter SDAR quality inputs test admission and recovery mechanisms, read budget caps are triggered only on the longer inputs used for efficiency benchmarks. Because quality reports omit read ratios and cap activations, evaluating performance on long-context workloads that match the efficiency tests is impossible. Although the 64K and 128K scalability tests are valuable and transparent, they cannot bridge this gap since task quality is evaluated only up to 32K.
Questions for authors
see in weakness.
3 comments
Thank you for the detailed responses and additional experiments. These have addressed my concerns, and I have no further questions. I will maintain my score of 3.
Dear Reviewer r1BM,
Thank you for taking the time to review our response and the additional experiments. We are glad that the results addressed your concerns. Your suggestions helped us strengthen the evaluation by testing block-size sensitivity and reporting realized KV reads and HOT-cap activity alongside retrieval quality. We appreciate your constructive feedback.
We appreciate your suggestion to connect the quality results to the amount of history actually read. We now report realized reads and HOT-cap activity alongside retrieval quality, and test sensitivity to block size and the read budget.
1. The ablations isolate recovery, admission, plan reuse, and the HOT cap. Recovery lowers KL at comparable reads, per-step reselection raises attention latency at the same reads, and on SDAR lifting the cap lowers KL while reading almost all history at a higher attention cost. Table 3 uses a 16K prompt and 4,096 generated tokens. Cells give attention ms/step / KV reads / KL ×10⁻³. Attention is profiled on one document per model. Reads and KL use six documents along dense’s trajectories.
Reducing the read fraction lowers realized reads at 16K. At 32K, the HOT cap holds reads nearly constant across the same change, and a smaller cap raises KL (Section 5.3, Appendix E, and Table 10). Speedup uses native 512-token decoding, while KL uses dense’s states.
2. RISA lowers both latencies with 32- and 64-token DreamReasoner blocks. We hold the HOT cap, COLD quota, and read fraction fixed, and keep the grace window and admission reservoir fixed in tokens. Each block size uses its own paired dense trajectory and KL reference. The cap binds at 32K with both block sizes (Section 5.3 and Table 8).
3. Capped reads keep needle recall within six percentage points of dense. On LLaDA2.2-mini at 128K, RISA matches dense needle recall while reading about a third as much history (Section 5.3 and Figure 10b). The default operating points in Table 7 pair per-request latency, KL, and reads on the fixed trajectory.
The needle test uses 32 prompts per DreamReasoner and SDAR setting (Table 11) and 12 per LLaDA2.2-mini setting (Figure 10b and Appendix E).
On SDAR at 32K, LoSA 1K achieves 78.1% recall and 34.4% exact match (Table 11). On 32K book-prefixed MATH-500, RISA also speeds up native requests and matches dense accuracy over 48 paired problems per model (Table 9; Section 6).
On book-prefixed MATH-500, COLD recovery also lowers KL to dense (Table 12).