acceptodds
Under review as a conference paper at ICLR 2027

RISA: Recoverable Sparse Attention for Block-Diffusion Language Models

Abstract

Block-wise diffusion language models (DLMs) refine multiple tokens in parallel, offering an alternative to token-by-token autoregressive decoding. Their long-context inference, however, remains bottlenecked by memory-bound attention: every denoising step repeatedly reads a growing historical key–value (KV) cache. Directly applying sparse attention does not fully solve this problem, because different queries can select different pages, inflating the block's collective read set, while repeated support construction adds overhead and permanent eviction can discard information needed later. We present RISA, a recoverable sparse-attention method with shared read plans for block-wise DLMs. RISA exploits the fact that historical KV remains fixed within a block while queries evolve across denoising steps. Newly generated history is organized into exact priority and recovery tiers; seed queries then build a protected, bounded read plan whose indices are reused across the block while attention is recomputed with fresh queries and current-block KV. This separates history preservation from repeated reading and amortizes sparse-support construction without changing the backbone or decoder. On DreamReasoner-8B, RISA retains comparable task performance and delivers sparse-attention and complete-step speedup over dense at a 32K-token prefix. In a 128K scalability study on H200, sparse-attention speedup rises to as the prefix grows under the same model and decoding stack.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
diffusion language models, sparse attention, efficient inference

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_r1BMOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper propose RISA to address the overhead of repeatedly reading historical key-value (KV) states during the denoising process in block diffusion language models. RISA decouples persistent HOT/COLD storage tiers from the transient history-read schedule utilized by individual blocks. This read schedule—shared across layers and KV heads—combines protected HOT pages, eligible selected HOT pages, retrieved COLD pages. Subsequent denoising steps reuse these historical indices while recomputing attention with fresh queries and current-block KV states.

Evaluation results indicate that performance across DreamReasoner-8B and SDAR-8B-Chat remains on par with dense attention. In DreamReasoner under a 32K prefix configuration, latency drops from 5.438 ms to 2.348 ms, and full-step latency decreases from 12.002 ms to 9.192 ms—yielding speedups of roughly 2.32× and 1.31×, respectively. At a 128K context length, the attention path achieves a 5.53× speedup.

Critical strengths

Clear Motivation: The paper clearly identifies repeated KV reads and construction as key bottlenecks in block-wise diffusion.

Practical Method: RISA combines recoverable history, bounded sparse reads, and block-level plan reuse without modifying the backbone or decoder.

Good Experimental Results: it preserves comparable accuracy while achieving strong and consistent speedups across models and context lengths.

Critical weaknesses

Limited Ablation and Hyperparameter Sensitivity Analysis:
Table 3 only evaluates No COLD recovery, Per-step selection, and No HOT cap, with accuracy measured on a limited AIME25 subset. The paper lacks a systematic quality–latency sensitivity analysis for key hyperparameters such as$q,\ \rho,\ K_{\max},\ G,$ as well as the block size.

Disconnect Between Quality and Latency Gains Evidence linking quality preservation to the sparsity levels driving latency gains remains incomplete. While the paper notes that shorter SDAR quality inputs test admission and recovery mechanisms, read budget caps are triggered only on the longer inputs used for efficiency benchmarks. Because quality reports omit read ratios and cap activations, evaluating performance on long-context workloads that match the efficiency tests is impossible. Although the 64K and 128K scalability tests are valuable and transparent, they cannot bridge this gap since task quality is evaluated only up to 32K.

Questions for authors

see in weakness.

3 comments

We appreciate your suggestion to connect the quality results to the amount of history actually read. We now report realized reads and HOT-cap activity alongside retrieval quality, and test sensitivity to block size and the read budget.

1. The ablations isolate recovery, admission, plan reuse, and the HOT cap. Recovery lowers KL at comparable reads, per-step reselection raises attention latency at the same reads, and on SDAR lifting the cap lowers KL while reading almost all history at a higher attention cost. Table 3 uses a 16K prompt and 4,096 generated tokens. Cells give attention ms/step / KV reads / KL ×10⁻³. Attention is profiled on one document per model. Reads and KL use six documents along dense’s trajectories.

VariantDreamReasonerSDAR
No COLD recovery2.54 / 56% / 46.71.68 / 48% / 44.3
COLD reads spent on HOT2.62 / 59% / 45.11.75 / 56% / 45.1
Flat retrieval2.92 / 60% / 5.211.77 / 54% / 2.60
COLD quota of 16 pages2.65 / 57% / 14.51.71 / 49% / 13.9
Recency admission2.83 / 60% / 4.101.76 / 54% / 1.70
Random admission2.74 / 60% / 4.141.76 / 54% / 1.72
Mass + learned correction2.81 / 60% / 4.081.85 / 54% / 1.76
Per-step reselection3.69 / 60% / 3.333.14 / 54% / 1.66
No HOT cap2.85 / 60% / 4.202.16 / 98% / 0.52
RISA (final)2.86 / 60% / 4.211.77 / 54% / 1.74

Reducing the read fraction lowers realized reads at 16K. At 32K, the HOT cap holds reads nearly constant across the same change, and a smaller cap raises KL (Section 5.3, Appendix E, and Table 10). Speedup uses native 512-token decoding, while KL uses dense’s states.

DreamReasoner settingKV readsKL ×10⁻³Per-request speedup
16K, default53.9%4.51.14×
16K, ρ=0.2530.1%13.20.99×
32K, default30.8%13.51.61×
32K, 2,048-token HOT cap12.6%57.51.52×
32K, ρ=0.2530.7%13.91.74×
32K, grace window of 4 blocks30.7%13.41.59×
32K, per-step reselection31.0%10.51.33×

2. RISA lowers both latencies with 32- and 64-token DreamReasoner blocks. We hold the HOT cap, COLD quota, and read fraction fixed, and keep the grace window and admission reservoir fixed in tokens. Each block size uses its own paired dense trajectory and KL reference. The cap binds at 32K with both block sizes (Section 5.3 and Table 8).

Block / contextAttention ms/step, dense / RISAPer-request s, dense / RISARISA KL ×10⁻³KV readsHOT cap active (% of blocks)
32 / 16K3.19 / 1.84 (1.73×)3.78 / 3.28 (1.15×)5.354%0%
32 / 32K5.95 / 2.06 (2.89×)4.14 / 3.06 (1.36×)14.431%100%
64 / 16K3.36 / 2.60 (1.30×)4.19 / 3.84 (1.09×)6.952%0%
64 / 32K5.81 / 2.76 (2.11×)4.31 / 3.38 (1.28×)16.129%100%

3. Capped reads keep needle recall within six percentage points of dense. On LLaDA2.2-mini at 128K, RISA matches dense needle recall while reading about a third as much history (Section 5.3 and Figure 10b). The default operating points in Table 7 pair per-request latency, KL, and reads on the fixed trajectory.

Model, 32KDense (s)Default RISA (s)KL ×10⁻³KV reads
DreamReasoner4.093.0014.431.1%
SDAR7.135.026.528.9%
LLaDA2.2-mini2.952.8119.334.1%

The needle test uses 32 prompts per DreamReasoner and SDAR setting (Table 11) and 12 per LLaDA2.2-mini setting (Figure 10b and Appendix E).

Model / contextRISA KV readsHOT cap active (% of sparse blocks)Needle recall (%), RISA / denseIn-order exact match (%), RISA / dense
DreamReasoner 16K53.8%0%92.7 / 92.746.9 / 46.9
DreamReasoner 32K29.7%100%92.7 / 93.840.6 / 43.8
SDAR 16K53.1%100%51.0 / 51.06.2 / 6.2
SDAR 32K29.1%100%65.6 / 68.812.5 / 18.8
LLaDA2.2-mini 16K65.3%0%77.8 / 77.833.3 / 25.0
LLaDA2.2-mini 32K47.8%100%80.6 / 86.125.0 / 33.3
LLaDA2.2-mini 64K36.5%100%69.4 / 75.016.7 / 8.3
LLaDA2.2-mini 128K31.9%100%80.6 / 80.633.3 / 25.0

On SDAR at 32K, LoSA 1K achieves 78.1% recall and 34.4% exact match (Table 11). On 32K book-prefixed MATH-500, RISA also speeds up native requests and matches dense accuracy over 48 paired problems per model (Table 9; Section 6).

ModelNative per-request speedupAccuracy, RISA / dense
DreamReasoner1.55×77.1% / 77.1%
SDAR1.47×62.5% / 62.5%

On book-prefixed MATH-500, COLD recovery also lowers KL to dense (Table 12).

ModelKL ×10⁻³, RISA / no COLD recovery
DreamReasoner9.0 / 47.9
SDAR3.4 / 26.9

Thank you for the detailed responses and additional experiments. These have addressed my concerns, and I have no further questions. I will maintain my score of 3.

Dear Reviewer r1BM,

Thank you for taking the time to review our response and the additional experiments. We are glad that the results addressed your concerns. Your suggestions helped us strengthen the evaluation by testing block-size sensitivity and reporting realized KV reads and HOT-cap activity alongside retrieval quality. We appreciate your constructive feedback.

Reviewer_UoSkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

RISA reduces repeated historical KV reads in block-wise diffusion language models. Generated history remains in a grace window before an observation-guided policy assigns exact KV to HOT or COLD storage. Seed queries construct a protected, bounded page plan that subsequent denoising steps reuse with fresh attention computations. Finalization still reads the full legal prefix.

Experiments on DreamReasoner-8B and SDAR-8B-Chat report task scores close to dense attention. On DreamReasoner, the reported attention-path and complete-step speedups are 2.32× and 1.31× at 32K. Attention speedup reaches 5.53× at 128K, explicitly evaluated as scalability beyond the configured model context range.

Critical strengths
  1. RISA combines recoverable generated history, shared page selection, and reuse across denoising steps while recomputing attention with current queries.

  2. Task scores remain close to dense attention while measured attention-path and complete-step costs decrease.

Critical weaknesses
  1. The central observation-guided history mechanism is neither fully specified nor directly validated against a simpler alternative. Section 4.1 leaves the priority-score function and aggregation of observations undefined; Figure 4 introduces a “learned corr.” without explaining its role or training. Algorithm 1 takes the priority rule as an input rather than defining it. This prevents reproduction of a central contribution.

Establishing the additional value of this mechanism is particularly important because MAGE v2 already selects shared indices from preserved history at each block and reuses them during denoising. RISA’s Table 3 establishes the utility of recovery, reuse, and a read cap, but does not isolate the proposed admission policy or tiered organization against a simpler policy using the same executor and comparable reads. The paper therefore provides stronger evidence for sparse retrieval and reuse in general than for its distinctive history-management design.

  1. The quality–efficiency advantage over existing sparse methods is not sufficiently established under comparable conditions. QUEST and MAGE use nominal 1,024-token budgets, whereas RISA allows up to 8,192 eligible HOT slots plus protected and mandatory rows. These are not equivalent realized read counts, and different budgets do not inherently invalidate a comparison. Nevertheless, a representative quality–latency tradeoff is needed to determine whether RISA offers a better operating point, rather than preserving quality by allocating more history while benefiting from a cheaper selector.

The efficiency comparison also needs to account for MAGE’s execution design: its v2 algorithm explicitly overlaps score construction and selection with the main forward pass on a separate CUDA stream. RISA does not establish whether this scheduling is retained. Moreover, timing uses 512-token-cap runs and reports latency per denoising step, while quality evaluation includes much longer generations. This leaves uncertainty about whether the reported quality and speed advantages hold together on the target workloads. These limitations concern the comparative claim; they do not negate the measured improvement over dense attention.

Questions for authors
  1. What is the complete admission/priority rule, including how observations are aggregated and what the “learned corr.” represents? If learned, what data and objective are used?

  2. Can you isolate the benefit of observation-guided tiering with a direct comparison against a simple admission policy or shared page retrieval over all preserved history, using the same executor and comparable total reads?

  3. Can you provide a representative matched-quality latency comparison with QUEST and MAGE on the evaluated workloads, reporting total decoding time and realized reads? For MAGE, please clarify whether v2’s asynchronous selection is retained and how overlap is reflected in the measurements.

2 comments

We appreciate your questions about the admission rule and the comparison at matched fidelity.

1. Final admission uses executed attention mass alone. When a 32-token block leaves the grace window, each layer and KV head scores its tokens by πi=log⁡(1+wi)\pi_i=\log(1+w_i). The accumulated mass sums the executed sparse attention probabilities the token received over grace-window blocks, denoising steps, query rows, and mapped query heads under the joint normalizer. The allocator selects scores above the 75th percentile of the reservoir of up to 24 previously matured blocks, keeping 2–24 tokens. It uses the top eight when the reservoir is empty and then adds all scores to the reservoir. Admitted tokens enter HOT and the rest enter COLD. Both tiers keep exact KV (Section 4.1 and Algorithm 1).

The earlier 801-parameter variant added 4tanh⁡(MLP(xi,eℓ))4\tanh(\mathrm{MLP}(x_i,e_\ell)) over grace-window mass, admitting-block mass, step-zero mass, top-eight hit count, key norm, block position, and an eight-dimensional layer embedding. Training used 10,332 block–layer samples from 12 development prompts. It was trained on states visited by the sparse decoder, with the frozen dense teacher's log attention mass from two blocks later onward as the target.

2. COLD recovery lowers KL at comparable reads, and separate tiers improve on flat retrieval. MAGE and RISA both reuse a selection within each block. All variants use the same executor, a 16K prompt, and 4,096 generated tokens (Table 3). Cells give attention ms/step / KV reads / KL ×10⁻³. Attention is profiled on one document per model. Reads and KL are measured on six documents along dense's trajectories.

VariantDreamReasonerSDAR
COLD reads spent on HOT2.62 / 59% / 45.11.75 / 56% / 45.1
Flat ranking over all history2.92 / 60% / 5.211.77 / 54% / 2.60
RISA (final)2.86 / 60% / 4.211.77 / 54% / 1.74

The HOT-only and flat controls isolate the value of recovery and tiering while holding read volume close. At 16K the three admission alternatives span KL 4.08–4.14 on DreamReasoner and 1.70–1.76 on SDAR (×10⁻³), compared with 4.21 and 1.74 for mass-only. Reads are 60% and 54%, respectively, for all four schemes. On seven AIME24 problems, mass-only admission approaches the learned correction and lowers KL relative to recency and random admission at similar reads. All five variants below use a 128-page COLD quota (Section 5.3 and Table 4).

Admission or retrievalKL ×10⁻³KV reads
Mass + learned correction8.6075.9%
Recency10.0876.2%
Random10.3076.0%
Flat retrieval11.1576.6%
Mass only (RISA)8.7276.0%

3. RISA lowers decode time at matched KL with MAGE's asynchronous selection enabled. MAGE overlaps selection with the forward pass on a separate CUDA stream and keeps its first two layers exact. Decode wall time includes exposed synchronization, and reads include protected, recent, and recovered rows. At 32K, each of the 28 baseline operating points has a faster RISA counterpart with no higher KL (Table 7). Timing excludes warm-up. KL (×10⁻³) and reads cover all paired documents on the fixed trajectory.

Model, 32KDense (s)Baseline budgetBaseline s / KL / readsRISA budgetRISA s / KL / reads
DreamReasoner4.09MAGE 8K3.39 / 6.8 / 33.9%16K, ρ=1.03.10 / 5.6 / 54.8%
DreamReasoner4.09QUEST 8K3.76 / 72.6 / 29.4%2K, ρ=0.52.93 / 49.5 / 13.3%
SDAR7.13MAGE 1K5.18 / 41.8 / 11.5%2K, ρ=1.04.85 / 28.0 / 10.6%
SDAR7.13QUEST 1K5.88 / 351.7 / 8.5%1K, ρ=1.04.82 / 44.4 / 7.6%
LLaDA2.2-mini2.95MAGE 8K2.88 / 11.5 / 39.8%16K, ρ=0.752.85 / 11.3 / 56.8%

On SDAR, RISA lowers both latency and KL at comparable or lower realized reads than MAGE and QUEST. The high-fidelity MAGE pairs on DreamReasoner and LLaDA2.2-mini use more reads for RISA and still reduce total decode time. On 32K book-prefixed MATH-500 under native decoding, RISA has the lowest measured per-request latency and matches dense answer accuracy. Each cell gives mean seconds / accuracy over 48 paired problems per model, with each method following its own decoding trajectory.

ModelDenseRISAMAGE 8KQUEST 8KLoSA 4K
DreamReasoner5.81 / 77.1%3.73 / 77.1%4.57 / 79.2%5.02 / 81.2%6.51 / 75.0%
SDAR6.90 / 62.5%4.70 / 62.5%5.32 / 62.5%6.46 / 60.4%7.52 / 56.2%

Dear Reviewer UoSk,

Thank you for asking us to isolate the contribution of the admission rule. On 23 held-out AIME24 problems, mass-only admission achieves nearly the same pooled KL and realized KV reads as the learned correction. It also yields lower pooled KL than recency, random admission, and flat retrieval at similar read volumes. All five rules use the same executor and a 128-page COLD quota (Table 4).

Admission or retrieval rulePooled KL ×10⁻³KV reads
Mass only (RISA)8.4375.5%
Mass + learned correction8.4075.3%
Recency8.9075.6%
Random9.2175.2%
Flat retrieval10.0576.3%

We would appreciate any further thoughts on these controls or the matched-fidelity comparison. If any point would benefit from more detail, we would be happy to address it here.

Reviewer_i8WKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RISA, a sparse attention method for block-wise diffusion language models. RISA organizes historical KV entries into hot and cold tiers without permanent eviction, and constructs a bounded read plan that is reused across denoising steps while attention is recomputed with updated queries. Experiments on multiple models shows competitive accuracy and promising efficiency improvements, achieving 2.32× attention speedup and 1.31× complete-step speedup.

Critical strengths
  1. The paper addresses an important bottleneck in block-wise diffusion inference. Separating historical KV preservation from repeated memory accesses is well motivated.

  2. The experiments demonstrate consistent efficiency improvements across models and context lengths.

Critical weaknesses
  1. The largest speedups are reported at 64K/128K, beyond the model's 32K context length. Moreover, these experiments primarily benefit from capped hot reads rather than cold recovery.

  2. A comparison with LoSA is missing. Comparing with relevant baselines is critical to validate the effectiveness of the proposed method.

  3. The relatively narrow measurement scope raises questions about how well the reported speedups generalize across requests with different decoding trajectories, generation lengths, and task characteristics.

Questions for authors

Please refer to the weakness section.

2 comments

We appreciate your comments on the scope of the efficiency evaluation.

1. RISA leads the tested sparse baselines throughout LLaDA2.2-mini's native window. LLaDA2.2-mini supports 16K–128K natively. The 64K and 128K DreamReasoner and SDAR measurements use YaRN and measure execution. All speedups below are relative to dense. Attention uses native decoding with block work amortized over dense's steps. The fixed-trajectory and native-decoding rows measure complete 512-token decodes.

LLaDA2.2-mini16K32K64K128K
RISA attention / fixed-trajectory request speedup1.10× / 1.07×1.70× / 1.06×2.33× / 1.16×3.37× / 1.25×
Best sparse baseline on each metric0.91× / 1.01×1.28× / 1.03×1.47× / 1.10×1.79× / 1.19×
RISA native request speedup0.98×1.12×1.11×1.20×
Best sparse baseline, native request0.86×1.06×1.01×1.06×

COLD recovery lowers KL at comparable read volumes. The HOT cap bounds repeated reads. The same-executor control spends the recovery quota on HOT pages, keeping total reads similar while changing which history is read. Cells give realized KV reads / KL ×10⁻³ after 4,096 generated tokens.

Model, prefixCOLD reads spent on HOTRISA
DreamReasoner, 16K59% / 45.160% / 4.21
SDAR, 16K56% / 45.154% / 1.74
LLaDA2.2-mini, 16K68% / 83.771% / 4.59
LLaDA2.2-mini, 64K51% / 35.452% / 12.0

2. Each tested LoSA budget has a faster RISA point with no higher KL. LoSA reuses attention outputs and recomputes the queries that changed most. Both methods use the shared paged executor. At 32K, the RISA point shown below also uses fewer reads than LoSA at both tested budgets. Cells give decode seconds / KL ×10⁻³ / realized KV reads on dense's fixed trajectory.

Model, 32KLoSA 1KLoSA 4KRISA, 1K HOT cap
DreamReasoner3.92 / 149.0 / 13.8%4.06 / 140.1 / 34.5%2.91 / 79.3 / 10.3%
SDAR6.65 / 161.1 / 12.4%6.90 / 156.5 / 31.9%4.82 / 44.4 / 7.6%
LLaDA2.2-mini3.10 / 137.5 / 18.5%3.15 / 128.6 / 36.6%2.78 / 76.8 / 14.2%

The attention-path profiles explain the efficiency advantage. At default budgets, RISA has the lowest summed kernel time among sparse methods in all 12 settings. Figure 7 shows where time goes. Cells give RISA / LoSA kernel ms per denoising step on H200, with block work amortized over dense's mean steps.

Model16K32K64K128K
DreamReasoner1.78 / 4.841.99 / 5.212.38 / 6.653.15 / 9.57
SDAR2.15 / 4.492.44 / 5.502.83 / 6.814.41 / 10.86
LLaDA2.2-mini0.77 / 1.991.11 / 2.391.13 / 2.871.63 / 3.84

3. The latency gains carry over to native requests. In the 512-token native sweep, RISA speeds up DreamReasoner by 1.20× at 16K and 1.51× at 32K, and SDAR by 1.16× and 1.47×. On 32K book-prefixed MATH-500, every method decodes its own trajectory on 48 paired prompts per model, with a 2,048-token output cap. RISA has the lowest mean decode time and matches dense answer accuracy on both models.

Model, 32KDenseRISAMAGE 8KQUEST 8KLoSA 4K
DreamReasoner5.81 / 77.1%3.73 / 77.1%4.57 / 79.2%5.02 / 81.2%6.51 / 75.0%
SDAR6.90 / 62.5%4.70 / 62.5%5.32 / 62.5%6.46 / 60.4%7.52 / 56.2%

Cells give mean decode seconds / accuracy. The paired latency and accuracy results accompany similar output lengths. The attention speedups below use one profiled request per method and setting.

ModelRISA attention speedupRISA request speedupMean output tokens, RISA / dense
DreamReasoner2.69×1.55×771 / 795
SDAR3.02×1.47×585 / 587

The context-adaptive cap lowers KL at long contexts. On 12 LLaDA2.2-mini documents, RISA-K reduces KL relative to the fixed cap while retaining per-request speedup over dense on the same fixed trajectory.

PrefixFixed-cap KLRISA-K KLRISA-K request speedup
32K0.0190.0121.04×
64K0.0230.0081.12×
128K0.0550.0121.17×

Dear Reviewer i8WK,

Thank you for raising the questions about native context lengths and whether the speedups carry over to task workloads. At 128K within LLaDA2.2-mini’s native window, RISA achieved a 1.20× per-request speedup over dense, compared with 1.00× for LoSA 4K (Table 6). On 32K book-prefixed MATH-500, RISA also reduced mean decode time on both DreamReasoner and SDAR while matching dense answer accuracy, with each method following its own decoding trajectory (Table 9).

If you have a chance to revisit the results, we would welcome any further questions or suggestions and would be happy to clarify any remaining points.

Comments on the paper

1 comment
Area_Chair_QrGPOpenReview ↗

Reminder: Reviewer Engagement During the Discussion Period

Dear Reviewers,

As the discussion period is drawing to a close, could you please review the authors' rebuttal and provide your feedback at your earliest convenience?

While I would like to thank Reviwer r1BM for the response to the authors' rebuttal, the other two reviewers are still absent from the dicussions.

Active engagement in discussions with the authors is an essential part of the peer-review process and plays an important role in ensuring the success of the conference. Your participation would be greatly appreciated.

Thank you for your time and contributions.

Best regards,

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.