- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes LISA, a training-free acceleration method for the token-level indexer in DeepSeek Sparse Attention. It decomposes the weighted-ReLU score into a linear term and a nonlinear correction, and exploits the shared-key structure to fuse the weighted queries from all indexer heads into a single query for full-context scoring. The paper further introduces candidate refinement and cross-layer candidate sharing, and reports substantial indexer and end-to-end TTFT improvements while maintaining similar RULER performance.
Critical strengths
-
The core head-fusion idea is simple but practically meaningful. By exploiting the shared-key structure of the DSA indexer, LISA changes the cost of full-context token scoring itself rather than only reducing how often the original scorer is applied. This gives the method a clear distinction from several recent sparse-indexing approaches.
-
The reported efficiency gains are substantial and translate to end-to-end inference. LISA reports a selector speedup and a TTFT speedup at 1M tokens on DeepSeek-V3.2, while LISA reaches . These are large enough to make the method practically relevant rather than only a kernel-level optimization.
Critical weaknesses
-
The main novelty is concentrated in the head-fused linear scorer, while the refinement and cross-layer sharing extensions largely follow existing design patterns. Although LISA‡ reduces 1M-token TTFT from 113.49s to 73.67s compared with LISA† (Table 2), the paper does not report an end-to-end IndexCache comparison on DeepSeek-V3.2, despite including IndexCache in the quality evaluation. This makes it difficult to assess the advantage of shared candidate refinement over simpler cross-layer index reuse.
-
The practical benefits of the refined variants are not fully established: at 128K on DeepSeek-V3.2, LISA† improves attention-mass recall from 86.26% to 89.41%, but RULER decreases from 94.82 to 94.37 and TTFT increases from 7.49s to 8.89s (Tables 2–4). LISA‡ further reduces recall to 87.24%, while the effects of the fixed candidate budget and layer-group size are not sufficiently analyzed. Moreover, quality evaluation stops at 128K, whereas the largest speedups are reported at 256K–1M.
Questions for authors
-
Could the authors report the end-to-end TTFT of IndexCache on DeepSeek-V3.2 using the same four-layer grouping as LISA‡? This comparison would clarify the additional benefit of shared candidate refinement over direct index reuse.
-
What is the practical motivation for LISA† when its higher recall does not translate into better RULER performance at 128K on DeepSeek-V3.2 (94.37 vs. 94.82), despite higher TTFT (8.89s vs. 7.49s)? Are there tasks or candidate-budget settings where the refinement produces a clear downstream benefit?
-
Can the authors provide quality results beyond 128K, especially at 256K–1M where the efficiency gains are largest? This would help establish whether the approximation remains reliable at the context lengths central to the paper's motivation.
Thank you for the constructive review. We have revised the manuscript with 256K–1M evaluations, matched baselines, and diagnostic analyses, and address your questions below. Thank you also for the constructive suggestions on task-level uncertainty, broader benchmarks, and alternative query positions, which we address in A1 and A3. In the revised PDF, added or revised sentences and reference entries are marked in blue, and new or updated table entries are highlighted.
A1. Quality at 256K–1M (W1, Q1)
Our initial quality suite stopped at 128K because DeepSeek-V3.2 has a native 160K context window. Because the original 256K–1M timing runs used length-converted RoPE scaling factors (64/128/256) and beyond-native evaluation requires quality calibration, we selected fixed-length YaRN factors using Native DSA—choosing 98, 184, and 322 as the best-performing tested settings—froze them across all methods at 256K, 512K, and 1M, and re-measured offline TTFT under the identical factors (A2). GLM-5.2 natively supports 1M tokens without RoPE overrides; thus, 256K–1M quality conclusions primarily rely on native-1M GLM-5.2, while DeepSeek provides supplementary extrapolation evidence with residual calibration uncertainty.
The tables below report RULER accuracy (13 × 100 = 1,300 examples per cell, except DeepSeek 1M, which uses a fixed shared 13 × 20 = 260-example subset due to evaluation cost). All entries are complete and included in revised Table 3.
GLM-5.2 RULER accuracy (%, 13-task mean)
DeepSeek-V3.2 RULER accuracy (%, 13-task mean)
Refinement improves RULER over plain LISA at all three 256K–1M lengths on both models; at 1M, scores rise from 79.16% to 82.42% on DeepSeek and from 77.53% to 80.00% on GLM.
Beyond RULER, we evaluated LongBench v2 on GLM-5.2 (503 shared inputs): Native scores 51.49%, LISA scores 51.09%, and LISA† reaches 53.28% (+2.19 pp over LISA); DeepSeek-V3.2 is omitted because variable prompt lengths lack a single calibrated YaRN factor. Revised Table 4 reports LongBench v2; Appendix C.2 includes task-level RULER scores at 128K and paired 95% bootstrap confidence intervals for the 4K–128K mean.
A2. Matched IndexCache comparison (W2, Q2)
We benchmarked DeepSeek IndexCache with the exact grouping of LISA‡ (G=4: layer 0 independent, remaining layers grouped in fours, k=2048). IndexCache reuses the donor layer’s final top-2048 indices directly, whereas LISA‡ shares the donor’s candidate pool (target M=8192) and applies each layer’s own exact scorer.
DeepSeek-V3.2 offline TTFT (s, median [min, max] across 3 runs)
Without cross-layer sharing (G=1), plain LISA achieves a 4.04× speedup over Native DSA at 1M and outperforms IndexCache-G4 at 512K and 1M. Under matched G=4 grouping, LISA‡ is 1.30× faster than IndexCache-G4 at 512K while improving RULER accuracy by +43.04 pp (74.05% vs. 31.01%), and is 1.76× faster at 1M while raising RULER from 24.10% to 66.62% on the shared 260-example subset (revised Tables 2–3; timing ranges in Appendix C.4, Table 10).