- Rating
- 4(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes MISA, an MoE-inspired approach to reducing indexer overhead in DSA. MISA uses dot products between queries and block-pooled keys to select 4 or 8 indexer heads, substantially reducing the indexer’s GEMM cost. It achieves a 1.42× TTFT speedup at a context length of 256K. To mitigate the resulting quality degradation, the paper further introduces a hierarchical variant, MISA†, which first performs coarse candidate filtering with fewer heads and then refines the scores through more precise computation. In the training-free setting, MISA† outperforms IndexCache and HISA.
Critical strengths
- S1: Addresses a relevant bottleneck. Indexer overhead has become a key bottleneck in long-context sparse attention. MISA’s MoE-inspired head selection offers a well-motivated approach to reducing this overhead.
- S2: Uses query–key interactions for routing. Compared with selecting heads solely based on , routing through query–block-pooled-key dot products explicitly accounts for query–key interactions. The reported experiments support the effectiveness of this design.
- S3: Demonstrates competitive quality. The reported improvements over IndexCache provide evidence for the value of the proposed approach.
Critical weaknesses
- W1: Absence of fair compassion. MISA can be interpreted as a form of dynamic indexer-head pruning, but the paper lacks a comparison with a fixed subset of indexer heads under the same active-head budget. This baseline would help clarify how much of the benefit comes from dynamic routing rather than simply reducing the number of heads.
Questions for authors
- Q1: Why do MISA† and MISA incur different Top-K runtimes? Could the authors clarify the number and sizes of the Top-K operations in each method?
- Q2: Since MISA reduces indexer computation along the head dimension, can it be combined with methods such as IndexCache?
4 comments
Answer for Q2
Yes. IndexCache reduces the number of indexer computations by reusing index results across layers, while MISA reduces the cost of each indexer computation along the head dimension, so the two can be combined. Our new GLM-5.2 experiments retain the base model's IndexCache cross-layer reuse mechanism and apply MISA or MISA† on top of it, directly demonstrating the feasibility of this combination. Single-stage MISA achieves an average RULER score of 87.87, below HISA's 90.45, whereas MISA† recovers to 97.08, only 0.49 points below the base model's 97.57. These results support that combining MISA† with IndexCache maintains task quality close to the base model.
The table below reports RULER results on GLM-5.2, where IndexCache denotes the base model and HISA, MISA, and MISA† follow the experimental settings in the paper. Avg. is the equally weighted mean over the six context lengths.
| Method | 4K | 8K | 16K | 32K | 64K | 128K | Avg. |
|---|---|---|---|---|---|---|---|
| IndexCache (base model) | 97.91 | 97.80 | 97.67 | 97.83 | 97.31 | 96.87 | 97.57 |
| HISA | 98.14 | 98.18 | 96.97 | 91.55 | 82.62 | 75.21 | 90.45 |
| MISA | 98.02 | 97.39 | 97.15 | 96.13 | 75.57 | 62.94 | 87.87 |
| MISA† | 98.17 | 97.95 | 97.54 | 97.46 | 96.96 | 94.39 | 97.08 |
Response to authors' rebuttal
Thank you for the additional experiments. Q1 is resolved: the stage-wise timings explain the Top-k difference between MISA and MISA†.
On Q2, the GLM-5.2 results show that MISA† stays close to the base model (97.08 versus 97.57) on top of IndexCache. This second-model evidence is what moves me.
Given that MISA† retains near-base quality on a second model and composes with IndexCache, I am raising my rating to 4.
We sincerely thank you for the thoughtful feedback, recognition of our clarifications and additional experiments, and positive reassessment of our work.
We thank the reviewer for the questions on a fixed-head baseline, the number and sizes of Top-K operations, and compatibility with IndexCache.
Answer for W1
The only thing MISA loses is the number of activated heads, which in DSA would have required pooling anyway. The reason DSA maintains multiple heads is primarily to preserve the representational capacity of the indexer. The MoE mechanism naturally meets the need for fewer activations with stronger expressiveness. To verify these claims, we conducted pre-training experiments:
We directly compared head pruning with MISA in terms of training effectiveness. We used a 30B-A3B DeepSeek-V3-like model with 64-head MLA for core attention. From scratch, we trained DSA indexers with 1–32 heads and MISA with 32 heads selecting 1–32 heads. The training data comprised 0.1B tokens with a sequence length of 32K.
Results show that DSA with 8 heads suffers a significant performance drop, while MISA maintains good results even when activating only a single head, confirming MISA's representational power. Moreover, MISA 4/32 and 8/32 outperform the 32-head DSA, demonstrating that MISA not only reduces computation but also improves quality.
Answer for Q1
Two factors should be distinguished when interpreting the different Top-K times of MISA and MISA† in the original results: first, the original table combines the Top-K times of both stages of MISA†, whereas MISA performs Top-K only once; second, the first stage of MISA† selects 8192 candidates, whereas MISA selects only 2048 tokens. In its second stage, MISA† rescores these 8192 candidates using all heads and then selects the final 2048 tokens.
To determine whether combining the two stages' times is the main reason for the difference, we conducted additional component-level tests with separate timings for each stage; the tests run the indexer directly on synthetic data with key lengths of 256K/512K and a fixed query length of 8K, and report runtimes in milliseconds. DSA, MISA, and the second stage of MISA† use the same vLLM Top-K kernel, while the first stage of MISA† still uses an external kernel to select 8192 candidates. At 256K, the first- and second-stage Top-K operations of MISA† take 8.26 ms and 0.31 ms, respectively; at 512K, they take 14.71 ms and 0.33 ms. The second stage takes only approximately 3.8% and 2.2% of the first-stage time, respectively, making its overhead negligible by comparison. This indicates that the additional second-stage Top-K operation is not the main source of overhead. These supplementary results therefore support the explanation that the original difference mainly stems from the different first-stage selection sizes: MISA† selects 8192 candidates from the long history, whereas MISA selects only 2048 tokens. We have updated the “Component latency breakdown” section of the revised MISA manuscript with these stage-wise timings, the measurement settings, and the Top-K explanation, and marked the section in blue.
Indexer component runtimes: key length 256K, query length 8K (ms)
Indexer component runtimes: key length 512K, query length 8K (ms)