- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents LiteTopK, a fused Indexer-TopK GPU kernel for long-context sparse attention. Motivated by the concentration of sparse-attention scores, it adopts a sample–filter–select workflow with score binning, online threshold updates, and local buffering to avoid materializing the full score matrix while preserving exact top-k selection. This design reduces intermediate memory traffic and the workload of the final selection stage. Experiments across multiple models and frameworks demonstrate kernel-level acceleration, reduced auxiliary memory usage, and end-to-end prefill speedups of up to 1.49×.
Critical strengths
I consider this work to have practical value. It fuses indexer scoring with top-k selection, reducing intermediate memory traffic while preserving exact top-k results. It demonstrates both kernel-level acceleration and actual end-to-end prefill speedups, along with reduced auxiliary memory usage, so the benefits are not limited to isolated kernel benchmarks. These improvements are meaningful for long-context inference and may also benefit training, although the paper does not demonstrate end-to-end training speedups.
Critical weaknesses
1.Modest novelty. The main contribution centers on kernel-level optimization of Indexer-TopK, while the algorithmic novelty appears relatively limited. I would like the authors to distinguish more clearly which design choices are new relative to existing selection methods and which are adaptations tailored to DSA. Given the paper’s emphasis, I personally feel that a systems or ML systems venue might be a more natural fit.
2.Insufficient comparison with other optimized top-k baselines. Comparisons against vLLM and SGLang are meaningful. However, while the paper states that the baselines are evaluated using their official implementations, it does not clearly specify the framework versions or the particular top-k backends used. This makes it difficult to determine which optimized implementations are already represented in the comparisons.
Questions for authors
1.Could the authors specify the versions of vLLM and SGLang and the top-k backends used in the experiments? Which optimized exact GPU top-k implementations are already covered by these baselines? For representative methods that are not covered, could the authors provide additional comparisons?
2.Could the authors provide targeted ablations to isolate the contributions of sampling, threshold refresh, and local buffering? In particular, how much does each component contribute to the overall performance gains, beyond the results reported for the complete implementation?