- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper improves threshold-based sparse decode attention by changing the traversal order of KV blocks. It uses previous-token block scores to prioritize likely high-scoring blocks, establishing a strong running maximum earlier. Skip decisions still use current-token scores. The paper analyzes the best traversal under a fixed skip rule and evaluates sparsity, task quality, attention-kernel performance, and decode throughput.
Critical strengths
S1. The paper clearly identifies how KV-block traversal order affects threshold-based skipping. The history-guided approach is simple and easy to understand.
S2. The fixed-score, fixed-threshold experiments help isolate the effect of traversal order. The paper also evaluates the tradeoff between task quality and sparsity.
Critical weaknesses
W1. The practical importance and incremental benefit over BLASST are insufficiently established. The motivation does not quantify the fraction of attention time and total decode time spent on traversal-induced extra value-side work over successive steps. The paper also lacks a direct performance comparison against BLASST at matched task quality, and the model-level throughput experiment omits BLASST. Headline speedups against Dense therefore do not isolate the benefit of the proposed traversal change.
W2. The throughput gain is not validated under a corresponding quality constraint. Quality experiments mainly use Qwen3-8B and Llama-3.1-8B, whereas throughput is measured on Qwen3-30B-A3B. No quality result is reported for that model at the corresponding sparsity setting. It is therefore unclear whether the reported throughput gain preserves task quality.
W3. Some quality comparisons use results taken from another paper; consistency of evaluation settings should be clarified. Table alignment and crowded half-width figures also need improvement.
Questions for authors
Q1. What fraction of attention time and total decode time is attributable to traversal-induced extra value-side work, and how much model-level throughput does HATS gain over BLASST at matched task quality?
Q2. What task quality does Qwen3-30B-A3B achieve at the 88.2% sparsity setting used for the reported throughput result?