- Rating
- 2
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SIEVE, a training-free clustering method for sparse attention. It combines a robust query-induced metric with retrieval-target-aware weighting to improve cluster-based retrieval and estimation. The authors derive cluster-level estimation-error bounds and evaluate the method on RULER, LongBench, AIME-25, and GPQA Diamond.
Critical strengths
- Well-motivated methodology. Section 4.1 and Appendix D identify and empirically characterize the mismatch between angular similarity and attention-logit geometry. Designing clusters around query-dependent errors is a meaningful contribution to retrieval and estimation.
- Controlled comparisons and informative ablations. The shared evaluation framework and matched retrieval budgets support comparisons with RetroInfer. Table 14 further shows that adding the target-aware term reduces the reported estimation-zone numerator error from 59.8 to 43.1 while improving task accuracy.
Critical weaknesses
- Limited connection between the theoretical bounds and final attention-output error. Appendix A.2 correctly extends Proposition 4.4 to non-mean centroids, with Equations (5)–(6) remaining valid and an additional first-order term in the log-mass ratio. However, Equation (6) alone does not control the query-dependent exponential factors retained in the estimation bounds. Table 14 reports estimation-zone numerator error, leaving the corresponding effect on the final normalized attention output insufficiently characterized.
- Modest aggregate gains and limited evidence of robustness. The macro-average gains over RetroInfer are modest on RULER (0.8–1.9 points) and LongBench (0.3–0.5 points), although Table 2 shows larger gains on some challenging RULER subtasks. Table 19 indicates comparable decoding speed and peak memory. The practical benefit therefore rests primarily on improved quality, but the stochastic reasoning gains lack uncertainty estimates and their attribution to the target-aware component remains unclear.
Questions for authors
- Can the authors compare robust-metric-only clustering with full SIEVE on identical queries and KV states, reporting signed relative estimation-zone mass error and relative L2 error of the final attention output?
- What motivates omitting target-aware weighting for newly generated keys? Please assess its contribution to reasoning by disabling it for the prefill index while keeping online indexing unchanged. Separately, please report paired, problem-level confidence intervals for the SIEVE–RetroInfer score differences.