- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper investigates whether successful reasoning-time KV cache compression requires identifying a uniquely important subset of states. With the prompt cache preserved and generated-state budgets matched, random page retention performs competitively with importance-based methods across four models and three reasoning domains. Analyses of successful random runs, selector overlap, and shared-core interventions suggest that multiple distinct cache configurations can support successful reasoning, although the mechanistic evidence is local to the evaluated states.
Critical strengths
-
The research question is insightful. Beyond comparing accuracy under compression, the paper asks whether successful reasoning depends on a uniquely important cache subset, offering a useful perspective on the role of importance-based selection.
-
The main experiments are well controlled. All compression methods preserve the full prompt and receive the same generated-state budget, reducing confounding from protection policies and cache capacity. The evaluation covers four models and three reasoning domains.
-
The paper provides complementary mechanistic evidence. It combines multiple retention seeds, same-state cache overlap analysis, and shared-core interventions, while correcting for overlap expected from random sampling. This provides stronger support for solution multiplicity than accuracy comparisons alone.
Critical weaknesses
-
Substantial overlap with Random Attention requires clarification. The proposed retention strategy closely resembles that of Random Attention [1]. The paper only employ block-level random selection instead of token-level ones. Moreover, all six shared Full-KV accuracy scores are identical, and the reported mean generation lengths for Qwen3-14B coincide across all three shared tasks. These observations do not establish inappropriate reuse, but the authors should clarify the relationship between the papers, disclose any shared experimental results or infrastructure, and explain the distinct contributions of this work.
-
Presentation quality needs improvement. The budget table in Section 4.2 lacks a caption. The main overview figure appears to be rasterized; a vector version would improve readability and presentation quality.
-
Experimental coverage and reproducibility details are limited. The capacity sweep and retention-granularity ablations are restricted to a single model–task setting. The paper also lacks end-to-end latency, peak memory, and throughput measurements, leaving the practical efficiency implications unquantified. Important implementation details, including the recent-window size and eviction-triggering rules, should be specified.
[1] Random Attention: Rethinking KV Cache Eviction for Efficient Reasoning
Questions for authors
Seeing Weaknesses. The authors should expain the relationship between the paper with Random Attention.