- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper observes that the effect of KV-cache eviction in long-context LLMs is input-dependent: for some inputs, evicting redundant prefill tokens mitigates attention dilution and matches or even exceeds full-cache accuracy, whereas for inputs that rely on precise retrieval of a few critical tokens, eviction causes accuracy to collapse. Accordingly, the authors partition inputs into dilution-prone and capacity-bound classes, and show that a single label-free scalar computed from prefill attention, namely the early-to-late-layer drop in pairwise top-k (Jaccard) head agreement, D, predicts per input and before any decoding which class an input belongs to. Building on this, they propose PAGE (Partition-Aware Gated Eviction), a training-free wrapper that invokes any SnapKV-style evictor when D exceeds a threshold and retains the full KV cache otherwise. PAGE is positioned as a safeguard against harmful eviction rather than a compressor. Experiments show that on the capacity-bound task in RULER, PAGE reduces the harm rate under 16× compression from 0.75 to 0.026.
Critical strengths
- The problem addressed is valuable and complements the shortcomings of current methods.
- The gating signal is simple and the method is modular, applicable to any SnapKV-style evictor.
Critical weaknesses
- Limited novelty. Both the dilution mechanism and the head-agreement signal come from prior work; the contribution mainly amounts to using them as a threshold gate around existing evictors.
- The threshold τ is set empirically and transfers poorly across models, requiring recalibration for each new model.
- Limited datasets. The main experiments are conducted on RULER, and validation beyond RULER is small in scale, so generalization to other tasks remains insufficiently validated.
- Limited model coverage. The main experiments focus on Qwen2.5 and Mistral-7B, with only limited validation on other model families, lacking validation on larger and more recent models, as well as on newer attention mechanisms and architectures (e.g., hybrid models, Multi-head Latent Attention (MLA), and DeepSeek Sparse Attention (DSA)).
Questions for authors
- How does PAGE perform on tasks beyond RULER?
- How does PAGE perform on larger and more recent models?
- For hybrid models or models with newer attention mechanisms such as MLA and DSA, does PAGE remain effective?
2 comments
Discussion Reminder
Thank you again for your time. We hope our responses have addressed your concerns. We would appreciate your reassessment and are happy to clarify anything further.
Response to Reviewer GqTU
Thank you for the review. We clarify the model and workload coverage below.
PAGE adds an input-level routing decision to existing eviction methods. Its prefill head-agreement signal identifies a failure-prone task class without labels at inference and outperforms the evaluated gates based on SnapKV, H2O, and OBCache scores (Appendix I.5, Table 41). It achieves 93.8% of a hindsight task-level policy's gain, allowing fractional routing within the baseline's final task to match the closed fraction (Appendix I.6, Table 42). The dilution mechanism and head-agreement statistic build on prior work, as acknowledged in the paper. Our response to Reviewer rro6 discusses these comparisons in more detail.
The standardized gate accounts for differences in the scale of D across models. It separates NIAH-MK3 from the other tasks in seven Qwen2.5, Mistral, Llama, and Yi cells, including the two held-out model families (Appendix A.3.4). Its selection gain is at least as large as the fixed threshold's on every model in Table 36, increasing from +0.3pp to +12.4pp on Qwen3-4B. Further results cover Qwen3-14B (Appendix I.7, Table 43), two MLA models, and three hybrid models (Appendix I.8, Table 44).
This supports transfer on the reported models under the evaluated calibration conditions. Applying the standardized threshold requires an unlabeled pilot of approximately 100 inputs per model, although fitting the standardized cutoff used task labels. Pilot composition affects calibration (Appendix I.4, Table 40), so we do not claim a universal threshold. Section 6 also documents failures of the fixed threshold τ of 0.07.
The three LongBench subtasks in Table 12 test classification before decoding. They do not establish selection over a random gate. We added a pre-registered experiment on NQ multi-document QA and BABILong to test selection on real documents. These tasks are harmed by eviction but have among the highest D scores in their workloads, and the gate achieves no positive selection (Appendix I.9, Table 45, “Selection” and “Holm p” columns). These near-tie tests do not establish positive selection. The other real-document runs (Appendix I.11, Tables 47- 49) show isolated gains on L-Eval, hotpotqa and QuALITY that do not hold on a second model.
The submitted evaluation already included Qwen2.5-32B, Qwen3-4B, Llama-3.1-8B, and Yi-1.5-9B (Appendix A.4.7). We now add explicit selection controls. The standardized gate gains +12.0pp, +12.4pp, and +4.0pp over random fallback on the first three models, respectively (Table 36).
We also added a pre-registered Qwen3-14B experiment (Appendix I.7, Table 43). The fixed threshold opens on 3% of inputs, so its gain is largely due to full-cache fallback. The standardized gate retains the full cache for 84% of NIAH-MK3 inputs, uses approximately half the cache overall, and gains +7.5pp [+5.9, +9.0] over the random gate.
We hope these results answer your questions and would be grateful if you could consider raising your score.