- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper compares KV-cache scoring rules and cache-budget allocation across layers on different models.
Critical strengths
The paper examines KV-cache eviction across five models and includes experiments on synthetic retrieval and real-world tasks, providing empirical observations across different model and workload settings.
Critical weaknesses
-
Poor presentation. The abstract is vague and the paper is hard to follow. Cumulative scoring, layer-wise allocation, and the configuration in Table 3 need clearer explanations and annotations.
-
Limited insight and theoretical analysis. The method offers little new insight and lacks substantive theoretical analysis beyond a simple formulation. The results vary across models without providing clear practical guidance.
-
Insufficient evaluation. The experiments lack a comparison with OBCache [1] and ablations on the calibration data used by the starvation probe.
[1] OBCache: Optimal Brain KV Cache Pruning for Efficient Long-Context LLM Inference
Questions for authors
-
Please clarify cumulative scoring and layer-wise allocation, and identify the proposed configuration in Table 3.
-
How do the choice and amount of calibration data affect the starvation-probe results?