- Rating
- 2
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates whether different KV-cache eviction methods should be assigned to different Transformer layers. It introduces POLYKV, a framework for constructing and executing static layer-wise plans using offline representation-preservation or perplexity profiles. Controlled experiments separate method routing, capacity allocation, and placement effects. The results support placement-sensitive routing in the evaluated settings.
Critical strengths
- The experimental design separates several potentially confounded factors. The routing-by-allocation factorial distinguishes method choice from cache capacity, while count-matched random placements isolate the contribution of layer assignment beyond the method mixture.
- The placement analysis provides evidence beyond a single favorable configuration. The guided route ranks second among count-matched assignments, and the frozen profile correlates with downstream performance across random controls.
- Static routing requires neither model retraining nor online search. The common execution interface enables composition of existing methods.
Critical weaknesses
- The performance gains appear modest and uneven across tasks. The evidence supports favorable average behavior, but does not yet establish a consistently useful improvement across workloads.
- Some comparisons do not fully isolate the benefit of layer-wise routing from differences in when compression is applied. This makes it difficult to attribute the observed improvements specifically to heterogeneous method assignments.
- The profiling procedure may not accurately capture a method’s suitability within a heterogeneous configuration. Local preferences can change when neighboring layers use different methods, while independent layer selection overlooks these interactions.
Questions for authors
How robust are the gains across repeated evaluations and task families, and what explains the tasks on which routing hurts performance?
Does routing retain its advantage when all baselines use identical compression phases and cache-management schedules?
How sensitive are method preferences to the surrounding assignments, and would profiling conditioned on the current route improve selection?