Utility, Coverage, and Exact Residency Can Rank KV-Cache Policies Differently
Abstract
What should a compressed KV cache preserve? The answer depends on the claim: behavior on a target query, retained annotated content, or intact original slots for a literal-residency claim. These are different targets, yet KV-cache evaluations often treat answer quality or token coverage as evidence for all three. FACT-KV aligns tested-query utility, original-position coverage, and exact annotated-unit residency on the same selected state. In a prospectively frozen 500-source Mistral/HotpotQA contract, cumulative retention improves F1 over recency by 0.220 and coverage by 0.086, yet reduces all-units completeness by 0.348. The conflict also occurs within sources: 70 show both higher F1 and lower completeness. An official HotpotQA re-score of the frozen predictions gives similar F1 (+0.217) and EM (+0.198) advantages. A placement reference reproduces much of the residency contrast, implicating retention geometry rather than semantic destruction. Capacity and granularity stress tests move exact residency systematically in the tested settings, without a corresponding monotone improvement in answer utility. A contract-adapted H2O audit, together with Llama, reuse, attribution, and repair audits, establishes the boundary: exact annotated-unit residency is a preservation-audit target, not a utility surrogate. Preservation claims should name the target, report its direct readout, and declare the layer/head aggregation rule.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.