acceptodds
Withdrawn from ICLR 2027

Auditing KV-Cache Retention with Saved Attention Outputs

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Can saved attention outputs certify a retention decision after the original KV cache is released? We study the evidence needed for this task. For all group-retention outputs at one query, we attain the minimum native observation count at every Value width. We also derive finite-error stability bounds and an ill-conditioned family. With a retained reconstruction, an exact normalizer-residual identity constrains one shared physical Key-Value error using causal output records. It yields grid-free convex certificates for later queries, masks, and output directions. These inputs may be selected after recording. Known appended rows enter without additional uncertainty. Checking fixed coefficients is linear in cache size at fixed record count. A sensitivity condition and same-code source witnesses establish strict information gain. Frozen-model checks demonstrate this gain and expose a nonlinear-error-dominated regime. A matched-byte control favors finer Value quantization over saved outputs on the tested short prefixes. Thus the certificate quantifies the value of evidence for a declared workload; it also identifies when another storage allocation is preferable.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
KV cache, attention, optimal recovery, uncertainty quantification, certified inference, cache retention, information complexity, transformer inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_gR45OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper proposes a method for quantifying the stability of recovery from compressed or otherwise size-reduced KV records. The idea is to store a compact representation of the worst case error that could result from using the reduced size (possibly reduced to zero) version.

For fixed queries, storing the attention output of a group of rows for a specific query means the rows themselves can be discarded. For future queries, an upper bound on the error may instead be provided. Intuitively, for a softmax decision, the output may not differ even if there is significant error in the KV input.

Critical strengths

The high-level idea of quantifying KV compression or pruning error is interesting. After quantifying the error, better informed decisions can be made about what rows to evict, compress or otherwise reduce. If the error can somehow be constrained to an individual attention layer, and made sure to not propagate further through the model (which would create an indecipherable maze of small errors throughout the model), and if decisions were made on large stretches of tokens so that it does not get in the way of efficient GPU compute, then this could perhaps become a practical means of reducing both KV storage footprint and attention compute, as well as some forms of continuous attention-level compaction.

Critical weaknesses

The evaluation is surprisingly limited, testing on a small models, with fixed queries and very few examples. This part is not confidence inspiring at all. Moreover, the authors demonstrate no understanding of the costs of selective compression and quantization at an individual row level, or the cost of performing the proposed checking.

The writing is typical choppy AI prose, almost unreadable in many cases.

Questions for authors

What potential opportunities does this method create for deployments? The costs of certification, and operating at various quantization and compression levels would seem to be vastly higher than simply storing rows with higher precision and enjoying fast parallel compute.

Reviewer_KAbCOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates certifying KV-cache retention using saved attention outputs after the original KV cache is released. It establishes recovery bounds for fixed queries, a convex SOCP certificate for new queries, and a feasible lifting construction, with experiments on SmolLM2-1.7B demonstrating performance gains and quantization insights.

Critical strengths
  • The paper introduces a new well-posed problem with rigorous normalization-aware theory. Theorem 1 constructs mask families matching its lower bound for all Value widths. Theorem 2 handles practical later queries by reusing the normalizer and a shared perturbation across records. The guarantees work for post-selected queries/masks without independence or the original prefix, and analyses separate identifiability from numerical stability.

  • Experiments bound prior ambiguity and compares it with validated posterior enclosure, verifies quantization schemes, and reports mixed/negative results. This strengthens the claim of strict information gain.

Critical weaknesses
  • The empirical evaluation has limited scope and poor storage trade-offs. Experiments are only conducted on one small model, single layer/head, four manually designed contexts and a 32-row uncertain prefix.
  • The work better shows information gain under a fixed codec than practical storage benefits.
  • The paper does not separate certificate conservatism from coefficient-search loss, and Key quantization brings strong limitations.
  • There are no comparisons against full convex optima on small cases, nor evaluations on downstream generation quality.
Questions for authors
  • Because record storage is O(N_rec*(d_k+d_v)) and independent of prefix length n, while quantization is O(n*(d_k+d_v)), have you run the matched-byte complete-head comparison at n=256, 512, or 2048 (or with FP16-quantized records) to test whether saved records beat finer Value precision once the prefix is long enough to amortize the fixed record size?

  • Tightness versus search: on representative short-prefix cases, how close are the one-ray and multi-candidate intervals to the full convex endpoint optimum, and how much of the 222/576-to-511/576 gap is recoverable by coefficient optimization?

Reviewer_Q5xUOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies whether saved attention outputs can certify later KV-cache retention decisions after the original cache has been released. It provides generic minimum observation counts for exact fixed-query group recovery, finite-error stability bounds, and a normalizer-residual certificate that constrains one shared physical Key–Value reconstruction error across historical records. The certificate supports subsequently selected queries, masks, and output directions. A summary-space formulation additionally establishes endpoint exactness when a feasible physical lift exists.

Validation uses frozen SmolLM2-1.7B-Instruct caches, independently checked numerical bounds, same-code source witnesses, complete-head retention tests, record ablations, matched-byte comparisons, and longer-prefix scalar experiments. Records improve certification at a fixed codec, while finer Value quantization performs better in the tested short-prefix matched-byte comparison.

Critical strengths
  1. A coherent theoretical treatment of post-release auditing. The central certificate preserves the attention normalizer and explicitly bounds nonlinear Key–Value error terms, while requiring all historical records to constrain the same physical cache perturbation. The matching generic observation bounds provide a complementary information-theoretic result. These contributions establish a meaningful capability beyond directly comparing compressed outputs with an available original cache.
  2. Careful distinction between valid bounds, information gain, and decision improvement. The experiments separate coefficient proposal from outward verification and use feasible same-code witnesses to establish genuine information gain. Complete-head tests connect this gain to retention decisions. Reporting the unfavorable matched-byte result also makes the empirical conclusions appropriately balanced.
Critical weaknesses
  1. The novelty is insufficiently explained, and the writing and equation organization lack clarity.

Section 2 introduces foundations such as optimal recovery and convex duality, but does not sufficiently distinguish applications of established tools from new theoretical contributions. The abstract and introduction provide insufficient explanation of the motivation, central idea, and relationships among the results. Sections 4–6 contain dense equations and theorem statements with limited intuitive explanation. Some key equations and definitions are unnumbered, and multi-step derivations lack clear structure, making the arguments harder to understand and verify.

  1. The comparison with existing certification approaches is insufficiently developed.

Section 2 describes differences in information access and task settings, but provides limited comparison of the actual certification guarantees and computational requirements. The experiments primarily evaluate the informational value of historical records and storage allocation, without further establishing how the proposed certificate relates to the cited verification approaches. Consequently, it remains unclear which capabilities arise from the certificate construction and which primarily reflect changes in available information or task assumptions, limiting assessment of the methodological contribution.

Questions for authors

Please respond to the concerns raised in W1 and W2.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.