acceptodds
Under review as a conference paper at ICLR 2027

Restoring Evidence Routing Does Not Restore Exact Matching in Compressed KV Caches

Abstract

Exact lookup requires both access to a record and discrimination of its complete address. We test record recovery, targeted restoration, and source-specific matching under query-agnostic KV-cache compression. Phase 1 reveals a recovery–decision gap in 45 of 144 positive queries under prefix-only attention selection. Both complete source records are independently recoverable from the same cache in these cases, yet the existence readout denies a match. Phase 2 uses independently coded records and equal-byte substitutions of evicted primary-record versus matched nonprimary positions. Primary restoration reduces false absence from 64/96 to 7/96 relative to sham, but also increases near-miss errors. Phase 3 separates the two sources factorially. Restoring A predominantly changes A-queries, and restoring B changes B-queries, yielding a predeclared crossover of percentage points with a conservative lower bound of points. Joint restoration reduces false absence by points while increasing one-field false alarms by points. Full and joint restoration both reject all 48 far negatives, yet accept 61 and 55 of 96 near negatives, respectively. This source-selective overmatching separates reinstated source influence from exact Boolean comparison. The resulting diagnostics separately measure record recovery, behavioral fidelity, and query discrimination.

Primary area:
interpretability and explainable AI
Keywords:
KV-cache compression, activation interventions, evidence use, source-selective overmatching, long-context evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_bJsNOpenReview ↗
Rating
1
Confidence
1
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper addresses a potentially interesting distinction between record recoverability, source-dependent decision behavior, and exact-match discrimination under KV-cache compression. However, the presentation makes the contribution and experimental logic difficult to access.

Critical strengths

None

Critical weaknesses
  1. The task setup and key concepts, including record address, primary and non-primary positions, restoration, source crossover, and source-selective overmatching, need clearer definitions, and a concrete example.
  2. Each experimental phase should clarify its hypothesis, intervention, control, and the inference.
  3. The numerical results lack content.
  4. The presentation is rather poor, and difficult to follow. The paper needs a major revision.
Questions for authors
  1. please clarify the task setup, and key concepts including record address, primary and non-primary positions, restoration, source crossover, and source-selective overmatching. It would be good to give a concrete example.
  2. Please clarify hypothesis, intervention, control, and the inference.
  3. Please provide physical meaning of the numerical results.
Reviewer_nWYEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the distinction between record recovery, source-specific influence, and exact matching under query-agnostic KV-cache compression. Three synthetic studies use independent readouts and equal-byte cache substitutions. In the main factorial experiment, restoring a source increases acceptance of both its exact queries and its one-field near misses. The results demonstrate that restoring the full-cache model's behavior need not restore correct lookup. Evaluation uses Qwen2.5-7B-Instruct, with 24 worlds per study.

Critical strengths

S1. The intervention is carefully controlled. Common donor removals, equal storage, source quotas, and independent cache clones address several obvious confounds. The large source crossover is a convincing result within the tested setting.

S2. The evaluation distinguishes fidelity from correctness. Exact/near query pairs reveal errors hidden by aggregate accuracy or agreement with Full. World-level analysis and explicit treatment of invalid responses are also appropriate.

Critical weaknesses

W1. The contribution remains a narrow behavioral diagnosis. The factorial design is useful, but the central result is established for one model, synthetic databases of roughly 3-4K tokens, and approximately 70% retained storage. The main intervention uses SNAP-PFX; CHUNK16 and random retention appear only in the earlier studies. There is no comparable validation with a dedicated query-agnostic method such as KVzip or Compactor. More importantly, Full already accepts 61/96 near negatives, while joint restoration accepts 55/96 and agrees with Full on 225/240 decisions (Table 4; Appendix F). Thus the study shows restoration of an already unreliable model, rather than a compression-specific failure of an otherwise accurate comparator. The authors acknowledge this distinction, but the limited evidence leaves the broader significance insufficiently established. A diagnostic paper need not propose a new algorithm; here, stronger generalization or a more discriminating explanation is needed to make the empirical contribution substantial.

W2. Important validation remains incomplete. Appendix C checks two split-execution paths but explicitly does not compare split Full with monolithic execution on the task. Given Full's high near-miss error rate, this is a consequential missing control, although it does not establish an implementation bug. The reproducibility statement also says the original model-level cache-manipulation runner is not distributed; recomputing statistics from saved outputs cannot independently validate the interventions. In addition, the Phase 2/3 cohorts change the coding and query design, and Phase 2 starts with no exact recoveries under SNAP-PFX. The later causal result therefore does not directly explain the recoverable denials that motivate Phase 1. These gaps reduce confidence in the broader interpretation rather than invalidate the reported crossover.

Questions for authors

Q1. Does monolithic Full reproduce the reported near-miss errors and exact/near decisions? Please report a direct comparison on the Phase 3 inputs, and clarify what runnable code is available to regenerate the cache interventions and readouts.

Q2. Does source-selective overmatching persist for another model or retention policy, particularly in a setting where Full reliably distinguishes exact from near matches? Such evidence would materially strengthen the significance of the result.

Reviewer_F7JJOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how query-agnostic KV-cache eviction (SNAP-PFX, a prefix-only adaptation of SnapKV scored on the final 64 tokens of the database prefix at 69.5%–70.0% cache retention) affects full-record recovery versus exact 4-field Boolean address matching (HAS_MATCH vs. NO_MATCH) on closed-world synthetic databases using Qwen2.5-7B-Instruct.

Across three sequential 24-world studies (with independent cache clones per readout head so recovery outputs never enter decision prompts), the authors report:

  1. Phase 1 (Recovery–Decision Dissociation): On identical compressed caches under SNAP-PFX, independent recovery heads extract the complete 6-field tuples for Source A (135/192) and Source B (134/192), yet the existence readout denies a match (NO_MATCH) on 100/144 positive queries. In 45/144 positive queries, both complete source records are recovered verbatim from the same cache while the existence query is denied (vs. 16/144 for FULL, a predeclared difference of 20.14 pp).
  2. Phase 2 (Equal-Byte Primary Restoration vs. Sham): Using independently coded fields (where SNAP-PFX recovers 0/48 primary records before intervention), substituting evicted primary-record K/V states in place of low-scoring retained "donor" positions reduces false absence from 64/96 to 7/96 relative to a position- and score-matched non-primary sham substitution, while increasing 1-field near-miss false alarms by 27 (out of 48 near-negative queries; Table 3 and Section 4.2, lines 260–264).
  3. Phase 3 (2×22 \times 2 Source-Factorial Intervention): Crossing Source A and Source B restoration (SS, AS, SB, AB\text{SS, AS, SB, AB}) while holding donor removal, persistent bytes, and per-source token quotas fixed yields a predeclared 79.17 pp source-routing crossover TT (positive in all 24 worlds; conservative Hoeffding lower bound 14.13 pp). Restoring A selectively increases HAS_MATCH on Exact-A (+70.83 pp) and 1-field Near-A (+44.79 pp) with negligible cross-source change on Exact-B (-1.04 pp) or Near-B (+2.08 pp); restoring B mirrors this (+84.38 pp on Exact-B, +52.08 pp on Near-B). Joint restoration (AB) agrees with uncompressed FULL on 225/240 literal decisions (93.75%), including 51 shared near-miss false alarms, while rejecting 48/48 far negatives.
Critical strengths
  1. Methodologically Exemplary Causal Cache Intervention (2×22 \times 2 Source-Factorial Design with Donor-Matched Sham Controls): The Phase 3 experimental design (Section 3.2, Equation 2, Appendix C) is a model of rigor for activation/cache patching in long-context transformers. By fixing a common retained backbone K0=K∖(DA∪DB)K_0 = K \setminus (D_A \cup D_B) across all four arms (SS, AS, SB, AB\text{SS, AS, SB, AB}), matching sham non-primary positions SsS_s to primary-record positions WsW_s lexicographically by original position distance and selector score (Table 7, median score residual 8.3×10−68.3 \times 10^{-6}), preserving exact original RoPE coordinates, and evaluating 17 readout heads on isolated cache clones, the study cleanly isolates the causal effect of primary-record K/V states from storage budget, positional density, or prompt-continuation leakage.

  2. Thorough Treatment of Output Encodings and Unresolved Completions: Testing literal HAS_MATCH/NO_MATCH (F) alongside reversed binary code mappings (CODE01 and CODE10, Table 10) and computing worst-case logical completion bounds over invalid/unresolved outputs (Appendices D, I, J) ensures that the 79.17 pp source-routing crossover is a genuine source-selective internal state effect rather than an affirmative label bias or formatting artifact.

Critical weaknesses
  1. Confound Between Base-Model Capability and Restoration Limits (FULL Already Fails on 61/96 Near Negatives):

    • While the authors candidly note in Sections 4.3 and 5 (and Appendix F) that FULL is a behavioral reference rather than an exact-lookup oracle and that joint restoration (AB) reproduces FULL's decisions on 225/240 (93.75%) of queries (including 51 shared near-miss false alarms), the paper's title ("Restoring Evidence Routing Does Not Restore Exact Matching in Compressed KV Caches") and headline framing invite readers to conclude that compressed-cache restoration fails to recover exact matching.
    • In Table 4 and Table 10(a), the uncompressed FULL baseline already accepts 61 out of 96 (63.5%) one-field near negatives (33/48 Near-A and 28/48 Near-B), achieving only 31/96 on QPAIR and 175/240 overall, while AB achieves 55/96 near false alarms, 36/96 on QPAIR, and 180/240 overall.
    • Because Qwen2.5-7B-Instruct itself relies on partial-match source familiarity rather than strict 4-field Boolean conjunction on this task even with an uncompressed cache, this experimental setup cannot distinguish between two fundamentally different hypotheses:
      1. A genuine limitation of KV restoration (where an uncompressed model possesses exact 4-field discrimination, but restoring primary-record K/V states into a compressed cache only recovers coarse source routing), versus
      2. Successful behavioral recovery of a base model that never performed exact matching in the first place.
    • Conversely, SNAP-PFX and SS only exhibit low near-FP rates (7/96 and 9/96) because they evicted the primary records and defaulted to NO_MATCH on ~80% of all queries (missing 77/96 and 78/96 positive queries).
  2. Narrow Empirical Scope (Single Model, Mild 70% Retention, Prefix-Tail Selector) and Incomplete Reproducibility Release:

    • Single model (Qwen2.5-7B-Instruct): All three phases evaluate only one 7B checkpoint where FULL fails on 63.5% of near negatives. Evaluating at least one stronger model where FULL achieves low near-miss false alarms is necessary to test whether primary-record K/V restoration in a compressed cache ever fails to recover exact matching that the uncompressed model actually possesses.
    • Mild compression budget (69.5%–70.0% retained) and SNAP-PFX pathology: Evicting only 30% of a 3.0K–3.7K token cache is a very mild compression setting. SNAP-PFX loses the primary records at 70% retention (0/48 recovery in Phase 2; 3/48 A and 0/48 B recovery in Phase 3, Table 4) because it applies SnapKV's 64-token observation window to the end of the database prefix (which contains auxiliary records at the end of Source B with zero lexical overlap with the independently coded primary records). As shown in Phase 1 (Table 2) and Phase 2 (Table 3), simple CHUNK16 or RANDOM retention at the same 70% budget cuts false absences by more than half, yet Phase 3 is restricted to SNAP-PFX.
    • Inconclusive Hoeffding bounds on QPAIR and Near-FP: With N=24N = 24 worlds, the conservative Hoeffding union-bound intervals in Table 5 and Figure 3 span zero for two of the four predeclared contrasts (Qpair: AB - SS is [-36.91, 93.16] and Near FP: AB - SS is [-17.12, 100.00]), relying on the exploratory non-studentized bootstrap to exclude zero.
    • Missing model-level cache-intervention code in the supplement: The Reproducibility Statement (lines 421–423) and Appendix K (lines 991–993) note that the supplement only provides saved readout tables and CPU scripts to recompute summary statistics, omitting the actual model-level KV-cache manipulation runner. Recomputing statistics from saved outputs does not allow independent reproduction of the neural cache interventions.
Questions for authors

Clarification Requests

  1. Reframing the Title and Main Claim Around Behavioral Fidelity vs. Base-Model Capability: Since AB (70% compressed cache with primary-record K/V restored) matches FULL on 93.75% of all queries and actually achieves a lower near-FP rate (55/96 vs. 61/96) and higher QPAIR (36/96 vs. 31/96) than FULL (Table 4), do you agree that the cache restoration itself succeeded in recovering full-cache behavior, and that the inability to reject 1-field near misses is a property of Qwen2.5-7B-Instruct on this task rather than a degradation caused by KV-cache compression?
  2. Behavior on a Model Where FULL Performs Exact Matching: Have you tested Phase 3 on a stronger model (e.g., Qwen2.5-14B/32B-Instruct, Qwen3-8B, or Llama-3.1-8B-Instruct) where FULL has a low near-FP rate on 1-field mismatches? On such a model, does AB also recover FULL's low near-FP rate, or does compressing the surrounding 30% (or 70–80%) of auxiliary context cause AB to overmatch where FULL does not?
  3. Why Did Phase 1 Recover 135/192 and 134/192 Primary Records Under SNAP-PFX While Phase 2 Recovered 0/48 and Phase 3 Recovered 3/48 and 0/48?: In Phase 1, SNAP-PFX recovered both primary records in ~70% of attempts (Table 2), motivating the "same-cache recovery–decision dissociation." Yet in Phase 2 and Phase 3, changing to independently coded fields caused SNAP-PFX recovery to drop to ~0% (0/48 and 3/96). Does this mean the Phase 1 recovery–decision dissociation was an artifact of shared field codes across records in Phase 1 (allowing the recovery head to guess or piece together fields from auxiliary records)?

Suggestions for Future Work

  1. Testing True Query-Agnostic Compressors at Realistic Budgets (10%–25% Retention): Evaluating Phase 1 and Phase 3 under genuine query-agnostic compressors (e.g., KVzip context reconstruction, KeyNorm, or ExpectedAttention) at 10%–25% cache retention would test whether purpose-built query-agnostic methods exhibit the same recovery–decision gap or source-routing crossover.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.