acceptodds
Under review as a conference paper at ICLR 2027

Positions Are Not Facts: The Mismatch Between KV Caches and Memory

Abstract

When a fact changes, how should a language model update the history stored in its key–value (KV) cache? Hiding the old record is cheap, but it may still contain needed details or answer questions about the past. We compare hiding whole records, hiding only replaced values, and deleting old text and recomputing the cache. In a controlled quantity task, masking makes all eight models prefer the new value more strongly, yet six lose complete answers through unit errors or failure to stop; keeping the unit preserves all current answers. Later states also retain useful information from earlier records: on multi-hop updates, rebuilding these states at unchanged positions lowers historical accuracy by 20–41 percentage points, whereas moving the existing states has little effect. Keeping object dependencies and unchanged revision passages prevents many losses. Recognition is a separate challenge. Learned readouts recover distinctions missed by fixed cache similarities on synthetic record pairs. On natural text, text-detector-selected masks show no clear advantage over random masks at the same rate in 14 same-model detector–generator comparisons. Query-dependent access can avoid some losses, with additional storage or access costs. These findings identify what must be preserved beyond the replaced value when using a KV cache as updatable memory.

Primary area:
foundation or frontier models, including LLMs
Keywords:
KV cache, memory updates, long-context language models, knowledge updates, cache recomputation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_oB6mOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether KV Cache can serve as an updatable long-term memory. The central challenge is that when new knowledge replaces outdated information, directly deleting or modifying the KV states associated with the old knowledge may also damage useful historical information. Through a series of controlled interventions, the authors compare strategies such as Whole-source Masking, Value-only Masking, and KV Recomputation, and analyze their effects across multiple models and tasks.

Critical strengths
  1. Clear Motivation: This paper studies the problems of using KV Cache as an updatable memory.
  2. Comprehensive Experiments: The experiments cover a wide range of models, tasks, and KV Cache updating strategies, with detailed controlled experiments.
  3. Interesting Findings: The paper reveals several interesting phenomena, such as KV Cache updates potentially damaging historical information and subsequent states retaining outdated information.
Critical weaknesses
  1. Unclear Necessity of KV Cache Editing: Although Appendix D.7–D.9 evaluates the cost of KV Cache updates, the paper lacks a sufficient comparison with Text Editing + Re-prefill under optimized deployment settings. For example, the official vLLM benchmark [1] reports a median TTFT of 763.6 ms on a single H100 with Llama-3.1-8B, approximately 20K input tokens, and a concurrency of 8.

  2. Fragmented Empirical Analysis: The paper includes many experiments, but the connections between them are not sufficiently clear. Although individual findings are analyzed, they do not form systematic scientific insights. For example:

    • Figure 2: Value-only Masking achieves perfect accuracy on the Quantity Task.
    • Table 1: The same strategy fails to consistently handle both current and historical questions on MQuAKE.
    • Figure 3: Source Masking produces opposite effects on Qwen and Gemma.

    The authors provide some local explanations but do not sufficiently identify general patterns across tasks and models.

  3. Limited Analysis Beyond Observed Phenomena: Some conclusions remain at the level of empirical observations. For example, in Section 6.1, Learned KV Readout achieves an AUC of 0.862, but Detector-selected Masking does not clearly outperform Random Masking on natural text. Although the authors distinguish Recognition and Intervention in controlled experiments, they lack a systematic analysis of how these two types of errors contribute to the results on natural text.


[1] The State of FP8 KV-Cache and Attention Quantization in vLLM

Questions for authors

See Weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer oB6m (1/2)

Dear Reviewer oB6m,

Thank you for your review, and for finding the motivation clear, the experiments comprehensive and the findings interesting.

W1. "Unclear Necessity of KV Cache Editing"

Prompt Cache, CacheBlend, EPIC and KVLink serve requests from stored KV caches; editing a cached history in place is a natural way to update memory without re-reading it.

We share your question about whether cache editing pays off. In our experiments it is hard to make work well, and deleting the old text and re-prefilling (Recompute in Tables 1 and 3) faces the same problem: both must decide what to keep.

Since Appendices D.7 and D.9 do not use an optimized kernel, we added a FlashAttention 3 test: Llama-3.1-8B-Instruct on one H200, one quantity fact at 10%, 50% or 90% of a 20K-token history, then its current and historical questions, each asked 16 times (3-run medians; ranges over positions and re-prefill variants):

OperationTime to applyCurrent answerHistorical answerUpdate + 32 requests
Mask whole record28–29 mswrong unithits token limit111–112 s
Mask old value29–32 mscorrectwrong number11.0–11.2 s
Delete record, re-prefill150–751 mswrong unitwrong number7.3–9.0 s
Delete value, re-prefill151–757 mscorrectwrong number8.8–9.5 s
  1. Optimized re-prefill can still lose many answers. Deleting the old record loses the unit here too. A faster kernel changes how fast the cache is rebuilt, not what it is rebuilt from, so the main-text losses carry over:

    • On MQuAKE, re-prefill lowers historical accuracy by 20.2–41.0 points relative to masking (Table 1).
    • On 60 unit-dependent quantity questions, re-prefill after deletion leaves Qwen3-32B with 5.0% (3) complete answers (right number and unit, normal ending), vs. 60.0% (36) with masking (Section 4).
  2. Cost depends on where the fact sits and how the edit changes generation. In your vLLM setting (Llama-3.1-8B-Instruct, ~20K tokens, 8 concurrent requests, prefix caching), TTFT after a one-word edit is 0.55 s when the fact is at 90% of the history, 1.9 s at 50% and 2.9 s at 10%, since only the prefix before the edit is reused. In the table above, masking the whole record takes ~30 ms, yet over 32 requests it is slowest (111–112 s) because its historical answers hit the token limit; Appendix D.9 shows the same for both Gemma models.

We have added both measurements to Appendix D.9 (Tables R4–R5) and Section 6.2.

W2. "Fragmented Empirical Analysis"

Thank you for this suggestion; we agree that the links between the experiments were not clear enough, and we have made them explicit at the end of Section 2 and in Section 8.

What connects the experiments is the mismatch in our title: hiding an outdated record's tokens is not the same as removing the outdated fact. An edit is exact about which positions it hides, but not about which information it removes, because a fact is not confined to its own positions. Your three examples show this:

  • Figure 2: hiding removes needed information. Masking the whole record also hides its unit, and six of eight models lose complete answers; masking only the old number keeps all 60 complete in every model.
  • Table 1: hiding leaves the old fact in later states. Rebuilding these states with the old records hidden (Table 2) raises Qwen3-32B's current accuracy from 84.2% to 95.7% but cuts historical accuracy from 82.3% to 44.8%. The leftover hurts current answers and supports historical ones, so no operation in Table 1 is best for both.
  • Figure 3: the net effect depends on the model. Masking raises Gemma from 29.5% to 49.6%, mostly by fixing outdated answers, but lowers Qwen from 90.1% to 80.4%. Yet Gemma's gain hides more damage: on the 109 questions both answer correctly without masking, masking keeps 96.3% for Qwen but only 88.1% for Gemma.

The general patterns are:

  • Across tasks, an edit fixed before seeing the question cannot know what a later question still needs from the old record. Edits avoid losses only when they keep that part (the unit, object movements, unchanged passages; Sections 3 and 5) or choose tokens after seeing the question (Section 6.2).
  • Across models, the same edit fixes outdated answers and breaks good ones; preference and overall accuracy can rise while complete answers fall (Figures 2 and 3), so edits must be judged by answers fixed vs. broken per question.

No edit we tested works across all tasks and models, so we do not claim a general update rule. Our results instead caution against treating a KV cache as a memory that can be updated by masking or deleting an old fact's positions.

AuthorsOpenReview ↗

Response to Reviewer oB6m (2/2)

W3. "Limited Analysis Beyond Observed Phenomena"

Thank you for this point. Appendix B.12–B.14 relates detector decisions to generated answers on natural text, but does not combine the two error types or count the cost of false alarms; we do both here.

On natural text, even a perfect detector would fix only 9 of the 116 changed answers, and recognition errors keep every detector at a net gain of 3 or less. Intervention errors appear when a question still needs the old record.

The 0.862 AUC is for a learned KV readout on synthetic record pairs. Natural-text masks are chosen by text detectors; Qwen3.8 correctly classifies all 300 synthetic test cases (Section 6.1).

A detector can err in two ways: it can miss a real change, or it can flag an unchanged record and hide information that is still correct. The 116 natural-text questions (OAKS) all contain a real change, so they reveal only misses. Appendix B.11 pairs each question with two controls in which nothing changed (the later passage repeats the earlier one or comes from another book). We have now generated Qwen3.8-27B answers for all 348 cases, so false alarms can break answers.

Masking every real change would fix 9 answers and break none, so a perfect detector gains 9. Masking every control would break 52 correct answers. Seven detectors, each reading the two passages alone or with the question, give 14 selectors; each masks the earlier passage when it flags a change, and Qwen3.8-27B answers every question:

Detector, inputChanges flagged (of 116)Answers fixed (of 9)False alarms (of 232)Answers brokenNet change
Qwen3-4B, passages352113−1
Qwen3-4B, + question713176−3
Qwen3-8B, passages5952860
Qwen3-8B, + question78380+3
Qwen3-32B, passages352120+2
Qwen3-32B, + question74380+3
Qwen3.8-27B, passages170000
Qwen3.8-27B, + question37100+1
Gemma 4 26B-A4B, passages16110+1
Gemma 4 26B-A4B, + question55300+3
Gemma 4 31B, passages40210+2
Gemma 4 31B, + question75300+3
Llama-3.1-70B, passages291310
Llama-3.1-70B, + question71321+2

Qwen3-8B on passages alone also fixes one control, so its net change is 0.

  1. Recognition errors. No selector gains more than +3 of the possible +9:

    • Strict detectors such as Qwen3.8 raise no false alarms but flag only 17–37 of the 116 changes.
    • Looser ones such as Qwen3-4B and Qwen3-8B flag 35–78, but their false alarms break up to 6 correct answers.
  2. Intervention errors. The natural-text questions ask about the state after the change, and no correctly flagged change broke an answer. Historical questions still need the old record: with true replacement labels, whole-record masking lowers historical accuracy from 100% (300/300) to 93.3% (280) for Qwen3.8 and 82.3% (247) for Gemma 4 31B (Table 37, Appendix B.13). In Wikipedia revision histories, masking only the known changed passages lowers complete answers from 68.3% to 66.2% (Qwen3.8) and from 54.9% to 52.1% (Gemma 4 26B-A4B), a small but significant drop on questions that mostly ask about earlier versions (Section 5.2).

We have added this breakdown and the new answers to Section 6.1 and Appendix B.14 (Table R2).

All changes in the revised PDF are marked in blue. We hope these results address your concerns, and we would welcome further discussion.

Best regards,

The Authors

Thank you for the thoughtful and detailed rebuttal. I appreciate the authors' additional experiments and their candid discussion of the limitations. The new analyses address several of my concerns.

I also see potential long-term value in KV Cache editing, especially for future models that autonomously manage and update their own contexts. However, its practical advantages over optimized re-prefilling remain limited at this stage.

Overall, the rebuttal has improved my assessment of the paper, but I will maintain my original score of 2 (I would lean toward a score of 2.5 if that option were available).

Dear Reviewer oB6m,

Thank you for reading our rebuttal carefully and for recognizing our new experiments and analyses.

We completely agree that optimized re-prefilling is widely used in practice; frontier agent harnesses, including Codex and Claude Code, rely on context compaction and re-prefilling every day. And yet, KV-cache management still gives us plenty of headaches in everyday use :)

We only want to clarify in good faith that our goal is not to propose a KV-cache editing algorithm to replace re-prefilling, but to show through experiments where the growing practice of KV-cache reuse and in-place editing works and where it falls short.

As our results show, in-place editing and compaction each have their own strengths and blind spots. The two approaches do not have to compete, and can be used together: depending on the question, a system can reuse cached prefixes, mask only replaced values, and re-prefill when later states need to be updated.

We hope these findings help the community design future context-management systems with a clearer sense of what each operation keeps and what it loses.

Thank you again for your time and constructive feedback throughout the review process.

Best regards,

The Authors

Reviewer_G6KKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper is, unfortunately, not in a sufficient state of clarity for me to write a coherent summary.

Critical strengths

None

Critical weaknesses

I think this paper is not ready for submission yet due to the following writing and scientific weaknesses:

  • ambiguous references (e.g., "all eight models" in the abstract),
  • no problem statement and formulation (e.g., what is it that you are editing? individual tokens? what is the objective?),
  • lack of central arguments and research questions,
  • sentences that have no clear contribution to any argument,
  • unclear experimental setup (e.g., the authors did not even properly introduce/cite datasets and list the models and baselines they use),
  • figures with unclear purpose and content,
  • excessively long appendices that cannot address the concerns above.

I could list individual problems in more details, but it is perhaps not a productive pursuit as much as the authors reflecting on their paper and fundamentally improving its clarity and content, before resubmission at some other venue.

Questions for authors

I would like not to ask any question, since no answer from the authors will change my assessment above.

4 comments
AuthorsOpenReview ↗

Response to Reviewer G6KK (1/2)

Dear Reviewer G6KK,

Thank you for taking the time to review our paper. We start with a brief overview of what it studies, then go through your points one by one.

This is an empirical paper about what happens to a model's answers when a fact changes and the outdated record in its KV cache is hidden or recomputed. The example from the introduction shows the problem. An earlier record says "Duration = 12 hours," and a later one says "Duration = 18; use the earlier unit." The current answer is "18 hours," but "hours" appears only in the old record, so hiding the whole old record hides the unit as well. And everything after the old record in the cache was computed while it was visible, so its influence stays even after its own tokens are hidden.

That is what the title, "positions are not facts," refers to: hiding an outdated record's tokens is not the same as removing the outdated fact.


W1. "ambiguous references"

"All eight models" are the eight models evaluated on the quantity task that this sentence of the abstract describes: "In a controlled quantity task, masking makes all eight models prefer the new value more strongly..." Figure 2 lists them: Qwen3-4B, Qwen3-8B, Qwen3-32B, Qwen3.8-27B, Llama-3.2-3B, Llama-3.1-70B, Gemma 4 26B-A4B and Gemma 4 31B.

W2. "no problem statement and formulation"

Section 2 ("Tasks, Cache Operations, and Evaluation") answers these questions in three paragraphs:

  • What counts as an update ("Records and replacements"): a later record replaces an earlier one when it gives a different current value for the same entity and attribute.
  • What we edit ("Changing access or recomputing states"): which token positions the model can attend to. The whole-source mask hides every token of the old record, and the value mask (labeled "number-only" in Figures 1 and 2) hides only the old value, the "12" above. Both leave the stored K/V and the model weights unchanged. Recomputation instead deletes the old text and rebuilds the cache from what remains.
  • The objective ("Questions and complete answers"): use the new value for current questions while still answering historical and unrelated questions correctly. In the quantity task, an answer counts as complete only if the number and the unit are right, in the requested format, and the model stops on its own.

W3. "lack of central arguments and research questions"

The introduction states the research question, and its four contributions are the central arguments. It poses the question in its second paragraph: "deciding which positions to hide after a factual update requires knowing which uses of the earlier information remain valid." It then says how the paper approaches it: "We study how to recognize a replacement and what happens when it is applied to a KV cache." Each of the four contributions opens with its claim in bold and names the section that supports it:

  1. "Preference and completeness can diverge" (Section 3): masking makes all eight models more likely to pick the new value over the old one, yet six of them give fewer complete answers.
  2. "Later states preserve useful relations" (Sections 4–5): on multi-hop updates, rebuilding the later states loses 20–41 points on historical questions, while compacting the existing ones has little effect.
  3. "Recognition and beneficial intervention are separate" (Section 6.1): better detection of a replacement does not by itself lead to better masking decisions.
  4. "Access should be evaluated for the question and reader" (Section 6): query-dependent selection and routing avoid some losses, and a cheap cache edit is not the same as efficient, answer-preserving reuse.

W4. "sentences that have no clear contribution to any argument"

Could you point us to one or two specific examples? We want to ensure that every paragraph earns its place in the main text, and we would be glad to clarify their role or refine them accordingly.

AuthorsOpenReview ↗

Response to Reviewer G6KK (2/2)

W5. "unclear experimental setup"

The four main datasets are introduced and cited where they are first used, and the main-text tables and figures name the models they evaluate.

  • Datasets: MQuAKE (Zhong et al., 2023) in Section 4, BABILong (Kuratov et al., 2024) in Section 5.1, MINTEval (Lee et al., 2026) in Section 5.2, and FactConsolidation (Hu et al., 2026) in Section 5.3. The quantity task in Section 3 is our own.
  • Baselines: Quest, FINCH and RefreshKV are named and cited in the first paragraph of Section 6.2. The other conditions, such as the full cache and recomputation, are defined in Section 2.
  • Models: Tables 5 and 50 give their sizes and architectures, and Appendix C.2 describes how the eight quantity-task models were run (precision, chat templates, decoding, stopping). We kept these specifications in the appendix so that the main text could focus on the experiments.

We did find one gap: OAKS and LongMemEval are cited in the appendix (Appendices C.8, D.1 and G), but not where they first appear in the main text. We will cite them at first mention, move the full comparison with Quest, FINCH and RefreshKV from Appendix D.6 into Section 6.2, and use the extra page to introduce each experiment more fully.

W6. "figures with unclear purpose and content"

Each main figure supports one comparison:

  • Figure 1 shows the operations of Section 2 on the duration example.
  • Figure 2 shows the main point of Section 3. For the same 60 questions, it puts two measurements side by side. The first is how much masking raises the model's probability of the new answer relative to the old one, scoring the two answers directly. The second is how many complete answers the model actually generates. The whole-source mask raises the first in all eight models, yet six of them give fewer complete answers, because of unit errors or because they don't stop. The value mask keeps every current answer.
  • Figure 3 explains a result in Section 5.3 that looks contradictory: masking helps Gemma 4 26B-A4B (complete answers 29.5% → 49.6%) but hurts Qwen3.8-27B (90.1% → 80.4%). The figure shows which answers masking fixed and which it broke, and the two models turn out to fail in different ways. Gemma often answered with the outdated value, and masking fixes many of those answers. Qwen rarely did; most of its losses are answers that keep re-checking the records and run out of tokens before finishing.

If a panel or label is unclear, please let us know.

W7. "excessively long appendices that cannot address the concerns above"

The main text tells the paper's story on its own: the research question, the definitions and the main results are all in the nine pages. The appendix is long because we had much more to share than nine pages can hold, and we put it there for readers who want to go further:

  • the details needed to check or reproduce the main results: full protocols, controls and per-model results;
  • many further experiments on the same question, which the main text summarizes briefly. For example, Appendix B asks whether the cache itself shows that a record was replaced, across 22 models (summarized in Section 6.1), and Appendix E asks whether a closely related model can reuse the cache, and whether masking still works the same way when it does (Section 6.2).

A reader can follow the main argument without any of it. The appendix already opens with a short guide that says which question each part addresses and which main-text section it supports, and we will add a table of contents so that readers can find what they need.

We hope this clarifies the paper. Since there is still time left in the discussion period, we'd welcome the chance to clarify anything that is still unclear, and we will reply promptly.

Best regards,

The Authors

I thank the authors for responding.

None of your answers changed my perception of this paper. I wish not to increase my scores.

Please excuse my language below.

If you have some conscience, and if I am even talking to a human, I hope you know that papers like this waste compute and human resources without advancing science in any way, before you decide to submit a 130-page slop to another conference.

We are replying not to ask for a higher score, but to put our response on the public record. Over many years of submitting and reviewing papers, we have never seen reviewers and authors as enemies. That is why your latest comment caught us off guard. Stepping outside scientific discussion to attack the authors personally goes against basic academic standards. Your initial review stated that no answer would change your mind; your latest comment contains outright insults. It is hard to see how a review of a paper came to this.

You have every right to reject our paper or to decide that its scientific contribution is insufficient. Good peer review, though, means explaining why. If you thought our research question did not matter, our experiments were flawed, or our results did not back up our claims, we would have been glad to discuss any of it. Calling a paper "slop" tells us nothing about the science. It only conveys contempt.

After we answered your first review point by point, you offered no new technical critique and responded to none of our explanations. Instead, you took your frustration out on us personally: questioning our conscience, questioning whether we are even human, and claiming our work wastes compute and human effort.

We actually ran these experiments, reported the results honestly, and analyzed them carefully. We spent a great deal of time trying to make sense of what we found and to share what was useful. You are free to think those findings are minor, or not worth publishing. Still, dismissing the entire work as contributing nothing to science, without engaging with any of our answers, is simply unfair. When authors put serious work into answering a review in good faith, and the reviewer refuses to discuss the substance and ends with insults instead, it is worth asking what is really wasting people's time.

If we ever suspected a paper was written entirely by AI, we would say so plainly and point to concrete reasons. We would never question whether the authors are human just because we disliked their paper, let alone question their integrity.

Like your first review, your latest comment raises no specific technical problem; it mentions only the paper's length. ICLR places no limit on appendix length and does not require reviewers to read appendices. We put reproduction details, further analyses and additional experiments there as a reference for anyone who wants them. You may dislike that choice, but disagreeing over appendix length is no excuse for personal attacks.

We do not know what made you dislike this paper so strongly, or why that turned into hostility toward us. We took your criticism seriously and hoped for a serious conversation in return. You told us to reflect on our conscience. We would ask, in turn, whether starting a comment with "Please excuse my language below" makes the personal attacks that follow acceptable.

To be honest, we could easily have left this unanswered. Having written this out, though, we actually feel a sense of relief. We will just chalk this up to a very strange experience in our academic lives. We wish you well, and we trust that the paper will be judged on its own merits.

Reviewer_7V9AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses how language models should update the history stored in their key-value cache when old facts are changed by the recent records. The paper compares hiding (masking) the whole old record, hiding only the replaced value, and deleting the old text and recomputing the cache, with the record pairs supplied so that retrieval is excluded. The paper finds that masking makes all eight tested models prefer the new value more strongly, but six of them lose complete answers, and that later cache states still retain useful information from the earlier records.

Critical strengths

This paper addresses an interesting and timely topic. The paper raises interesting research questions, and the experiments answer them.

Critical weaknesses

(1) There are many previous methods that handle conflicts and changes in the history or previous context. For example, papers on agentic memory, including Mem0, A-MEM, and Zep, and papers on KV cache compression as KVzip, SnapKV, and H2O, as in Appendix G. I do not clearly understand the fundamental difference between the scenarios addressed in these papers and the scenario of this paper. More explanation on this is needed.

(2) Since the appendix is too long, it is hard to say that I have fully understood the paper. Still, judging mainly from the main text, the comparison with existing methods is relatively weak. The paper states that it compares with Quest (ICML'24), FINCH (TACL'24), and RefreshKV (ACL'25) in Appendix D.6, but the comparison is hard to read, and it is difficult to see a large difference from the existing method FINCH (Table 97). I may have missed something. However, direct comparisons with these existing methods (which share the same motivation) should be covered in the main paper, and the comparison design and the analysis of the comparison results should be done in depth.

(3) Could you check whether the performance degradation caused by masking is also observed in stronger models, such as those larger than 70B or closed-source API models?

Questions for authors

Please answer the following questions, considering to the weaknesses above.

(1) What is the fundamental difference between the scenario of this paper and the scenarios addressed in related works, including agentic memory and KV cache compression? (2) What do direct comparisons with these existing methods (Quest, FINCH, and RefreshKV) show? These comparisons should be covered in the main paper.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 7V9A (1/2)

Dear Reviewer 7V9A,

Thank you for your review, and for finding the topic timely, the research questions interesting and the experiments answering them.

We began this work hoping to build a method for updating facts in a KV cache, and tried many ways to do it: masking the whole old record or only its value, recomputing the cache, masking only what a detector flags, and choosing cached tokens by the question. Each worked in some settings and lost answers in others. Rather than propose another method that works only in some settings, we wrote an empirical paper to share why. The reason is the mismatch in our title: hiding an outdated record's tokens is not the same as removing the outdated fact. Masking can hide details the current answer still needs, and later states still carry the old record.

W1 / Q1. "The fundamental difference between the scenarios"

Thank you for raising this; Appendix G covered it only briefly, and we have added the explanation below to Section 7.

These methods decide what to store or keep; our paper examines what an edit to a KV cache actually removes. It does not compete with them, and its findings bear on any method that updates a KV cache by masking or deleting positions.

  • Agentic memory (Mem0, A-MEM, Zep) keeps memory as text and decides which facts are current; the model reads the retrieved text from scratch, so no computed state carries an old fact. Inside a cache, deciding correctly is not enough: even with true replacement labels, masking the outdated record lowers historical accuracy by 6.7–19.0 points in all five models tested (Table 37). Zep already keeps superseded facts, so preserving old values is not our contribution.
  • KV cache compression (KVzip, SnapKV, H2O) drops cached tokens to save memory while keeping the full cache's answers. It assumes every cached fact is still valid, and its signals do not tell outdated records apart: attention to the old record recognizes replacement with AUC 0.356–0.525, where 0.5 is chance (Table 23).

The mismatch matters for KV-cache reuse and for recent preprints that edit or erase cached facts, such as Models Take Notes at Prefill (Li, 2026) and KVEraser (Li et al., 2026). Our results caution against treating a KV cache as a memory that can be updated by masking or deleting an old fact's positions.

W2 / Q2. What the Quest, FINCH and RefreshKV experiments show

Thank you for this suggestion; we agree that these results belong in the main text, and we have moved them from Appendix D.6 into Section 6.2 (Table R1) and added a table of contents to the appendix.

Since the paper is an empirical study, Quest, FINCH and RefreshKV are not baselines for a method of ours. A mask is fixed when the update arrives, before any question is asked; these methods, designed to save memory or computation, instead choose which cached tokens to read for each question. Section 6.2 uses them to test whether choosing with the question avoids the losses of masking. It does: FINCH and RefreshKV match the full cache, which supports our finding that what can be hidden depends on the question, not only on what changed.

In Table R1 (Section 6.2), each of 300 histories has an earlier record A and a later record that changes A's value (150), confirms it (100), or updates a different entity, so A still holds the answer (50). The whole-source mask hides A in every history; the three methods keep as many tokens visible as the mask does. Complete answers on Qwen3-32B:

MethodAll (N=300)Answer changed (N=150)Different entity (N=50)
Full cache98.3% (295)100% (150)90.0% (45)
Whole-source mask83.0% (249)99.3% (149)0% (0)
Quest81.0% (243)86.7% (130)54.0% (27)
FINCH98.3% (295)100% (150)90.0% (45)
RefreshKV (reimplemented)98.7% (296)100% (150)92.0% (46)
  1. The difference appears where the old record is still needed. 45 of the mask's 46 lost answers are on different-entity updates (Qwen3-4B shows the same pattern). Even when the record was replaced, masking with true labels still loses historical answers (Table 37, W1).
  2. When the answer changed, the later record is enough, and every method except Quest matches the full cache; 37 of Quest's 57 misses are malformed JSON, not old values.
  3. Choosing with the question has a cost. FINCH must read each question before it compresses, and RefreshKV keeps the full cache to revisit while decoding, so neither gives one updated cache that serves every later question.
AuthorsOpenReview ↗

Response to Reviewer 7V9A (2/2)

W3. "Stronger models, such as those larger than 70B or closed-source API models"

Yes, the loss also appears in larger models. We have now run the 60 quantity questions of Section 3 (Figure 2) on four open models above 70B: all four lose complete current answers under the whole-source mask, three of them most answers, while masking only the old number keeps all four at 100%.

In these questions, the later record says "Duration = 18; use the earlier unit", so the unit appears only in the old record. A complete answer has the right number and unit and ends normally.

ModelFull cacheWhole-source maskValue mask
GLM-4.7 (355B, 32B active)100% (60/60)13.3% (8/60)100% (60/60)
Qwen3-Coder-480B (480B, 35B active)100% (60/60)13.3% (8/60)100% (60/60)
Qwen3-235B (235B, 22B active)100% (60/60)23.3% (14/60)100% (60/60)
Mistral-Large (123B, dense)100% (60/60)91.7% (55/60)100% (60/60)
  1. All failures are unit errors: GLM-4.7 and Qwen3-Coder-480B omit the unit, Qwen3-235B writes the number followed by "the same as in record A" in 40 of its 46 misses and omits the unit in 6, and Mistral-Large's 5 misses use the wrong unit. When the later record repeats the unit, all four keep 100% under either mask.
  2. Preference and completeness diverge here too: compared with full access, the whole-source mask raises each model's log-probability preference for the new value over the old one by 2.8–10.0 nats (all 95% intervals above zero), while complete answers fall.

Closed-source API models do not expose their KV cache, so neither mask can be applied to them. We have added these results to Appendix C.2 (Table R3) and summarized them in Section 3.

All changes in the revised PDF are marked in blue. New tables are numbered R1–R5, so the table numbers in the reviews and in our replies are unchanged.

We hope these answers address your concerns, and we would welcome further discussion.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.