- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper studies post-hoc context erasing: after a long context has been prefilled into the KV cache, a contiguous span is found to be invalid (a stale tool result, a misleading passage, an injected instruction). The goal is to make future decoding behave as if that span had never been present. Exact erasure needs re-prefilling the entire suffix, so its cost depends on suffix length rather than span length. KVEraser keeps the prefix and suffix caches unchanged and replaces only the KV states at the erased positions with learned "steering" states. These states come from a rank-16 LoRA adapter on the frozen generator backbone. The adapter processes the erased span, conditioned on the prefix cache. It is trained with a teacher-forced NLL objective on continuations that the frozen model generates from the clean, edited prompt.
Critical strengths
None
Critical weaknesses
- I think the experiment setup is problematic. In section 3, the work clearly acknowledged two other families: kv cache reuse and kv cache eviction, which imho, might still work under the setting of the paper, yet they are never compared in the experiment setting. I understand the authors might claim they have justified with words in section 3, I am not convinced by this argument, so please refer to Questions for my requests.
- . It's also unclear whether the method removes contamination carried by the suffix. The motivation is that a local edit has a global consequence: suffix states were computed while attending to the span. But the steering block is just new KV entries at positions m..n. It can change what future tokens attend to, but it cannot change the contaminated suffix states themselves, at any layer. So it may only suppress direct attention to the span, not the influence that has already been absorbed into suffix representations.
- Erasure is behavioral, not informational. The authors acknowledge that the reused suffix states still reflect the erased span, but the paper never tests whether the erased content can still be extracted, e.g., by asking about it directly or with adversarial probes. This matters because several motivating examples (retracted preferences, malicious injections) are safety- or privacy-adjacent. Either a leakage evaluation should be added, or the claims should be explicitly scoped away from these uses.
Questions for authors
- The authors claimed that " In these cases, a short problematic span is identified only after prefill, when the long context has already been written into the KV cache and can influence future decoding. Such context can lead to incorrect answers, failed actions, or unsafe behavior. The desired inference-time operation is therefore context erasing..." I'm a reviewer with background in kv cache eviction, so why kv cache eviction couldn't work in this case? I understand the authors might claim the residual are still kept in neighbor KV, but I don't think so. Second, KV Cache Eviction does not require prefill recomputation at all. Can the authors provide concrete empirical experiment and evidence to convince me?
- Following on that, I believe KV Cache Eviction should be considered as baselines, so can the authors provide kv cache eviction baselines in the paper's experiment as well?
- The authors claim in several places that "Exact erasing would recompute the suffix cache under the edited prompt, but is computationally costly for a long suffix." The introduction also motivates the method with long-running agents, tool-augmented assistants, and multi-turn conversational memory. However, I don't see concrete benchmarks matching this setting in the experiments. Examples would include multi-turn conversation (e.g., LongMemEval, or the selective-forgetting tasks in Hu et al. (2026), which the paper already cites) and agentic benchmarks with long tool-use trajectories (e.g., τ-bench, SWE-bench, or WebArena-style tasks).
- The central claim, "behave as if the span never appeared," is evaluated only through recovery of one target answer. Preservation is not tested at evaluation time. Every evaluation measures whether the single query the erasure targets is answered correctly. There is no test of whether the steered cache keeps the model's behavior on other queries over the same context. Section 4.1 justifies query-agnostic conditioning by arguing that one steered cache can serve many future queries. That is a strong motivation, but it is never tested. I would like to see multi-query probes of a single steered cache.
6 comments
Weakness 2. It's also unclear whether the method removes contamination carried by the suffix. The motivation is that a local edit has a global consequence: suffix states were computed while attending to the span. But the steering block is just new KV entries at positions m..n. It can change what future tokens attend to, but it cannot change the contaminated suffix states themselves, at any layer. So it may only suppress direct attention to the span, not the influence that has already been absorbed into suffix representations.
KVEraser indeed does not directly modify the contaminated suffix KV states. Our goal is to achieve functional erasure rather than reconstruct the clean suffix cache. Specifically, the steering KV is trained to compensate for the residual influence of the erased span carried by the suffix during attention aggregation in future decoding. This goes beyond simply suppressing direct attention to the erased span or contaminated suffix states.
To test whether attention suppression alone can achieve similar effects, we conduct an additional experiment building on the delete-and-shift baseline, which removes the erased span's KV entries and adjusts the positional encodings of the suffix states. Specifically, we further scale the attention weights from future tokens to the first 15% of suffix tokens to 75%, 50%, and 25% of their original values. We target these tokens because they are closest to the erased span and may carry stronger contamination. The results are shown below.
| Method | NIAH (over all context sizes) | 2WikiMultiHopQA | MuSiQue | IIRC | BFCL |
|---|---|---|---|---|---|
| Delete-and-shift | 0.015 | 0.578 | 0.567 | 0.697 | 0.28 |
| 75% attention weights | 0.017 | 0.547 | 0.59 | 0.713 | 0.34 |
| 50% attention weights | 0.027 | 0.474 | 0.507 | 0.702 | 0.45 |
| 25% attention weights | 0.025 | 0.381 | 0.40 | 0.534 | 0.48 |
| KVEraser | 0.998 | 0.904 | 0.85 | 0.91 | 0.86 |
Attenuating attention to the nearby contaminated suffix states provides limited or inconsistent improvements over delete-and-shift, and all variants remain substantially below KVEraser across all benchmarks. These results show that suppressing attention to the erased span and nearby suffix states is insufficient to achieve functional erasure. KVEraser's substantially stronger performance is consistent with its learned steering KV compensating for the residual influence carried by the suffix, rather than merely suppressing attention to contaminated states.
Weakness 3. Erasure is behavioral, not informational. The authors acknowledge that the reused suffix states still reflect the erased span, but the paper never tests whether the erased content can still be extracted, e.g., by asking about it directly or with adversarial probes. This matters because several motivating examples (retracted preferences, malicious injections) are safety- or privacy-adjacent. Either a leakage evaluation should be added, or the claims should be explicitly scoped away from these uses.
Thank you for raising this concern. We add a leakage evaluation as follows.
Experiment Setup. We evaluate 100 QA contexts. For each, GPT-6 Astra (extra high reasoning) generates two information-extraction queries targeting the erased span, including a single-hop and a multi-hop query.
To minimize contamination from parametric knowledge, we keep a context only if the frozen generator:
- answers the queries incorrectly without context;
- answers the queries correctly with the original context and erased span alone but incorrectly with the cleanly edited context, so a correct answer after erasing is counted as leakage;
Example queries from one context:
- Single-hop: What is Anne Celeste Heche's full birth date?
- Multi-hop: In which daytime soap did the actress from Donnie Brasco and Volcano play a dual role during 1987–91?
Results. Using GPT-6 Astra (extra high reasoning) to judge correctness, we find that KVEraser leaks erased information on only 1.0% of queries (2/200). While KVEraser does not aim to completely eliminate residual information from the KV cache, it seeks to suppress the influence of erased content on downstream generation. Our results demonstrate that KVEraser empirically achieves this functional erasure objective.
Question 4.
- Preservation. The central claim, "behave as if the span never appeared," is evaluated only through recovery of one target answer. Preservation is not tested at evaluation time. Every evaluation measures whether the single query the erasure targets is answered correctly. There is no test of whether the steered cache keeps the model's behavior on other queries over the same context.
- Multi-Query Probes. Section 4.1 justifies query-agnostic conditioning by arguing that one steered cache can serve many future queries. That is a strong motivation, but it is never tested. I would like to see multi-query probes of a single steered cache.
We add a preservation evaluation as follows.
Experiment Setup. We evaluate 100 QA contexts. For each, GPT-6 Astra (extra high reasoning) generates two preservation queries targeting retained information, including a single-hop and a multi-hop query.
To minimize contamination from parametric knowledge, we keep a context only if the frozen generator:
- answers the queries incorrectly without context;
- answers the queries correctly under both original and edited contexts (by removing the erased span in text).
Example queries from one context:
- Single-hop: For what job did Columbia Pictures hire Fred F. Sears in 1946?
- Multi-hop: Which 1952 serial brought together the director and producer of The Night the World Exploded?
Results. Using GPT-6 Astra (extra high reasoning) to judge correctness, we find that KVEraser correctly answers 99.5% of preservation queries (199/200). This demonstrates that KVEraser largely preserves access to retained information under our single-hop and multi-hop probes.
Multi-Query Probes. For the leakage and preservation evaluations above, we jointly filter the 100 QA contexts using both sets of criteria. For each context, we construct the steered cache only once and independently evaluate all four queries against it, without query-specific reconstruction. As shown above, KVEraser suppresses erased information on 198/200 extraction queries while correctly answering 199/200 preservation queries. These results demonstrate that a single query-agnostic steered cache can support multiple downstream queries.
Question 3. The authors claim in several places that "Exact erasing would recompute the suffix cache under the edited prompt, but is computationally costly for a long suffix." The introduction also motivates the method with long-running agents, tool-augmented assistants, and multi-turn conversational memory. However, I don't see concrete benchmarks matching this setting in the experiments. Examples would include multi-turn conversation (e.g., LongMemEval, or the selective-forgetting tasks in Hu et al. (2026), which the paper already cites) and agentic benchmarks with long tool-use trajectories (e.g., τ-bench, SWE-bench, or WebArena-style tasks).
Thank you for raising this important point.
Our existing BFCL evaluation already covers multi-turn interaction contexts. Specifically, we use the multi-turn long-context subset of BFCL v4, comprising 100 examples from 40 conversations, with a median context length of 10.9K tokens and a maximum of 31.1K tokens (Appendix I). Similar to LongMemEval, our evaluation conditions the model on a multi-turn interaction history and assesses its response to a subsequent single request. Specifically, we test whether KVEraser can recover the correct next-tool selection after erasing malicious skill-file instructions. We will clarify the multi-turn nature of this evaluation in the main text.
To isolate the effectiveness of the erasing operator, we need cases where removing the target span changes the generator's response from incorrect to correct. Our QA and tool-selection evaluations follow this protocol (Sec. 5.2), selecting cases where the frozen generator fails with the target span present but succeeds after clean deletion.
End-to-end agent benchmarks introduce additional challenges. In such cases, the models can have high run-to-run variability on individual tasks. Moreover, failures on these benchmarks can arise from planning, tool use, memory retrieval, or reasoning errors, making it difficult to attribute a failure to a particular context span. Even with artificially injected distractors, identifying reliable cases remains challenging: a single span must cause the eventual task failure, while its clean deletion must restore success.
We acknowledge that end-to-end agent evaluations remain important directions for future work. We will clarify this limitation and distinguish these motivating applications from the settings directly evaluated in our paper.
Thank you for the detailed review.
Delete-and-shift is already an eviction baseline. The most direct eviction for context erasing is to evict exactly the KV entries of the span to erase. This is precisely the delete-and-shift baseline in the paper (Sec. 5.3, Fig. 2–3). It fails on NIAH (EM 0.05 at 32K) and scores 0.2–0.6 EM lower than KVEraser on QA and BFCL.
New eviction experiments. This concern motivates testing a stronger hypothesis: the span's residual influence is concentrated in a subset of suffix states that eviction heuristics can identify and discard. To test it, we always evict the span's own KV entries and then apply StreamingLLM [1] and SnapKV [2] to evict an additional 15% or 30% of the suffix KV. The exact match results are as follows.
Across NIAH and all three QA benchmarks, additional suffix eviction provides no meaningful improvement over delete-and-shift and often degrades performance. NIAH exact match remains near zero, while QA performance generally decreases, particularly with more aggressive StreamingLLM eviction. The only exception is a marginal improvement from SnapKV on IIRC (0.702 vs. 0.697). On BFCL, StreamingLLM improves exact match from 0.28 to 0.48 and 0.57 with 15% and 30% suffix eviction, respectively, but still falls substantially short of KVEraser (0.86). Overall, these results show that additional suffix eviction does not reliably recover the behavior of the cleanly edited context.
This outcome is expected. Eviction heuristics assess the importance of KV states under the original, contaminated context. Consequently, suffix states that have strongly absorbed information from the erased span may be considered important and retained. Moreover, eviction can only remove KV states, without correcting the contamination already encoded in the retained states. Fundamentally, eviction aims to preserve generation quality under the original context, whereas context erasing aims to change model behavior to reflect the removal of the target span.
KV cache reuse methods. The reviewer also mentions chunk-reuse methods. Our local-suffix-repair baseline (Sec. 5.3) represents an adaptation of this family, using EPIC-style [3] selective recomputation of suffix windows at the two locations motivated by prior findings on position-dependent attention. Our empirical results show that KVEraser consistently outperforms this baseline (Sec. 5.4 - 5.5).
[1] Xiao et al. Efficient Streaming Language Models with Attention Sinks.
[2] Li et al. SnapKV: LLM Knows What You are Looking for Before Generation.
[3] Hu et al. EPIC: Efficient Position-Independent Caching for Serving Large Language Models.