acceptodds
Under review as a conference paper at ICLR 2027

Agent Memory Can Only Forget What Its Deletion Path Can See

Abstract

Existing forgetting evaluations can report near-perfect deletion while the forbidden fact remains fully retrievable. Across three widely deployed agent memory systems, surface-form retrieval scoring gives 97–100% while direct queries for the same forbidden fact give 0%: the fact was rewritten rather than removed, and the old scoring credited the rewrite. What a memory system can forget is bounded by what its deletion path — the control plane responsible for memory mutation — can see; existing benchmarks evaluate the recall plane instead. We show this with a controlled ablation that holds the store, model and prompts fixed and moves only the call site. Write-time knowledge helps only when the deletion path can read it: granting the deletion step access to the same annotations, with no new information and no new authority, recovers part of the gap. But a model placed before the request cannot see what the user later asks to remove. On the 345 cases that measure forgetting, mutation-time placement raises the pass rate from 71.0% to 91.3% (+20.3 pt); including a separate 40-case partial-edit diagnostic, the full suite rises from 63.6% to 88.8%. The same contract takes a third-party store from 0/38 to 38/38 on the affected canonicalization categories. We release ForgetEval, a 1385-case benchmark with deterministic scoring and an Adapter Protocol for heterogeneous stores. Reliable forgetting depends on the control plane and cannot be inferred from retrieval quality alone.

Then back it, or bet against it.

Related papers

Open the market on this paper to see 7 more related papers.