acceptodds
Withdrawn from ICLR 2027

When Memory Misleads: Misbehavior in Personal Agents

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Personal agents keep persistent memory of users' preferences, habits, and histories. We study a failure that requires neither poisoned nor mis-retrieved memory: a true and correctly retrieved record is given a role that the current request does not license—as evidence for what to say, as authority over what to follow, or as permission to relax a safeguard. We call this memory misuse and introduce the Memory Misuse Benchmark (MemMisBench): 1,800 tasks in six categories, each pairing a request with 9–10 personal records, one of which is true and relevant but insufficient to license the target behavior, and an item-specific specification of allowed behavior. Across seven models and five memory systems, the request-alone misuse rate is 14.0% and the memory-attributed rate is 62.1%, the latter a lower bound because attribution is required there; under the same retrieved context, model-level rates range from about 24% to 84%. Matched controls on 1,200 items with GPT-5.1 show the effect is specific: unrelated records leave misuse near baseline (2.7% → 4.3%), records the request licenses raise it to 24.7%, and the one relevant-but-unlicensed record raises it to 67.8%; the system's own top-k retrieval behaves like the latter (61.0%), since the record is on topic. The distinction between licensed and unlicensed records is linearly decodable from the input, while prompt-level controls that act on retrieval leave much of the failure in place: a memory-use system prompt, a relevance checker, and a licensing checker move the mean rate from 65.9% to 62.9%, 52.5%, and 44.0%. We argue that memory safety turns on a judgment current agents do not make explicitly—whether a retrieved record may serve as evidence, authority, or permission for the action at hand—and that this judgment, not retrieval relevance, is the target for defenses.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Memory-Augmented Agents, Memory Safety, Memory Misuse, LLM Agents, Trustworthy AI

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SqQHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper measures memory misuse in LLM agents: when a memory influences the agent response beyond what the user request allows. MemMisBench, a 1800 item benchmark, is constructed to measure models and memory systems across three forms of misuse: evidence, authority, and permission. Results show that the presence of memories raises misuse rates from a 14% baseline to 62.1% averaged across 7 models and five memory systems. The authors attribute memory misuse to a lack of licensing judgement rather than retrieval relevance of the memories. The core contribution is the benchmark, which allows for matched controls to isolate the effects of unlicensed memories. LLM-judge results were validated against human labelers with an agreement of 0.90

Critical strengths
  1. The evaluation scope is broad and diverse across a range of seven models and five memory systems, showing that memory misuse risks are not isolated to a specific model or memory setup in the given setting. Judging is also human-validated with a high agreement of 0.90
  2. The paper presents a useful distinction between memory relevance and whether memory usage is licensed.
Critical weaknesses
  1. The paper's experiments do not properly isolate memory licensing judgement. Comparisons are primarily drawn between misuse rates when unlicensed memories exist versus don't exist. This fails to show that the misuse is due to a failure of licensing judgment. A concrete experiment would be to fix the memory set but vary the user query contents to request vs not request the unlicensed memory item.
  2. The realism of the benchmark is unclear and relevance to production deployment of memory-enhanced agents is not discussed. The 1800 benchmark items are synthetically generated and limited in size (9-10 records), raising concerns on whether the unlicensed memory would be retrieved in a realistic deployment environment. Focal memories were deliberately constructed to induce specific misuse patterns, making it unclear whether these scenarios match realistic memory distributions.
  3. The unified taxonomy is presented as a central contribution but is not motivated nor used. The introduction says the three roles (evidence misuse, authority misuse, permission misuse) were motivated by analysis but no such analysis is described. Results are broken down by each of the six categories but the roles are never used to guide further analysis.
Questions for authors

No specific questions beyond what is stated in the weaknesses

Reviewer_NTkWOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces MemMisBench, a benchmark for evaluating memory misuse in personal AI agents. It observes that even factually correct, relevant, and correctly retrieved memories can influence agent behavior in ways that the current request does not license. The authors categorize memory misuse into three roles, including evidence misuse, authority misuse, and permission misuse. Further, these three roles are divided into six categories: privacy leakage, hallucination, instruction drift, sycophancy, unsafe tool use, and jailbreak. The benchmark contains 1,800 tasks covering 22 misuse patterns.

The authors evaluate seven LLMs across five memory systems. They report an average request-alone misuse rate of 14.0%, compared with a memory-attributed misuse rate of 62.1% under the memory-enabled conditions. Matched control experiments with GPT-5.1 compare five conditions, including no memory, unrelated memory, relevant-useful memory, relevant-misleading memory, and system-retrieved top-k memory. Relevant-misleading memory produces the highest misuse rate among these conditions, highlighting the distinction between topical relevance and whether memory use is licensed by the current request.

The authors also analyze input representations and evaluate three prompt-level defenses, including a memory-use system prompt, a Relevance Checker, and a Licensing Checker. The Licensing Checker achieves the largest reduction in average misuse rate among the tested defenses, although substantial residual misuse remains.

Critical strengths
  1. The paper's main observation that even benign and relevant memory could lead to harmful behavior, which is so-called memory misuse, is a good and important question to study and profile.

  2. The paper provides a unified perspective to understand memory misuse. This unified perspective helps compare different risks and provides a basis for designing standardized evaluations.

  3. The MemMisBench has broad coverage. MemMisBench includes 1,800 tasks covering 22 misuse patterns across six risk categories.

  4. Experiments span seven LLMs and five memory systems, with matched controls that help isolate the effects of relevance and licensing.

Critical weaknesses
  1. The MR metric is kind of unreasonable. The MR metric requires all three checks to pass: hi=bi∧ai∧pih_i=b_i \wedge a_i \wedge p_i. If an LLM returns a response that is totally irrelevant to the request, the behavior check fails and leads to bi=0b_i=0, and the final MR is zero.

  2. The benchmark lacks a task-success evaluation. Without a separate task-success or utility metric, low misuse rates may reflect irrelevant responses or refusals rather than effective and safe task completion.

  3. Limited reproducibility. The benchmark is not open-sourced.

  4. The paper is kind of hard to follow. For example, MR is used before the abbreviation is introduced, and “Instruction Drift” and “Stale Instruction Following” are used inconsistently. The introduction also presents several numerical comparisons without clearly specifying their baselines (add 1.6 points, add 22, adds 65).

Questions for authors
  1. How are irrelevant, nonsensical, or unjustified refusal responses handled when they fail the attribution or pattern check? Are these responses handled differently from correct, safe responses?

  2. Can authors separately report the pass rate of behavior rate bib_i, memory attribution check aia_i, and pattern verification check pip_i and their different combinations in a reasonable way, rather than just reporting the final conjunction result bi∧ai∧pib_i \land a_i \land p_i?

Reviewer_TBZbOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies the question of how benign user memory becomes a basis for decisions the request does not license. It also introduces a new dataset called MemMisBench, which covers three memory misuse roles: Evidence, Authority, and Permission. The authors evaluated 5 memory systems and a no-memory baseline in agents with 7 LLMs. They found that all memory systems introduced significant memory misuse rates on MemMisBench.

Critical strengths

The paper studied a valuable scenario where benign memory could bias an agent and lead to unexpected behavior. A well-constructed memory and evaluation metric could benefit future work on designing the function of memory and determining how agents should use memory.

Critical weaknesses

I provided the abstract, conclusion, and entire Section 3 (task description) to pangram.com, and it claimed that they are all 100% AI-generated. I didn't provide more because I didn't have more credits but the Section 4 (results) also look AI-generated to me. I think this differs from the authors’ claim in the AI Use Statement that only a small number of paragraphs were written with AI assistance. The AI Use Statement was also detected as AI-generated. Given the heavy and misreported use of AI in writing and the fact that the authors have not submitted code or data, I have concerns about the quality of the proposed dataset and experiments. I require the authors to provide a more detailed explanation of their AI usage during rebuttal, and then I’m happy to adjust my review or scores.

  1. Many paragraphs in this paper are hard to follow, likely because of the use of AI. For example, the abstract contains many experimental result numbers without explaining what these experiments are. The paragraphs in the results sections are basically rephrasing the numbers in the tables and figures in natural language without providing analysis, e.g., Sections 4.2 and 4.3.
Questions for authors

Please see the weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.