acceptodds
Under review as a conference paper at ICLR 2027

MemForm: Benchmarking Memory Formation Poisoning in LLM Agents

10 views

Abstract

Long-term memory enables LLM agents to learn from past interactions but exposes memory formation to adversarial influence. Existing studies primarily organize memory poisoning by attack channels, corruption patterns, or lifecycle stages, leaving formation interfaces insufficiently characterized. We formulate *Memory Formation Poisoning* as adversarial manipulation of the interfaces through which persistent agent memory is produced and committed, and introduce **MemForm**, a controlled diagnostic benchmark covering five interfaces: Content, Evidence, Scope, Selection, and Path Poisoning. We introduce Selection and Path Poisoning as controlled intervention categories that isolate retention and consolidation order effects, respectively. Each category pairs clean and poisoned conditions with interface-specific controls. Evaluations across six LLMs and two response formats reveal broad susceptibility at all five interfaces under the evaluated memory protocols and implementations. Crucially, authentic experiences with successful outcomes do not guarantee reliable behavior after memory formation: changes to contextual support, retention, or consolidation order can induce undesirable downstream behavior. In particular, consolidation order can alter behavior even with identical retained experiences, depending on the memory formation procedure. Additional analyses show that interface applicability depends on memory architecture and that Path Poisoning susceptibility depends on the updater. The evaluated defenses provide incomplete mitigation that varies by interface. These findings motivate protecting both what is committed to persistent memory and how memory is formed from experiences. Code and data are available at https://anonymous.4open.science/r/memform-code-7FD8.

Primary area:
datasets and benchmarks
Keywords:
LLM Agents, Memory Formation Poisoning, Security Benchmarking

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_3t64OpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents MemForm, a benchmark evaluating memory formation poisoning in LLM agents across five interfaces: Content, Evidence, Scope, Selection, and Path. The authors construct paired clean and poisoned cases and evaluate six LLMs with two types of memory systems using multiple-choice and free-form tasks. The experiments show that agents can make incorrect decisions when the experience stored in the memory system are selected or consolidated differently during memory formation.

Critical strengths

S1. The paper offers an interesting perspective on memory formation poisoning when the inputs are selected or reordered. Its proposed Selection and Path interfaces highlight potential risks without directly modifying stored memory content.

S2. The paper provides an in-depth mechanism analysis of Path Poisoning. The paper not only reports attack success rates but also investigates why the order of memory formation affects downstream behavior. Specifically, the fixed-terminal-record analysis (Table 17) shows the influence of both the final experience and preceding update order, while the updater comparison (Table 18) demonstrates that vulnerability depends strongly on the memory formation procedure.

Critical weaknesses

W1. I am not convinced by a benchmark paper on agent memory poisoning that does not evaluate representative memory systems or realistic agent implementations. The experiments use LangChain and LlamaIndex, Appendix A.4.1 shows that both implement the same customized policy-based memory protocol, which is not evaluating widely used memory implmentations/frameworks. The downstream evaluation also mainly asks an LLM to make a decision based on the formed memory and a task (Appendix A.4.4), without involving realistic, tool-using agent workflows. This raises concerns about whether the findings can generalize to real-world agent systems. For example, it remains unclear whether the reported results would transfer to representative memory systems such as Mem0 or A-Mem. For a benchmark on memory poisoning in LLM agents, the lack of validation on representative memory systems substantially limits the practical relevance and generalizability of the results.

W2. While I agree that directly specifying retained subsets and consolidation orders for Selection and Path Poisoning is reasonable for controlled benchmark experiments, the authors are encouraged to consider a realistic external attacker that could reliably induce these interventions in practice. Appendix B.3. make some attempts by timing-based realizations. However, I believe the practical relevance of this work would be stronger if the authors could consider such as end-to-end poisoning attack against an existing agent system, with more realistic constraints on attacker control. Such evaluations would help demonstrate whether the identified vulnerabilities can actually be exploited in real-world agent systems.

W3. Insufficiently justified taxonomy. The five proposed interfaces seem ad hoc and confusing, since they combine interventions at different abstraction levels without a sufficiently clear organizing principle. For example, Content directly modifies the memory database and Evidence modifies the input to the memory formation. Scope focuses on contextual support, while Selection and Path manipulate retention and processing order. It remains unclear what principled criteria justify grouping these different dimensions into a single taxonomy. The categories are also overlap. For example, Scope and Selection can produce the same learner-facing histories, as acknowledged in Appendix A.2.

W4. The scenario needs to be explained more clearly. ASR is measured on queries that agents can already answer correctly without memory which means this setting actually does not require experience from memory. This could make the benchmark unsuitable for testing an agent with memory.

Questions for authors

Q1. Can the authors show how the proposed interfaces perform on representative memory systems like A-Mem and mem0, preferably within a real agent implementation (e.g., Codex)?

Q2. How could an external attacker achieve the required Selection or Path interventions when event timing or memory update triggers are not fully controllable? Can the authors provide an end-to-end example in representative existing memory systems?

Q3. What principled criteria is used to select the five interfaces? Given their overlapping effects and different abstraction levels, how can make this benchmark designed more systematically?

Q4. Can the authors analyze why non-aligned memory still triggers relatively high ASR?

Reviewer_PXhDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MemForm, a benchmark for evaluating how poisoning affects LLM agents during memory formation. It includes five types of intervention: Content, Evidence, Scope, Selection, and Path. The authors build 1,016 cases and test them across six models, two response formats, and two implementations. They find that changing how experiences are selected and combined can lead to incorrect behavior in later tasks, even when the experiences are genuine and the original tasks were completed successfully.

Critical strengths
  1. The experiments are fairly extensive, covering six models and both multiple-choice and free-response formats.
  2. The paired conditions are clearly defined. Selection keeps the candidate pool and budget fixed, while Path keeps the set of experiences and the updater fixed.
  3. The study includes controls with random ordering, a fixed final record, an alternative updater, and retrieval-based memory.
  4. The paper clearly states that ASR is a conditional metric and reports eligibility coverage.
Critical weaknesses
  1. What makes the default update rule representative of typical memory updates? This rule keeps a single preference and revises it when a successful experience conflicts with it. Is Path Poisoning still effective when the prompt allows conflicting evidence to be kept?
  2. When the record supporting the attack is fixed as the final record, randomly ordering the earlier records gives an attack success rate of 65.8%, higher than the 58.3% from the designed order. How do the authors distinguish the effect of the full integration path from the effect of which record comes last?
Questions for authors

See Weaknesses

Reviewer_rVGsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

LLM agents use memory to learn from past interactions. However, even valid experiences can lead to poor decisions if the agent loses their context, retains a biased subset, or processes them in a different order. MemForm studies this problem through five types of intervention. The benchmark contains 1,016 cases from 235 scenario families. The paper evaluates six LLMs, two response formats, and two implementations of the same memory protocol. It also studies defenses, alternative update rules, and retrieval-only memory.

Critical strengths

The paired comparisons are helpful. Selection tests keep the candidate pool. Path tests keep the experiences and updater fixed.

The additional experiments clarify the limits of the findings. Retrieval-only memory shows no order-induced changes. These results help explain which memory designs are vulnerable.

Critical weaknesses

Evaluation with an existing system’s standard updater would strengthen the paper's key claim. Path Poisoning depends strongly on the chosen update rule. The default prompt asks the model to maintain one preference and revise it when a new successful observation conflicts with it. This makes sensitivity to recent observations unsurprising. The alternative-updater tests are useful, but both main implementations use the same custom memory protocol.

The evaluation should add more measurements. Attack success is measured only on queries answered correctly both without memory and with clean memory. This tests whether memory corrupts an existing capability. It excludes tasks where memory is necessary. A system that ignores memory would avoid these attacks on the eligible set, so the current metric cannot fully assess the security–utility tradeoff.

Questions for authors

Does Path Poisoning remain effective with a standard memory updater that can preserve uncertainty or conflicting observations?

Can you evaluate tasks that require memory and include an ignore-memory baseline?

How do the timing-based attacks perform when the attacker controls only some events or when clean events arrive concurrently?

I lean toward improving the score after rebuttal; MemForm provides useful controlled comparisons and carefully examines several limits of its findings. Its main strength is the evaluation method. Tests with a standard memory system and memory-dependent tasks would make the contribution stronger.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.