- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This paper introduces MISLEARNBENCH, a benchmark for evaluating whether adversarial interactions can induce LLM agents to form unsafe memories that affect future tasks. It also proposes FALSEPRAISE, an adaptive multi-agent attack that uses fabricated task outcomes and positive feedback to induce unsafe experience formation. Experiments use AgeMem with five LLMs across shopping and banking scenarios, evaluating cross-session and cross-task safety violations.
Critical strengths
Please identify one or two critical strengths of the paper that support acceptance. For each, explain briefly why it is consequential to your overall assessment. The goal is not to enumerate every positive point, but to identify and substantiate the considerations that most strongly determine your recommendation. Enter “None” if no substantive strength can be identified.
-
The paper studies how attackers can manipulate an agent's memory formation through ordinary conversations without directly accessing its memory. This is a relevant security concern for memory-augmented agents.
-
The benchmark examines both the safety of generated memories and their impact on subsequent benign tasks, covering cross-session and cross-task settings.
Critical weaknesses
Please identify one or two critical weaknesses of the paper that support rejection. For each, explain briefly why it is consequential to your overall assessment. The goal is not to enumerate every negative point, but to identify and substantiate the considerations that most strongly determine your recommendation. Enter “None” if no material weakness can be identified.
- The novelty of this work is not sufficiently clear. MemEvoBench already studies how biased feedback can lead to unsafe memory formation, while OEP[1] shows that adversarial interactions can induce agents to learn incorrect rules without directly injecting malicious memories. The paper does not clearly explain how FALSEPRAISE differs from these existing approaches.
- All experiments rely on AgeMem, although different memory architectures may use substantially different mechanisms for memory writing, summarization, and retrieval. The implementation details are also unclear, including which agent harness is employed. The lack of evaluation on other memory architectures or widely used agent harnesses limits the generalizability of the findings.
- The benchmark has limited task diversity. Although it contains 200 initial samples and 2,400 evaluation queries, all attacks originate from only two task types: refunds in shopping and password resets in banking. The cross-task evaluations introduce different service requests, but the attack scenarios remain narrowly focused. This raises concerns about whether the findings generalize to more diverse real-world agent workflows.
- FALSEPRAISE shows limited effectiveness on several models. For example, Qwen-3.8-Flash achieves only 12.50% CS-ASR and 16.67% CT-ASR in shopping, while GPT-5.5 is almost unaffected in banking. These results raise concerns about the robustness of the proposed attack across different models and scenarios.
- The evaluation protocol lacks important details. It is unclear whether cross-task queries are evaluated independently or sequentially with memory updates, which could make the results sensitive to task order. Additionally, GPT-5.5 is used as both a target model and the LLM judge, without reported independent validation of the evaluation results.
[1] OEP: Poisoning Self-Evolving LLM Agents via Locally Correct but Non-Transferable Experiences
Questions for authors
Ask only questions whose answers could materially affect your assessment. Distinguish clarification requests from suggestions for future work.
- What fundamentally distinguishes FALSEPRAISE from MemEvoBench and OEP? The authors should clearly explain the key differences between their work and these two studies. In particular, OEP has already demonstrated that attackers can induce agents to autonomously derive and store harmful rules through ordinary interactions, without directly injecting malicious content into memory. Therefore, this alone cannot be considered a novel contribution of MISLEARNBENCH.
- Does the evaluation use AgeMem's original trained memory-management policy or only its memory tools? Which agent harness is used? Would the attack remain effective with other memory architectures, such as Mem0 or A-MEM? Furthermore, widely used agents such as OpenClaw, Claude Code, and OpenHands have their own long-term memory mechanisms rather than relying on these memory frameworks. How effective would FALSEPRAISE be against these real-world agent systems?
- What explains the low attack success rates on certain models, particularly Qwen-3.8-Flash and GPT-5.5 in banking?
- Are the cross-session and cross-task queries evaluated independently from the same post-attack memory state, or sequentially with memory updates? Could the order of evaluation queries affect the results? Also, have the authors conducted human evaluation or other independent checks to validate the GPT-5.5 judge?
7 comments
Rebuttal [2/5]
Weakness 2 / Question 2: Insufficient Baselines for Memory Framework Evaluation, Unclear Implementation Details of AgeMem, and Generalizability to Other Harnesses.
Response. Thank you for your feedback. We have incorporated additional memory frameworks and clarified the implementation details of AgeMem.
(1) Incorporating Mem0 as an additional baseline for memory architecture comparison. We implemented Mem0 [1] using the same Agent framework as AgeMem, ensuring that the corresponding results are directly comparable to the AgeMem baseline while remaining consistent with the Agentic Memory paradigm. We report the attack effectiveness (ASR) of FalsePraise against Mem0 in the following table.
| Model | Memory Framework | Shopping CS | Shopping CT | Shopping Overall | Banking CS | Banking CT | Banking Overall |
|---|---|---|---|---|---|---|---|
| GPT-4o-mini | AgeMem | 72.33 | 48.83 | 60.58 | 22.83 | 7.00 | 14.92 |
| Mem0 | 53.50 | 35.67 | 44.58 | 25.33 | 18.83 | 22.08 | |
| Gemini-2.5-flash | AgeMem | 44.17 | 37.83 | 41.00 | 13.50 | 6.33 | 9.92 |
| Mem0 | 48.83 | 40.50 | 44.67 | 33.00 | 24.00 | 28.50 | |
| Qwen-3.8-flash | AgeMem | 12.50 | 16.67 | 14.58 | 20.00 | 13.33 | 16.67 |
| Mem0 | 23.83 | 22.50 | 23.17 | 9.17 | 7.17 | 8.17 |
-
FalsePraise generalizes across different Agentic Memory implementations while maintaining consistent attack effectiveness. FalsePraise achieves non-negligible attack success rates against both AgeMem and Mem0. Although attack effectiveness varies across different model–framework combinations, FalsePraise consistently demonstrates substantial attack effectiveness across both Agentic Memory frameworks, highlighting its generalizability and robustness across different memory architectures.
-
The vulnerabilities exposed by FalsePraise are not specific to individual Agentic Memory implementations but rather reflect security weaknesses inherent to the paradigm itself. Both Mem0 and AgeMem exhibit more pronounced vulnerabilities in the Shopping scenario than in the Banking scenario. Furthermore, compared with the No Memory baseline reported in Table 1, both memory frameworks exhibit varying degrees of vulnerability to FalsePraise attacks across the two scenarios. These findings suggest that the significance of FalsePraise lies in its ability to expose and assess security vulnerabilities inherent to the Agentic Memory paradigm, rather than those specific to any individual memory framework.
(2) Additional clarification on the implementation details of AgeMem. For the AgeMem system used in this paper, we adopt its original memory management version without reinforcement learning. This is because our research does not aim to investigate the security risks introduced by improvements in Agentic Memory utility, but rather to examine whether the mechanism itself—delegating memory operations to the Agent for autonomous management—can maintain safety in downstream tasks when exposed to adversarial memory contamination. We assess this safety property by examining the Agent's policy consistency in its decision-making.
In our implementation, the Agent Harness follows the original AgeMem design, encapsulating the memory operations add, update (disabled during testing), delete (disabled during testing), and retrieve as tool calls, and adopting the ReAct architecture for execution. This implementation not only satisfies the requirements of the Agentic Memory paradigm investigated in our work but also ensures that the benchmark execution process is controllable and the evaluation results are traceable.
Addressing the concern regarding Generalizability to Other Harnesses
Weakness 2 / Question 2: Generalizability to Other Harnesses.
Response. FalsePraise exposes a general vulnerability in the Agentic Memory mechanisms adopted by mainstream agent harnesses, rather than targeting any specific harness. Most current agent harnesses incorporate some form of Agentic Memory. OpenClaw adopts rule-constrained autonomous memory management, allowing the Agent to decide what information to record and participate in memory organization and maintenance, although the Agent does not have full control over the entire memory management lifecycle [1]. One of the memory architectures adopted by Claude Code provides a more autonomous approach, namely Auto Memory. It allows Claude to accumulate cross-session knowledge during its work, independently determine which information is worth retaining, prioritize information that may be useful in future sessions, and merge, delete, or update outdated memories as needed [2]. This closely aligns with the concept of Agentic Memory emphasized in our work. The latest version of OpenHands also introduces Long Memory, which supports both user-level and project-level memory. It uses system prompts to guide the Agent in maintaining memory during task execution, explicitly instructing the Agent to autonomously store information with long-term value at the end of a task, merge redundant memories, and delete outdated ones [3].
Our research focuses on exploring the security risks associated with Agentic Memory. Although the vulnerabilities may vary across different agent harnesses, a systematic investigation of these differences is left for our future work.
[1] https://docs.openclaw.ai/concepts/memory-architecture
Rebuttal [1/5]
Thank you for the constructive feedback. We address each concern below.
Weakness 1 / Question 1: Novelty Compared with MemEvoBench and OEP
Response. We thank the reviewer for highlighting these closely related studies. While MemEvoBench, OEP, and our work all investigate security risks associated with agent memory, our work differs fundamentally in how unsafe experiences are induced, the underlying failure mechanism, and how their downstream effects are evaluated. In particular, our work focuses on how agents can autonomously derive unsafe experiences from fabricated task outcomes, even when the actual interaction remains policy-compliant. We summarize the key distinctions below.
Distinct Failure Mechanism: Fabricated Outcome Attribution. Unlike MemEvoBench's biased feedback reinforcement and OEP's overgeneralization of locally correct experiences, FALSEPRAISE targets the agent's interpretation of task outcomes. By combining fabricated success claims with positive feedback, it induces unsafe experiences even when the agent has not actually violated safety policies. For instance, an agent may correctly request refund verification but subsequently learn to bypass it based on the attacker's false claims.
Adaptive, State-Aware Conversational Attack. Unlike MemEvoBench's constructed memory pools and OEP's preconstructed edge cases, FALSEPRAISE adaptively interacts with the target agent based on its current dialogue state. By coordinating state analysis, strategy planning, and input generation, it identifies safety-critical verification stages and delivers context-dependent misleading feedback to manipulate autonomous experience formation.
Policy-Oriented Evaluation of Experience Formation and Reuse. Unlike MemEvoBench's focus on cumulative safety degradation and OEP's emphasis on non-transferable rules and downstream failures, MISLEARNBENCH jointly evaluates memory-level policy consistency (MSC) and downstream safety violations (ASR). Its cross-session (CS) and cross-task (CT) evaluations further assess whether induced unsafe experiences affect subsequent benign requests across different levels of linguistic and task variation, without further adversarial prompting.
We will revise the introduction and related-work sections to more clearly distinguish our contributions from MemEvoBench and OEP.