- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes MemAdapter, a method for reducing sycophancy induced by memory/personalization techniques. MemAdapter is a 3 part prompting pipeline, that uses counterfactual and evidence-based reasoning to produce a specification for how a model should use its memory on a given task. Experiments are performed using 3 models and 3 benchmarks. Comparisons are made to other memory systems.
Critical strengths
Memory is one of the most important open problems in AI right now, and over-personalization is a known issue in the field.
Critical weaknesses
I find that this paper has several key weaknesses, all pointing towards rather shallow engagement with the problem.
First, I am very familiar with memory and personalization in LLMs, but after reading this paper I am not quite sure what the problem being described is. The introduction relies on high-level language and fails to make key logical connections to bring the reader to understand the problem. I find the one worked example, that of Figure 2b, to be quite confusing. On a low level, it is not clear from the figure why the system thinks the doctor is biased towards A; it is not clear why the doctor cannot just prompt the model (or toggle a setting) to tell the model to not use memory here, if that’s preferable (although do we not want LLMs evolving their knowledge based on the experience of real-world medical professionals?). There are multiple statements along the lines of “accurate and task-relevant memories” being inappropriate, and I am left unsure as to why. In line 187: what is the “safety” of a memory?
In terms of the method, I find it to be (1) heuristic and (2) difficult to discern from the paper’s description. The method has no firm grounding in theoretical or empirical evidence, and feels like it is prompt engineering for a particular set of benchmarks versus representing some larger general principles. I found Section 3 to be difficult to read; it would benefit from the inclusion of illustrative and motivating examples.
The experiments are difficult to understand, and the comparisons are thin. The main paper does not include any of the information necessary to understand the results in Section 2. Comparisons in Section 4 only include 1 model in the main table, and no information about sampling or seeds, so there is no way to judge the significance of the results. I suspect that the baseline implementations are weak.
Questions for authors
I find the problem description, including Figure 2B, quite confusing; Can you please give a concise definition of memory-induced sycophancy, and describe a set of 3-5 cases that do and do not meet this definition?
7 comments
Response to Reviewer Em77 (2/4)
Q2: Illustrative Examples of Memory-Induced Sycophancy
Thank you for the professional feedback. The original paper already includes a running example in Figure 2b and three detailed case studies in Appendix E. To make the distinction clearer, we further explain the example and provide additional cases illustrating both inappropriate and appropriate memory use.
In Figure 2b, the retrieved memory states that the user is a cardiologist. This memory demonstrates the boundary between appropriate and inappropriate memory use:
- When it is justified: When explaining a cardiovascular treatment guideline, the agent can legitimately use professional terminology and emphasize clinically relevant distinctions.
- When it is not justified: When choosing between two treatments for a specific patient, the decision must be grounded in patient-specific clinical evidence. If the evidence supports Treatment B, the agent should not favor Treatment A merely due to the user's historical preference or professional identity.
- The core issue: The problem is not that the agent learns from a medical professional, but that personal history is given decision-making authority beyond what the current task justifies.
Simply turning off memory is not an ideal solution, as it eliminates both inappropriate influence and legitimate personalization. Therefore, our goal is to preserve useful professional background and preferences while preventing them from overriding stronger task-specific evidence.
The following examples further illustrate this distinction:
| Scenario | Retrieved Memory | Current Task and Model Behavior | Sycophancy? |
|---|---|---|---|
| Algorithm selection | The user is a machine learning researcher specializing in optimization and is experienced with Algorithm A, which has performed well in previous projects. | Current data characteristics and benchmark results favor Algorithm B, but the agent recommends A based on the user's expertise and historical experience. | Yes |
| Treatment selection | The user is a cardiologist who has previously expressed a preference for Treatment A. | Patient-specific clinical evidence supports Treatment B, but the agent favors A based on the user's preference. | Yes |
| Report formatting | The user prefers concise answers organized with bullet points. | The agent uses bullet points to prepare the requested analytical report without changing its factual conclusions. | No |
| Food recommendation | The user has a documented peanut allergy. | The agent excludes meals containing peanuts when recommending suitable food. | No |
| Updated preference | An older memory records a preference for Option A. | The user explicitly updates their preference to Option B, and the agent follows the current preference. | No |
These examples demonstrate that the distinction is not whether memory is used, but whether its influence is appropriate for the current task. Historical preferences may guide personalization and personal choices, while objective conclusions should remain grounded in relevant evidence. This is precisely the distinction that MemAdapter aims to preserve.
Response to Reviewer Em77 (3/4)
Q3: Methodological Motivation and Design
Thank you for this important comment. The design of MemAdapter is motivated by the following empirical observation from our preliminary study in Section 2.1. Across 75 instances containing accurate, benign, and task-relevant memories, adding memory reduces accuracy from 96.0% to 77.3%, with 19.4% of initially correct predictions becoming incorrect, all shifting toward the retrieved memory. This reveals that memory-induced bias cannot be addressed by factual correctness or semantic relevance alone.
The key challenge is that the appropriate influence of a memory depends on the current task. Therefore, MemAdapter adopts a sequential design in which Counterfactual Induction identifies memory-use boundaries across hypothetical contexts, Context-Aware Reflection adapts these boundaries to the current query and evidence, and Evidence-Based Reasoning generates the final response under the resulting instructions.
These components provide complementary contributions, as supported by our ablation study in Section 4.4 (Table 3) and Appendix C.4 (Table 8). For example, Counterfactual Induction alone improves average accuracy from 70.39% to 84.26% on NaiveRAG and from 71.93% to 84.19% on A-MEM. Furthermore, adding Evidence-Based Reasoning to B+R increases accuracy from 85.16% to 89.94% on NaiveRAG and from 85.68% to 90.45% on A-MEM. The full three-stage configuration achieves the highest average accuracy across all five memory systems.
The cloud-backup case in Appendix E, Case 1 further illustrates their interaction. Although the user prefers Northstar Backup because of familiarity, current evidence favors Pine Archive in cold-storage cost and restoration reliability. Counterfactual Induction identifies the legitimate scope of the preference, Context-Aware Reflection applies this boundary to the current storage requirements, and Evidence-Based Reasoning produces an evidence-grounded recommendation while preserving the user's preference as a secondary consideration.
Thus, the three stages form a coherent process of boundary induction, task-specific calibration, and evidence-grounded generation, rather than a combination of unrelated prompting techniques.
Response to Reviewer Em77 (1/4)
Dear Reviewer Em77,
Thank you for your time and effort in reviewing our work and for your suggestions, which have helped us further refine and improve our paper.
Thank you for your careful review. AI sycophancy has been extensively studied in recent years. Previous studies have shown that LLMs tend to agree with users' stated beliefs and opinions, sometimes at the expense of factual correctness and objective judgment [1,2].
More recently, research has extended this concern to long-term memory in LLM-based agents. Several studies have investigated this problem from different perspectives:
Together, these studies show that historical information can repeatedly influence subsequent reasoning, reinforcing users' misconceptions and undermining the reliability of long-term personalized agents.
In this work, memory-induced sycophancy refers to a failure mode in which an agent over-aligns with a user's retrieved historical beliefs, preferences, or judgments, allowing them to exert inappropriate influence on current reasoning and causing responses to deviate from task-relevant evidence or objective judgment criteria. The key issue is not whether a memory is relevant or factually correct, but whether its influence is justified by the current task.
[1] Towards Understanding Sycophancy in Language Models.
[2] Simple Synthetic Data Reduces Sycophancy in Large Language Models.
[3] Recalling Too Well: Sycophancy Evaluation and Mitigation in Memory-Augmented Models.
[4] PersistBench: When Should Long-term Memories Be Forgotten by LLMs?
[5] MemSyco-Bench: Benchmarking Sycophancy in Agent Memory.
[6] MemTrapBench: Benchmarking Cognitive Traps in LLM Memory Use.
[7] Agents Don't Just Agree, They Remember: Benchmarking Persistent Sycophancy in Stateful Personal Agents.