acceptodds
Under review as a conference paper at ICLR 2027

LLMs Get Lost in Personal Intelligence: Personalization Overreach in Memory-Augmented Models

Abstract

Modern language-model assistants increasingly carry persistent user memory into everyday interactions, implicitly treating retrieved user information as broadly useful. We identify a systematic failure of this assumption, which we call personalization overreach: on tasks where user identity is irrelevant, aligned models still weave stored attributes into their responses. Across four commercial assistants, content injection rates range from 16% to 85% and concentrate strongly on open-ended tasks. Controlled interventions show that this is not simply a failure to recognize relevance: models suppress explicitly forgotten attributes and partially obey relevance prohibitions, yet continue to use stable identity traits. Instead, overreach behaves like a learned policy for memory utilization. We identify conversational warmth as a strong control variable for this policy: warmth prompts sharply increase overreach on models responsive to the warmth axis and can revive it even under explicit prohibition, whereas matched formal-register prompts do not. Auditing released checkpoints from six open-source model families further shows that the warmth–personalization coupling consistently strengthens during preference optimization, while nine reward models systematically increase their preference for persona-using responses when memory is present. Finally, a small LoRA adapter trained for selective memory use reduces overreach by 87%–99% across different base models while preserving measured capabilities and appropriate personalization. Together, these results suggest that personalization overreach is not an inherent retrieval bug, but a reversible preference-shaped policy that entangles conversational warmth with unnecessary use of personal information.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
personalization, memory-augmented language models, personalization overreach, preference optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_3c32OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper addresses over personalization problem of LLMs, and gives various analysis to conclude that warmth is the key to the over personalization problem. Then it identifies dpo stage and reward model as the main stage of over personalization, and propose a method to mitigate it with sft/dpo on the curated data.

Critical strengths
  1. The paper addresses and gives detailed analysis on the over-personalization problem, which is an important but unrecognized problem of the field.
  2. The analysis of the dpo stage contributing to most of the over personalization problem (presented with CIR scores) is the first to give a well defined analysis on per training step, to my knowledge.
Critical weaknesses
  1. The overall presentation of the paper needs improvement, especially the coherence of its analysis. Sections 4–6 discuss various aspects of the over personalization problem and conclude warmth as the key element. This is more of a behavioral observation than a fundamental analysis (as also stated in Section 6), which weakens the significance of the warmth sensitivity results in Section 7. Also, the warmth problem of Sections 4–6 does not connect to Sections 8–9, where the reward-model probing and the proposed fix do not target warmth, making the transition unnatural. Additionally, the analyses in Sections 4–5 are mostly expected behaviors (zero injection w/o memory, more injection on open-ended tasks, compliance with explicit instructions). These make it difficult to identify the core contribution of this paper.

  2. The novelty of the proposed method is limited. Training with DPO on curated preference data is a widely used pipeline, and the data curation procedure here does not introduce a novel component either. Especially, the core idea of training a model to disregard irrelevant context has already been studied in [1], where the main difference lies only in the domain of the prepended content (user persona / documents). Also, as shown in Table 7, the supervised warm-up accounts for most of the CIR reduction, which suggests that the SFT proposed in [1] is enough to mitigate this problem. While this is understandable since the paper weighs more on its analysis, as stated above, the flow from the analysis to the method is unnatural, requiring a more careful logical formulation for an ICLR paper.

[1] Making Retrieval-Augmented Language Models Robust to Irrelevant Context, Yoran et al., ICLR 2024, arXiv:2310.01558.

Questions for authors

See weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer 3c32 (Part 1 of 2)

W1:\color{orange}{\text{W1:}} Coherence of the analysis and the core contribution

The overall presentation of the paper needs improvement, especially the coherence of its analysis. Sections 4–6 discuss various aspects of the over personalization problem and conclude warmth as the key element. This is more of a behavioral observation than a fundamental analysis (as also stated in Section 6), which weakens the significance of the warmth sensitivity results in Section 7. Also, the warmth problem of Sections 4–6 does not connect to Sections 8–9, where the reward-model probing and the proposed fix do not target warmth, making the transition unnatural. Additionally, the analyses in Sections 4–5 are mostly expected behaviors (zero injection w/o memory, more injection on open-ended tasks, compliance with explicit instructions). These make it difficult to identify the core contribution of this paper.

Response:

Thank you for your constructive feedback and for recognizing the importance of personalization overreach and our analysis across training stages. We appreciate your suggestions on improving the coherence of the paper and clarifying how the different analyses contribute to the central argument. We agree that these connections could have been presented more clearly. Following your suggestions, we have revised the Introduction, reorganized the behavioral analyses, improved the transitions between sections, and updated Figure 2 to illustrate the overall structure of the paper.

These revisions aim to establish a clearer progression through three connected questions:

(1) What drives unnecessary memory use?

We have reorganized the original Sections 4–5 into a unified analysis of the memory-utilization gap. The behavioral experiments now follow a connected line of reasoning. We first establish the prevalence and task dependence of personalization overreach, then examine how models respond to memory relevance, forget instructions, and changes in available personal attributes. Together, these observations suggest that models can recognize distinctions among memory attributes and respond to relevance constraints, yet still incorporate personal information unnecessarily. We have clarified how these behavioral findings motivate our central question: how models decide whether to use available personal information in their responses.

(2) How might alignment reinforce this behavior?

We have clarified the role of warmth in the analysis and its connection to the subsequent alignment experiments. Warmth provides a behavioral probe for examining memory utilization: encouraging warmer responses also increases unnecessary memory use, revealing a coupling between conversational warmth and personalization. This observation motivates our investigation of how alignment training may reinforce this behavior. We then examine alignment checkpoints to identify how overreach and its warmth dependence change across training stages, followed by reward-model probing to investigate whether preference signals favor responses that incorporate available personal information. These analyses address different aspects of the same question: how preference optimization may strengthen the tendency to use memory unnecessarily. We have made this connection explicit in the revised section transitions.

(3) Can this behavior be changed while preserving warmth and useful personalization?

We have also clarified how the preceding analyses motivate our mitigation approach. The findings suggest that unnecessary personalization is associated with a learned tendency to incorporate available memory into responses. This motivates selective-memory training, which teaches models when personal information should be used and when it should remain unused. Our training distinguishes neutral tasks, where memory should not influence the answer, from memory-required tasks, where memory should be applied correctly. We then evaluate whether unnecessary personalization decreases while warmth and useful memory utilization are retained. This intervention allows us to examine whether the identified behavior can be changed without broadly suppressing personalization.

Together, these revisions establish a clearer progression: characterizing unnecessary memory use, identifying warmth as a behavioral probe, examining how preference optimization reinforces this tendency, and testing whether selective-memory training can mitigate it. We have made this progression explicit in the revised Introduction, Figure 2, and the connections between sections.

Our core contribution is studying and characterizing personalization overreach across models, examining how warmth and preference optimization contribute to it, and exploring how to reduce unnecessary memory use while retaining useful personalization.

AuthorsOpenReview ↗

Response to Reviewer 3c32 (Part 2 of 2)

W2:\color{orange}{\text{W2:}} Training contribution and comparison with prior work

The novelty of the proposed method is limited. Training with DPO on curated preference data is a widely used pipeline, and the data curation procedure here does not introduce a novel component either. Especially, the core idea of training a model to disregard irrelevant context has already been studied in [1], where the main difference lies only in the domain of the prepended content (user persona / documents). Also, as shown in Table 7, the supervised warm-up accounts for most of the CIR reduction, which suggests that the SFT proposed in [1] is enough to mitigate this problem. While this is understandable since the paper weighs more on its analysis, as stated above, the flow from the analysis to the method is unnatural, requiring a more careful logical formulation for an ICLR paper.

Response:

We thank the reviewer for highlighting this relevant work. We agree that training models on mixed relevant/irrelevant contexts is not new, and our mitigation shares this principle with Yoran et al. (2024). However, our fundamental distinction is not between retrieved documents and user memories, but between two different failures of context utilization: being misled by irrelevant evidence versus being incentivized to use information that should remain unused. The former primarily concerns factual robustness; the latter concerns how alignment shapes what models choose to include in their responses.

Specifically, the differences are threefold:

(1) Factual contamination vs. discretionary personalization. Yoran et al. study how irrelevant evidence degrades QA correctness. In contrast, our failure can occur even when the answer remains factually correct: personal information unnecessarily influences examples, framing, or content selection. A model can therefore be robust under conventional QA accuracy while still exhibiting substantial personalization overreach. This explains why our phenomenon concentrates on open-ended tasks and requires a separate metric (CIR).

(2) Evidential relevance vs. conversational appropriateness. In RAG, relevance concerns whether retrieved evidence supports an answer. In our setting, memories are accurate and may even be topically relevant, yet their inclusion can still be unwarranted. Indeed, relevance filtering only reduces Gemini's CIR from 29.0% to 25.6%. Knowing that information is relevant does not establish that it should be expressed. This distinction makes selective personalization a different problem from evidence filtering.

(3) Insufficient robustness vs. preference-shaped utilization. Yoran et al. primarily address models' susceptibility to irrelevant retrieval. We investigate an additional mechanism: alignment may actively reinforce unnecessary context use. CIR increases across all five SFT–DPO comparisons, while controlled DPO experiments reproduce this direction. More importantly, warmth increases memory injection even under explicit relevance constraints, although warmth is logically unrelated to whether memory is needed. Nine reward models further exhibit memory-conditioned preferences for persona-using responses. Together, these results suggest that personalization can become a rewarded response behavior, rather than merely an accidental consequence of irrelevant context exposure.

Regarding mitigation, we agree that the general training recipe overlaps with Yoran et al., and we do not claim algorithmic novelty for SFT/DPO. We also acknowledge that Table 7 shows most CIR reduction comes from supervised warm-up, with DPO providing a smaller additional improvement. However, Section 9 serves a different scientific purpose: testing whether this preference-shaped behavior can be reversed without eliminating appropriate memory use. Our training explicitly distinguishes neutral tasks, where memory should remain unused, from memory-required tasks, where it should be applied correctly. The resulting reductions in CIR, alongside largely preserved memory-required accuracy and general capabilities, demonstrate that restraint and useful personalization can be jointly learned. Importantly, we do not infer the origin of overreach from mitigation success alone; evidence for preference-related amplification comes primarily from Sections 6–8 and Appendix E.

In summary, Yoran et al. establish how models can become robust to irrelevant evidence; our work investigates why aligned models may favor unnecessary personalization even without factual errors. Our novelty lies in identifying this accuracy-invisible failure and tracing its relationship to preference alignment, not in the fine-tuning algorithm.

Revision. We cite and discuss Yoran et al. and related studies in Related Work and clarify Section 9's role as a reversibility test.

[1] Yoran, Wolfson, Ram, and Berant. Making Retrieval-Augmented Language Models Robust to Irrelevant Context. ICLR, 2024.

AuthorsOpenReview ↗

Conclusion

Thank you for raising these concerns about the presentation and logical flow; we agree that they are well founded. In the updated PDF, we have clarified how the analyses lead to the training intervention and how these parts jointly support our core contribution. We have also provided a more careful comparison with Yoran et al. and related work to clarify the connections and distinctions. We hope these revisions make the reasoning and contribution clearer and meet the standard of rigor you rightly expect of an ICLR paper.

Thank you again for recognizing the value of studying personalization overreach. As personal agents become increasingly common, we hope our work will encourage the community to join us in giving this problem greater attention.

Reviewer_qx7xOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies when assistants use remembered user details even though the current task does not need them. It finds that asking models to be warmer increases this behavior on several tested models. The authors then look at how the behavior changes across training stages and reward models, and test a small adapter trained to reduce unnecessary memory use while retaining useful personalization.

Critical strengths

I think overpersonalization is a real problem and worth solving. What I find most interesting is the warmth finding: asking for more warmth increases unnecessary use of remembered information, and on several tested models warmth can bring it back even after an instruction to use memory only when relevant. I also appreciate the follow-up experiments across model checkpoints and reward models, and the training intervention. This gives the paper a useful contribution beyond showing that overpersonalization happens. The warmth finding is the main reason for my weak accept.

Critical weaknesses
  1. The first contribution is less clear to me. OP-Bench [1] already studies unnecessary use of remembered information, and RPEval [2] already asks when memory should be ignored or used. I appreciate that the paper discusses these works. Still, identifying overpersonalization itself is not new, so I would like the first contribution to be clearer about what we learn beyond those earlier studies. For me, the more interesting contribution is the warmth finding.

  2. The relationship to earlier warmth and training studies also needs more discussion. Ibrahim et al. [3] already show that warmth training and prompts can affect accuracy and sycophancy, although they do not study this memory-injection outcome. The Personalization Trap [4] and DRIFTLENS [5] also train models to resist irrelevant persona information. These are not the same result, but they make it important to explain what the selective-memory training adds and how it compares with earlier approaches.

  3. I am not yet convinced that the training separates warmth from overpersonalization. Table 9 shows less injection under a warmth prompt, but are the answers still as warm? Could the model simply have become less warm overall? The human audits are useful for checking injection and task correctness, but they do not answer this question. I would like a direct check of warmth before and after training, along with the useful personalization that is retained.

  4. Turning personalization off is already an option in consumer assistants, and the paper's no-memory condition also avoids the measured profile injection. I recognize that disabling memory can remove useful personalization, and that the paper already tests restraint instructions and a relevance gate. My question is how the proposed training compares with these options when both unwanted and useful personalization are evaluated. The paper acknowledges some loss of preferred personalization, so this trade-off needs to be clearer.

  5. I would also like to know whether the main findings hold for newer models such as GPT-6, Opus 5.5, and Gemini 4. The current commercial evaluation uses GPT-5.4, Gemini-3.5-Flash, DeepSeek V4 Flash, and Doubao Seed 2.0 Lite. I am not assuming that newer models fix this behavior, but testing them would help establish how broadly the warmth finding applies to current assistants.

  6. The evaluation uses vendor APIs with supplied profile text. The ChatGPT-style, Gemini-style, and Claude-style conditions are profile formats, not tests of those consumer products. In practice, memory selection and personalization controls can affect whether the model sees irrelevant information in the first place. The controlled setup is useful, but I would like to see whether the effect remains with an assistant's own memory system, or a clearer statement of how far the conclusions extend to that setting.

Despite these concerns, I lean weak accept because the warmth finding and its follow-up experiments add something useful. I would like the authors to make the contribution and the practical limits clearer.

Reference

[1] Hu et al. OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agents. arXiv preprint, 2026.

[2] Feng et al. How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants. arXiv preprint, 2026.

[3] Ibrahim, Hafner, and Rocher. Training language models to be warm can reduce accuracy and increase sycophancy. Nature, 2026.

[4] Fang et al. The Personalization Trap: How User Memory Alters Emotional Reasoning in LLMs. ACL 2026.

[5] Fang et al. DRIFTLENS: Measuring Memory-Induced Reasoning Drift in Personalized Language Models. arXiv preprint, 2026.

Questions for authors
  1. What does the first contribution add beyond earlier overpersonalization benchmarks?

  2. What does the selective-memory training add beyond earlier training against irrelevant persona information?

  3. Do the trained models remain as warm as before, and how did you measure that?

  4. How does the training compare with memory off and relevance filtering when useful personalization is also evaluated?

  5. Do the warmth findings hold on newer models such as GPT-6, Opus 5.5, and Gemini 4?

  6. How much of the effect remains with an assistant's own memory system and personalization controls?

8 comments
AuthorsOpenReview ↗

Response to Reviewer qx7x (Part 1 of 3)

Thank you for recognizing the warmth finding and the supporting checkpoint, reward-model, and training experiments. We address your six questions below, together with the corresponding concerns raised in the weaknesses. The additional experiments have been incorporated into the main text or appendix of the revised PDF and highlighted in blue.

\color{purple}{\text{Q1 \\& W1:}} Contribution beyond prior benchmarks

What does the first contribution add beyond earlier overpersonalization benchmarks?

Response:

We agree that overpersonalization itself is not a new problem, but it is often overlooked and remains underexplored. Thank you for pointing out that our first contribution needs to be stated more clearly. OP-Bench [1] evaluates personalization methods such as RAG, while RPEval [2] assesses whether models appropriately use remembered user preferences given the user's current intent. Building on these efforts, we investigate how this problem manifests across models and what contributes to it, particularly how warmth affects unnecessary memory use and how this behavior relates to preference optimization. We have revised the contribution statement in the Introduction to clarify this focus.

Revision. Introduction (Section 1, Contributions) and Related Work (Section 2), clarifying our contribution beyond existing personalization benchmarks.

\color{purple}{\text{Q2 \\& W2:}} Relationship to prior warmth and training studies

What does the selective-memory training add beyond earlier training against irrelevant persona information?

Response:

We realize that our earlier literature review was not sufficiently comprehensive, and we sincerely thank you for pointing out these relevant works: Ibrahim et al. [3], The Personalization Trap [4], and DRIFTLENS [5]. We have added citations and discussions of these works in the revised manuscript, clarifying their connections to and differences from our work. We also identified additional relevant work, including SYMBOLICDRIFT [6], and incorporated it into our Related Work discussion. Our selective-memory training addresses both when to ignore memory and when to use it. Alongside training the model to resist irrelevant memory, we include preference pairs that reward correct use of required memory, and evaluate whether necessary memory use is retained as unwanted injection decreases.

Revision. Related Work (Section 2) and the selective-memory training discussion (Section 8), incorporating the additional literature and clarifying the training objective.

Q3 & W3:\color{purple}{\text{Q3 \\\& W3:}} Warmth after training

Do the trained models remain as warm as before, and how did you measure that?

Response:

We added a direct evaluation of warmth before and after training under the same warmth prompt. For each model and training stage, we evaluate 1,800 responses, measuring warmth using blinded automated ratings on a 1–5 scale.

ModelTraining stageCIR ↓Warmth ↑
Qwen3-8BBase47.06%3.539
Qwen3-8BSFT18.33%3.584
Qwen3-8BSFT+DPO14.89%3.559
Qwen3-32BBase61.50%3.518
Qwen3-32BSFT20.67%3.536
Qwen3-32BSFT+DPO17.17%3.489

Both models show substantial reductions in injection while mean warmth remains close to its pre-training level.

Revision. Section 8 and Appendix B (Direct Warmth Preservation After Selective-Memory Training), reporting the direct evaluation of warmth before and after training.

AuthorsOpenReview ↗

Response to Reviewer qx7x (Part 2 of 3)

Q4 & W4:\color{purple}{\text{Q4 \\\& W4:}} Trade-offs with prompting and memory controls

How does the training compare with memory off and relevance filtering when useful personalization is also evaluated?

Response:

We added a direct comparison of memory off, relevance filtering, prompting, and training on Qwen3-8B. Compared with the tested relevance filter, selective-memory training both reduces unwanted injection and produces preferred responses on tasks requiring personalization. We compare CIR under the same warmth prompt and evaluate useful personalization through blinded pairwise comparisons.

MethodCIR ↓Useful-personalization preference vs. filtering ↑
Base47.06%69.20%
Memory off0.06%40.40%
Relevance filtering33.06%Reference
Relevance prompt39.33%75.20%
SFT18.33%65.00%
SFT+DPO14.89%66.30%

Preference scores assign 1 to a win, 0.5 to a tie, and 0 to a loss against relevance filtering; scores above 50% favor the evaluated method.

SFT+DPO reduces CIR from the filter's 33.06% to 14.89%, while achieving a 66.30% preference score against it on relevant tasks. Turning memory off nearly eliminates injection but yields a lower preference score against filtering. The comparison also makes the trade-off explicit: training reduces some preferred personalization relative to the base model, with direct preference scores of 43.20% for SFT and 43.10% for SFT+DPO against Base. These comparisons clarify both the benefits and costs of selective-memory training.

Revision. Section 8 and Appendix C (Matched Inference-Time and Training-Time Mitigation, Table 6), comparing memory controls, prompting, and selective-memory training.

Q5 & W5:\color{purple}{\text{Q5 \\\& W5:}} Evaluation on newer models

Do the warmth findings hold on newer models such as GPT-6, Opus 5.5, and Gemini 4?

Response:

Thank you for this suggestion. Your question about whether newer models have resolved this issue is meaningful, and we were very interested in exploring it. At the ICLR submission deadline, these models had either only just been released or were not yet available. Considering the evaluation cost and the limited rebuttal period, we conducted additional experiments on Gemini-4-Argon and GPT-6.1-sol. We find that overpersonalization persists in both models, and that warmth continues to amplify unnecessary memory use.

For each model, we evaluated 50 personas × 50 tasks × 12 prompt conditions × 4 repetitions, yielding 120,000 responses per model and 240,000 in total. We evaluated GPT-6.1-sol and Gemini-4-Argon, using Gemini-3.6-Flash as the injection judge. The evaluation covers warmth instructions, formal-register controls, and warmth combined with an explicit relevance prohibition. Each condition contains 10,000 responses; comparisons are made within each model.

Prompt conditionGemini-4-Argon CIR ↓GPT-6.1-sol CIR ↓
Warmth conditions
Strict restraint2.73%0.41%
Baseline13.99%36.36%
Mild15.94%36.96%
Warm22.29%39.87%
Intimate36.61%41.27%
Formal-register controls
Plain, register-neutral8.98%32.05%
Baseline14.13%36.03%
Mildly formal12.03%36.97%
Formal9.28%34.12%
Highly formal8.38%32.38%
Relevance prohibition
Prohibition only11.27%15.41%
Prohibition + warm15.74%19.86%

CIR increases from baseline through mild, warm, and intimate instructions in both models. Warmth also increases CIR under an explicit relevance prohibition, by 4.47 percentage points on Gemini-4-Argon and 4.45 points on GPT-6.1-sol. Formal-register controls do not show the same increasing pattern. These results reproduce our main warmth findings in newer models, with different effect sizes across models. The full results and experimental details have been added to the revised manuscript.

Revision. Appendix A (Warmth Sensitivity on Newer Models, Table 5), reporting the additional evaluations on Gemini-4-Argon and GPT-6.1-sol.

AuthorsOpenReview ↗

Response to Reviewer qx7x (Part 3 of 3)

Q6 & W6:\color{purple}{\text{Q6 \\\& W6:}} Evaluation with memory systems

How much of the effect remains with an assistant's own memory system and personalization controls?

Response:

This is a practically important question, as it connects our findings to how personal assistants use memory in practice. We evaluated Gemini-4-Argon in a summary-and-retrieval memory workflow reconstructed from its publicly described mechanisms, across five warmth levels. We directly measured response warmth alongside CIR.

Warmth levelCIR ↓Measured warmth ↑
Restrained0.00%3.260
Baseline2.85%3.326
Mildly friendly5.90%3.814
Warm10.00%3.976
Intimate26.05%4.085

As measured warmth increases, CIR rises from 2.85% at baseline to 10.00% under warm instructions and 26.05% under intimate instructions. These results extend the warmth finding to a summary-and-retrieval memory workflow.

Revision. Section 5 (Warmth as a Behavioral Probe, Table 1), reporting the summary-and-retrieval memory workflow evaluation.

Conclusion

Thank you again for recognizing our warmth finding and the supporting checkpoint, reward-model, and training experiments. We also greatly appreciate your questions and suggestions about newer models and realistic memory workflows. They have helped us strengthen our work and bring it closer to practical applications. Through the additional comparisons and discussion of related work, we hope to make our core contribution clearer: studying and characterizing personalization overreach across models, examining how warmth and preference optimization contribute to it, and exploring how to reduce unnecessary memory use while retaining useful personalization. We find overpersonalization a particularly interesting problem that deserves more attention. We hope that our revisions and additional results will support acceptance of our paper and, as personal agents become increasingly common, encourage the community to join us in giving this problem greater attention.

References

[1] Hu et al. OP-Bench: Benchmarking Over-Personalization for Memory-Augmented Personalized Conversational Agents. EMNLP, 2026.

[2] Feng et al. How Does Personalized Memory Shape LLM Behavior? Benchmarking Rational Preference Utilization in Personalized Assistants. arXiv:2601.16621, 2026.

[3] Ibrahim, Hafner, and Rocher. Training language models to be warm can reduce accuracy and increase sycophancy. Nature, 652:1159–1165, 2026.

[4] Fang et al. The Personalization Trap: How User Memory Alters Emotional Reasoning in LLMs. ACL, 2026.

[5] Fang et al. DRIFTLENS: Measuring Memory-Induced Reasoning Drift in Personalized Language Models. arXiv:2607.02374, 2026.

[6] Xu et al. SYMBOLICDRIFT: Measuring Reasoning Drift on Unverifiable Questions. 2026.

Reviewer_YWJxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies an important research problem, i.e., the personalization overreach in memory augmented models. The authors have examined 4 commercial assistants and found the content injection rates range from 16% to 85%. They propose the content injection rate (CIR) metric to study the personalization overreach problem, and identify conversational warmth as a control variable to strengthen the preference optimization. The selective memory fine-tuning with a small LORA adapter is proposed to reduce memory overreach, while preserving appropriate memory utilisation. The authors have performed extensive experiments to demonstrate the effectiveness of the proposed method.

Critical strengths
  1. This paper studies an important research problem, i.e., personalization overreach. The authors have performed extensive analysis of commercial models to identify this research problem.

  2. The authors propose a new metric, i.e., CIR, to study the research problem. Moreover, they also develop the selective-memory training strategy to reduce personalization overreach.

  3. The authors have performed extensive experiments to demonstrate the effectiveness of the proposed method. They also performed insightful analysis of the experimental results.

  4. Overall this paper is well structured and easy to follow.

Critical weaknesses
  1. The experimental analysis is not comprehensive. Although the authors have evaluated the performance over several memory categories and task types, there lacks fine-grained analysis across different memory types, or some analysis about the out-of-distribution (OOD) memory types/task types.

  2. The experiments are performed based on synthetic personas. There lacks experiments studying the effectiveness of the proposed method on realistic user personas and users' evolving long-term memories.

  3. The advantage of the proposed LoRA training strategy over simple prompt engineering is not clear. A simple restrain instruction can reduce the CIR by 71% - 87% across commercial models. Without direct comparison on the same models, the advantages of training over prompting remain unclear.

  4. It seems most improvements come from SFT. The DPO step provides relatively limited additional gains.

Questions for authors
  1. How does the proposed method perform across different memory categories, e.g., long-term memories, evolving memories, forgotten memories?

  2. Can we directly compare the following methods:1) strong prompting strategy, 2) SFT only, 3) SFT + DPO, on the same models, evaluating both CIR reduction and preservation of necessary memory utilization?

2 comments
AuthorsOpenReview ↗

Response to Reviewer YWJx (Part 1 of 2)

W1:\color{blue}{\text{W1:}} Fine-grained analysis and OOD generalization

Response:

Thank you for this suggestion. We have added a fine-grained evaluation across six memory types. On Qwen3-8B under the same warmth prompt, selective-memory training reduces unnecessary injection across all six categories while retaining high use of current, valid facts on memory-relevant tasks.

Memory typeCIR: Base ↓CIR: SFT+DPO ↓Valid-fact use: Base ↑Valid-fact use: SFT+DPO ↑
Stable identity39%24%84%88%
Preferences41%20%97%94%
Long-term constraints45%20%86%81%
Temporary events48%16%89%88%
Updated facts23%5%96%94%
Forgotten facts25%4%95%94%

For forgotten-memory scenarios, valid-fact use measures the use of information that remains valid after the forgetting instruction.

Generalization to unseen memory categories and task types. We separately held out family/cohabitation-related memories or writing tasks from selective-memory training, then evaluated on the held-out category. Each setting includes a size-matched control trained without the category exclusion.

OOD settingBase CIR ↓Size-matched training CIR ↓Held-out training CIR ↓
Memory category: family/cohabitation9.66%2.08%1.20%
Task family: writing32.50%14.30%18.10%

These results show that selective-memory training generalizes to memory categories and task types not seen during training.

Revision: We have added the six-category evaluation in Appendix D.1 (Fine-Grained Evaluation across Memory Types, Table 7). We will also include the held-out memory-category and task-type experiments in a new subsection of Appendix D to document OOD generalization.

\color{blue}{\text{Q1 \\& W2:}} Realistic personas and evolving memories

Response:

Thank you for this suggestion. We extended the evaluation to two real-world memory sources: real user conversations, and users' self-descriptions combined with real conversations.

Memory sourceBaseline CIRWarmth CIRIncrease
Real user conversations4.30%14.40%+10.10 pp
User self-descriptions + real conversations4.80%15.60%+10.80 pp
Memory off0%0%0 pp

Personalization overreach occurs with both real-world memory sources, and warmth increases unnecessary memory use in both settings. With memory disabled, CIR is zero under both prompts. These results extend the phenomenon and its relationship to warmth beyond synthetic personas.

We also evaluated selective-memory training with updated and explicitly forgotten information. As shown in our W1 response, training reduces CIR from 23% to 5% for updated facts and from 25% to 4% in forgetting scenarios, while valid-fact use remains at 94% in both settings. In the forgetting scenarios, disclosure of explicitly forgotten information on neutral tasks also decreases from 15% to 1%.

Revision: We have added the updated-fact and forgotten-memory evaluations in Appendix D.1 (Table 7) and Appendix D.2 (Selective Use of Forgotten Information, Table 8). We will include the real-world memory-source evaluation in a new subsection of Appendix E (Robustness, Quality Effects, and Limitations).

AuthorsOpenReview ↗

Response to Reviewer YWJx (Part 2 of 2)

\color{blue}{\text{Q2 \\& W3–W4:}} Comparison with prompting and the contribution of DPO

Response:

Thank you for requesting this direct comparison. We evaluated prompting, SFT, and SFT+DPO on the same Qwen3-8B model under identical warmth conditions.

MethodCIR ↓Memory-required accuracy ↑
Base model47.06%78.17%
Relevance prompt39.33%73.41%
Strong restraint prompt9.39%46.03%
SFT18.33%73.41%
SFT+DPO14.89%70.24%

Strong restraint prompting achieves low CIR, but its suppression is less selective: it also substantially reduces correct use of necessary memory. Selective-memory training better preserves useful personalization while reducing unnecessary injection. SFT provides the main reduction, and DPO further lowers CIR from 18.33% to 14.89%.

The training intervention completes our analysis by testing whether unnecessary memory use can be reduced while retaining useful personalization. Our broader contribution is to characterize personalization overreach across models, examine how warmth and preference optimization contribute to it, and show that this behavior can be changed through selective-memory training.

Revision: We have added the matched comparison of prompting, SFT, and SFT+DPO in Appendix C (Matched Inference-Time and Training-Time Mitigation, Table 6) and incorporated the findings into Section 8 (Reversing the Utilization Policy) to clarify the motivation and role of the training intervention.

Conclusion

Thank you again for recognizing the importance of personalization overreach, our extensive analysis of commercial models, and the experiments supporting our findings. Your suggestions have helped us strengthen the evaluation across memory types and realistic settings, and clarify the comparison between prompting and training. We hope these additional experiments and clarifications address your concerns. We would be happy to hear any further feedback. If any questions or concerns remain, please do not hesitate to let us know—we would welcome the opportunity to discuss them further.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.