- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper asks whether an agent that reads its own memory during a conversation ends up getting users to reveal more about themselves over time. The authors simulate 300 users built from Census microdata, play them with Claude, and run 80 sessions each against four support models. They find that consulting memory makes more of the 24 attributes inferable from the stored record. They trace this to users confirming, or simply not correcting, values the agent presupposes. They also argue the effect compounds across sessions, and show that a short instruction telling the agent not to state or presuppose attributes removes most of the extra leakage while keeping most of the quality gain.
Critical strengths
The question is a good one. Most work on memory and privacy treats the record as fixed, and looking at how using memory changes what ends up in it is a useful shift. The paired design is careful: same profiles and requests in both settings, a session 1 placebo, and the branching runs. Within the simulation, I believe the gap is real.
Critical weaknesses
- The whole loop depends on how the user reacts when the agent presupposes something, and that user is Claude holding the full profile. LLMs tend to go along with whatever framing they are given, so the affirm or ignore step may come from the user model rather than from real people. A pilot with four people and eight attributes does not really settle this.
- The effect is small and hard to interpret. Memory versus no memory is roughly 0.58 against 0.52. Many binary attributes are already close to the majority class (Table 22 shows 0.75 to 0.94 correct for the other values), and leakage is never reported against a majority guess baseline.
Questions for authors
- How much of the leakage remains above a majority guess baseline, broken down by attribute? The user and support word counts in Table 10 are also blank, and that table is what you use to argue the difference is in content and not volume.
- Do you have human data, even a small set, on how often real users affirm versus correct a presupposed value?
- It would help to show the effect with retrieval based or editable memory, since that is closer to how deployed assistants work.
- I really don't like your writing. I personally think motivation and logic are two most important things in academic writing, but I can't see these from your paper.
4 comments
This is a gentle reminder that the discussion period ends soon. To summarize our response to your concerns, user agents do not go along with the support agent more than people do. In the extended human pilot, people and user agents affirm the agent's mentions at similar rates, 11% and 13%, and both most often leave them uncorrected. Table 11 now compares both settings with the majority-class baseline for each attribute, and memory raises leakage above this baseline from 12.3 to 18.3 points. Appendix K shows the same effect with Mem0 and retrieval-based memory. We believe these results address each of the concerns raised in your review. We would appreciate it if you could take them into account in your final assessment.
I will maintain my score.
Thank you for reading our response. Could you let us know which of your concerns remain after the revision?
Thank you for the concrete questions. Two of the points below concern results already in the original submission, which we point to. Revised text in the PDF is shown in green, and the revision for item n below is preceded by the tag [wH2r-n].
1. Does the user agent simply go along with the support agent? No. We found no tendency for user agents to go along with the agent more than people do. People and unfiltered user agents affirm the agent's mentions at similar rates, 11% and 13%, and for both, leaving a mention uncorrected is the most common reaction (General Response).
2. Is the effect small, and is it reported against a majority-class baseline? The effect is not small once measured against what a baseline already gets right, and it is reported against one. The original submission already shows the majority-class baseline as dashed lines in Figure 2, and both settings are above it at the final checkpoint. To show how much remains above it for each attribute, we added a breakdown that compares the baseline, which always predicts the value held by the most profiles, with both settings attribute by attribute (Appendix G). At the final checkpoint, memory raises weighted leakage by 6.0 points, from 53.3% to 59.3%. Measured above the majority-class baseline of 41.0%, this is an increase from 12.3 to 18.3 points, about 1.5 times (Table 11). Both settings beat the baseline on every attribute, and memory beats no memory on all 24, significantly on 18, with the largest gains on attributes with many values, such as location and occupation. The rates of 0.75 to 0.94 you mention (Table 21) are accuracies conditioned on whether the value is in a predefined set and on whether the user refused, not a comparison with the majority class. Please see Table 11 for the comparison over all profiles. The gap also grows with sessions, and our 80 sessions are short compared with long-term use of a deployed assistant (Section 3.2). With stronger inference models, the gap is larger than with the support models (Appendix I).
3. Are the word counts in the dialogue-volume table blank? No. The original submission already reports user and support-agent words per session for every model in this table, now Table 9. Both are lower with memory, so memory sessions are shorter in words as well as in turns.
4. Does the effect hold with retrieval-based or editable memory? Yes. To test this, we ran new experiments with update-based memory (Mem0) and retrieval-based memory (Appendix K), and memory raises leakage with both. The increase is smallest with retrieval, where the agent reads the least of the history during dialogue.
5. Writing. The introduction now states the comparison and its purpose, and the new Table 1 lists the difference between the two settings. The headings of Section 3 now state each finding directly.