- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper studies how personalization in language models affects both task utility and identity risk. It formalizes leakage as the posterior identification gain, i.e. how much more probability an observer assigns to the intended user after seeing the model response. The authors then build a simple baseline that clusters users, writes a persona for each cluster, and retrieves documents from the cluster instead of from the individual. This is compared with no personalization and full individual personalization on several long- and short-form personalization benchmarks with several open-weight generators, using a re-identification attacker based on a language model and an LLM judge for personal voice. The take-away is that group-level context keeps part of the utility gain of individual context with less user attribution, while the benefit of finer personalization depends on the task.
Critical strengths
- Privacy in personalized language models is an important and timely problem.
- Viewing identity leakage through the posterior identification gain, and relating it to information measures and to conditional k-anonymity for personalization, is a clean way to formalize the problem.
- The empirical results give useful evidence that finer personal context adds user attribution without a consistent utility gain on some tasks, while the benefit of individual context varies across tasks.
Critical weaknesses
- The theoretical results are mostly direct consequences of standard tools (Bayes' rule, the data processing inequality, chi-squared and Brier risk identities, and k-anonymity). I do not see a new insight beyond the known fact that reducing the distinction between users' response distributions reduces what an observer can infer.
- The theory and the experiments are only loosely connected. Theorem 1 and Theorem 2 require identical response distributions within a group and a uniform prior within a group, but the implemented baseline (k-means clusters with a persona and cluster-wide BM25 retrieval) does not average response distributions. No experiment checks how close the baseline is to these conditions (e.g. an estimate of the within-group response difference or of the within-group posterior). The empirical results therefore do not support the k-anonymity claim.
- Using group- or cluster-level personas and retrieval is already studied in prior work such as ClusterRAG (cited in the related work), and the paper's baseline is a simple variant of it. The novelty therefore appears limited.
Questions for authors
- Can you provide evidence about how close the implemented group baseline is to the conditions of Theorem 1 and Theorem 2, e.g., by estimating the response difference between members of the same cluster and the within-cluster posterior of the attacker?
- How sensitive are the conclusions to the attacker? E.g., how do the results change when the number of candidates grows, or when a stronger attacker (e.g. a fine-tuned model that sees many samples per user) is used?
3 comments
Response to Reviewer ygdQ [2/2]
Response to Q2
"How sensitive are the conclusions to the attacker? E.g., … when a stronger attacker (e.g. a fine-tuned model that sees many samples per user) is used?"
The conclusions hold under a stronger attacker, and the stronger attacker makes them clearer (G2 in our general response, Table R2). GPT-6 Luna is more accurate than the Qwen attacker in every task and condition. It still shows no significant increase from a GrRAG response within a group, with or without reasoning, and on some tasks a GrRAG response even lowers its accuracy: a response that fits the whole group gives no reason to prefer the intended user over other members. The difference between Full Pers. and GrRAG widens on Tweet Paraphrase, Review Writing, and Topic Writing. This is what Eq. (3) implies: a better attacker extracts more from a response that contains identity information and nothing from one that does not.
The result within a group holds for a fine-tuned attacker as well. Eq. (3) is a property of the response distributions: for any attacker that observes the query, the odds between two members of the same group are the same with and without a GrRAG response. Training on many samples per user can sharpen the query-only guess and the identification of the group, not the separation of members.
"… how do the results change when the number of candidates grows …"
The conclusions hold from 2 to 20 candidates: a GrRAG response gives no significant increase within a group, and Full Pers. exceeds GrRAG on four tasks at every tested count.
Table R7. Change in top-1 identification accuracy relative to the query-only control, shown as GrRAG / Full Pers., with the other candidates from the target's group (Table 34 in Appendix K). Values are in percentage points for 2, 5, 10, and 20 candidates, on 300 items per task. Generator: Qwen3.5-9B. Attacker: Qwen3.6-35B-A3B. Candidate evidence: personal summaries. Items come from groups with at least 20 users who have a personal summary. A bold Full Pers. value is significantly above the GrRAG value after Holm correction. The test was run at 2, 10, and 20 candidates.
| Task | 2 | 5 | 10 | 20 |
|---|---|---|---|---|
| News Headline | +2.7 / +4.7 | +3.5 / +12.7 | +2.8 / +11.8 | +0.8 / +9.4 |
| Scholarly Title | +1.7 / +2.5 | −0.8 / +0.4 | +0.5 / +4.2 | 0.0 / +3.3 |
| Tweet Paraphrase | −2.3 / +15.2 | −1.5 / +27.2 | −0.8 / +23.3 | +0.4 / +21.7 |
| Abstract Generation | −2.8 / +6.8 | +0.7 / +14.2 | 0.0 / +13.8 | +1.7 / +12.4 |
| Review Writing | −5.3 / +18.5 | −4.4 / +30.2 | −1.3 / +28.9 | +0.6 / +31.6 |
| Topic Writing | −4.2 / +28.2 | −0.1 / +49.4 | +1.1 / +44.3 | −2.0 / +50.5 |
Results. Full Pers. exceeds GrRAG in every cell, and on four tasks the difference stays above 10 points at 20 candidates, where chance is 5%. Table R6 (response to 3xQb) shows the same comparison with candidates from the population.
Response to W3
"Using group- or cluster-level personas and retrieval is already studied in prior work such as ClusterRAG (cited in the related work), and the paper's baseline is a simple variant of it. The novelty therefore appears limited."
GrRAG differs from ClusterRAG in the property that determines identity leakage: all members of a group receive the same prompt. ClusterRAG keeps, for each user, the top most similar users within the same cluster, and its best setting combines their documents with the target user's own profile. Its context is specific to each user, so the condition of Theorem 1 does not hold. GrRAG is simple by design: one persona and one document pool per group give group members the same prompt.
Our evaluation isolates the retrieval pool, the choice on which the two methods differ. With the retriever and depth fixed, group retrieval has lower mean Leak than own-history retrieval in all nine comparisons, three LongLaMP tasks with three generators (Section 4.4).
Our contribution is to characterize and measure identity leakage for this growing line of methods, which personalize with other users' data and report task metrics only (ClusterRAG, CoPersona). We give a condition under which a response preserves within-group prior odds, a baseline that meets it, and an attack evaluation with controls on six tasks, with three generators and a second, stronger attacker.
We hope the new results and clarifications address your concerns, and we would be grateful if you would consider raising your score. We are happy to answer any further questions.
Follow-up before the discussion closes
Thank you again for your careful review. As the discussion period ends today, we wanted to gently follow up. We hope that our response addresses your concerns about how close GrRAG is to the conditions of the two theorems and about the sensitivity of the conclusions to the attacker, with new experiments on both in the revised paper. We look forward to any further questions or comments you may have.
Response to Reviewer ygdQ [1/2]
Thank you for your review and for recognizing the value of posterior identification gain. In response, we ran three new experiments: a stronger attacker (GPT-6 Luna), 2 to 20 candidates, and group averaging (Eq. 5) run directly. All three are reported in Appendix K of the revised paper.
Yes. GrRAG meets the condition of Theorem 1 by construction, and with two attackers a GrRAG response gives no significant increase in identification within a group after Holm correction. G1 and G2 in our general response give the details.
On the k-anonymity claim (W2): within a group, the query identifies a member and the GrRAG response does not. GrRAG satisfies Eq. (3) for any prior: no response can raise a member's probability above its value given the query and the group, which is 1/kG under a uniform conditional prior (Eq. 4). The median user belongs to a group of 20 to 252 users by task. On these benchmarks the query alone identifies the intended user among five group members well above the 20% chance level (Table R5 in our response to 3xQb).
A GrRAG response reveals the group, not the member, as Eq. (3) allows. On Topic Writing, it raises the Qwen attacker's accuracy by 20.1 points against candidates from the population and not at all against candidates from the same group. GPT-6 Luna shows the same pattern.
Thank you for this comment. The theory gives two results that can be tested, and both hold: a shared prompt gives no significant increase in identification within a group for two attackers, and under GrRAG the query, not the response, carries the identity information. To our knowledge, both are new for LLM personalization.
References
[1] Nkhata et al. ClusterRAG: Cluster-Based Collaborative Filtering for Personalized Retrieval-Augmented Generation. arXiv:2605.18769, 2026.
[2] Zhang et al. CoPersona: Collaborative Persona Graphs for Robust LLM Personalization. KDD 2026.
[3] Salemi and Zamani. Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculations. SIGIR 2026.
[4] Li et al. Profile-DP: Differential Privacy for User-Controlled Profile Release in Personalized LLM Inference. arXiv:2610.27213, 2026.
[5] Huang et al. InferPriv: Evaluating Inferential Privacy Leakage in LLM Assistants. arXiv:2610.34640, 2026.