acceptodds
Under review as a conference paper at ICLR 2027

Personalization Done Right: Balancing Personalization and Privacy

Abstract

Personalized language models typically condition on user-specific context to better serve individual needs. However, the very data that enables effective personalization, such as user profiles, interaction histories, and preference signals, simultaneously increases privacy risk. Despite the growing importance of personalization and privacy in language modeling, their joint analysis remains limited. In this work, we study how personalization granularity affects task utility and user identifiability. We formalize a setting in which a user issues a prompt and the model has access to a user profile of varying granularity, and we characterize identity leakage through posterior identification gain. We then show that idealized group averaging reduces identity leakage under stated conditions, with utility change bounded by within-group utility differences. We study this tradeoff across a spectrum of personalization techniques, including retrieval-augmented generation, persona-based prompting, profile-augmented prompting, and provide extensive empirical findings that support its practical implications. Our findings have significant implications for the development of personalized language models that respect user privacy.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Personalization, Privacy

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ygdQOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how personalization in language models affects both task utility and identity risk. It formalizes leakage as the posterior identification gain, i.e. how much more probability an observer assigns to the intended user after seeing the model response. The authors then build a simple baseline that clusters users, writes a persona for each cluster, and retrieves documents from the cluster instead of from the individual. This is compared with no personalization and full individual personalization on several long- and short-form personalization benchmarks with several open-weight generators, using a re-identification attacker based on a language model and an LLM judge for personal voice. The take-away is that group-level context keeps part of the utility gain of individual context with less user attribution, while the benefit of finer personalization depends on the task.

Critical strengths
  • Privacy in personalized language models is an important and timely problem.
  • Viewing identity leakage through the posterior identification gain, and relating it to information measures and to conditional k-anonymity for personalization, is a clean way to formalize the problem.
  • The empirical results give useful evidence that finer personal context adds user attribution without a consistent utility gain on some tasks, while the benefit of individual context varies across tasks.
Critical weaknesses
  • The theoretical results are mostly direct consequences of standard tools (Bayes' rule, the data processing inequality, chi-squared and Brier risk identities, and k-anonymity). I do not see a new insight beyond the known fact that reducing the distinction between users' response distributions reduces what an observer can infer.
  • The theory and the experiments are only loosely connected. Theorem 1 and Theorem 2 require identical response distributions within a group and a uniform prior within a group, but the implemented baseline (k-means clusters with a persona and cluster-wide BM25 retrieval) does not average response distributions. No experiment checks how close the baseline is to these conditions (e.g. an estimate of the within-group response difference or of the within-group posterior). The empirical results therefore do not support the k-anonymity claim.
  • Using group- or cluster-level personas and retrieval is already studied in prior work such as ClusterRAG (cited in the related work), and the paper's baseline is a simple variant of it. The novelty therefore appears limited.
Questions for authors
  • Can you provide evidence about how close the implemented group baseline is to the conditions of Theorem 1 and Theorem 2, e.g., by estimating the response difference between members of the same cluster and the within-cluster posterior of the attacker?
  • How sensitive are the conclusions to the attacker? E.g., how do the results change when the number of candidates grows, or when a stronger attacker (e.g. a fine-tuned model that sees many samples per user) is used?
3 comments
AuthorsOpenReview ↗

Response to Reviewer ygdQ [1/2]

Thank you for your review and for recognizing the value of posterior identification gain. In response, we ran three new experiments: a stronger attacker (GPT-6 Luna), 2 to 20 candidates, and group averaging (Eq. 5) run directly. All three are reported in Appendix K of the revised paper.

Response to Q1 (and W2)


"Can you provide evidence about how close the implemented group baseline is to the conditions of Theorem 1 and Theorem 2, e.g., by estimating the response difference between members of the same cluster and the within-cluster posterior of the attacker?"

Yes. GrRAG meets the condition of Theorem 1 by construction, and with two attackers a GrRAG response gives no significant increase in identification within a group after Holm correction. G1 and G2 in our general response give the details.

  • Response difference between members of the same group: zero by construction. For a fixed query, GrRAG gives each group member the same prompt, so their response distributions are identical. Theorem 1 requires only this, not the averaging of Eq. (5). We have stated this link in Section 4.1.
  • Posterior of the attacker within a group: a GrRAG response carries no more identity signal than a response with no user information. Both shift the probability the attacker assigns to the intended user by a few points at most, while Full Pers. shifts it by up to 49.9. In top-1 accuracy, GrRAG is never significantly above the No Pers. response for either attacker, while Full Pers. is above it on at least five of six tasks.
  • Theorem 2: GrRAG gives up no measurable Util relative to group averaging (Eq. 5). We ran group averaging under a uniform conditional prior: GrRAG has higher Util on Tweet Paraphrase and no significant difference on the other five tasks (Table R1 in G1).

On the kk-anonymity claim (W2): within a group, the query identifies a member and the GrRAG response does not. GrRAG satisfies Eq. (3) for any prior: no response can raise a member's probability above its value given the query and the group, which is 1/kG1/k_G under a uniform conditional prior (Eq. 4). The median user belongs to a group of 20 to 252 users by task. On these benchmarks the query alone identifies the intended user among five group members well above the 20% chance level (Table R5 in our response to 3xQb).

A GrRAG response reveals the group, not the member, as Eq. (3) allows. On Topic Writing, it raises the Qwen attacker's accuracy by 20.1 points against candidates from the population and not at all against candidates from the same group. GPT-6 Luna shows the same pattern.

Response to W1


"The theoretical results are mostly direct consequences of standard tools (Bayes' rule, the data processing inequality, chi-squared and Brier risk identities, and k-anonymity). I do not see a new insight beyond the known fact that reducing the distinction between users' response distributions reduces what an observer can infer."

Thank you for this comment. The theory gives two results that can be tested, and both hold: a shared prompt gives no significant increase in identification within a group for two attackers, and under GrRAG the query, not the response, carries the identity information. To our knowledge, both are new for LLM personalization.

  • Recent work does not state or test it. Methods that use other users' data report task metrics only [1, 2]. Recent defenses protect the profile. They give no formal result on the response [3] or do not test identification of the user [4]. A benchmark of inference from responses offers no guarantee [5].
  • The known fact is qualitative, and Eq. (3) is exact. For any attacker that observes the query, a GrRAG response leaves the odds within a group unchanged.
  • The known fact does not say what to change. Shortening the context is not enough. Sharing it is what matters. Within a group, a response from an individual summary raises accuracy by up to 54.8 points, while one from a group persona alone never raises it (Table 12, rows "summaries, within cohort").
  • The known fact is silent on the query. The query alone identifies the intended user far above chance. Without a query-only control, GrRAG would appear to leak up to 40.5 points above chance.

References

[1] Nkhata et al. ClusterRAG: Cluster-Based Collaborative Filtering for Personalized Retrieval-Augmented Generation. arXiv:2605.18769, 2026.

[2] Zhang et al. CoPersona: Collaborative Persona Graphs for Robust LLM Personalization. KDD 2026.

[3] Salemi and Zamani. Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculations. SIGIR 2026.

[4] Li et al. Profile-DP: Differential Privacy for User-Controlled Profile Release in Personalized LLM Inference. arXiv:2610.27213, 2026.

[5] Huang et al. InferPriv: Evaluating Inferential Privacy Leakage in LLM Assistants. arXiv:2610.34640, 2026.

AuthorsOpenReview ↗

Response to Reviewer ygdQ [2/2]

Response to Q2


"How sensitive are the conclusions to the attacker? E.g., … when a stronger attacker (e.g. a fine-tuned model that sees many samples per user) is used?"

The conclusions hold under a stronger attacker, and the stronger attacker makes them clearer (G2 in our general response, Table R2). GPT-6 Luna is more accurate than the Qwen attacker in every task and condition. It still shows no significant increase from a GrRAG response within a group, with or without reasoning, and on some tasks a GrRAG response even lowers its accuracy: a response that fits the whole group gives no reason to prefer the intended user over other members. The difference between Full Pers. and GrRAG widens on Tweet Paraphrase, Review Writing, and Topic Writing. This is what Eq. (3) implies: a better attacker extracts more from a response that contains identity information and nothing from one that does not.

The result within a group holds for a fine-tuned attacker as well. Eq. (3) is a property of the response distributions: for any attacker that observes the query, the odds between two members of the same group are the same with and without a GrRAG response. Training on many samples per user can sharpen the query-only guess and the identification of the group, not the separation of members.

"… how do the results change when the number of candidates grows …"

The conclusions hold from 2 to 20 candidates: a GrRAG response gives no significant increase within a group, and Full Pers. exceeds GrRAG on four tasks at every tested count.

Table R7. Change in top-1 identification accuracy relative to the query-only control, shown as GrRAG / Full Pers., with the other candidates from the target's group (Table 34 in Appendix K). Values are in percentage points for 2, 5, 10, and 20 candidates, on 300 items per task. Generator: Qwen3.5-9B. Attacker: Qwen3.6-35B-A3B. Candidate evidence: personal summaries. Items come from groups with at least 20 users who have a personal summary. A bold Full Pers. value is significantly above the GrRAG value after Holm correction. The test was run at 2, 10, and 20 candidates.

Task251020
News Headline+2.7 / +4.7+3.5 / +12.7+2.8 / +11.8+0.8 / +9.4
Scholarly Title+1.7 / +2.5−0.8 / +0.4+0.5 / +4.20.0 / +3.3
Tweet Paraphrase−2.3 / +15.2−1.5 / +27.2−0.8 / +23.3+0.4 / +21.7
Abstract Generation−2.8 / +6.8+0.7 / +14.20.0 / +13.8+1.7 / +12.4
Review Writing−5.3 / +18.5−4.4 / +30.2−1.3 / +28.9+0.6 / +31.6
Topic Writing−4.2 / +28.2−0.1 / +49.4+1.1 / +44.3−2.0 / +50.5

Results. Full Pers. exceeds GrRAG in every cell, and on four tasks the difference stays above 10 points at 20 candidates, where chance is 5%. Table R6 (response to 3xQb) shows the same comparison with candidates from the population.

Response to W3


"Using group- or cluster-level personas and retrieval is already studied in prior work such as ClusterRAG (cited in the related work), and the paper's baseline is a simple variant of it. The novelty therefore appears limited."

GrRAG differs from ClusterRAG in the property that determines identity leakage: all members of a group receive the same prompt. ClusterRAG keeps, for each user, the top kk most similar users within the same cluster, and its best setting combines their documents with the target user's own profile. Its context is specific to each user, so the condition of Theorem 1 does not hold. GrRAG is simple by design: one persona and one document pool per group give group members the same prompt.

Our evaluation isolates the retrieval pool, the choice on which the two methods differ. With the retriever and depth fixed, group retrieval has lower mean Leak than own-history retrieval in all nine comparisons, three LongLaMP tasks with three generators (Section 4.4).

Our contribution is to characterize and measure identity leakage for this growing line of methods, which personalize with other users' data and report task metrics only (ClusterRAG, CoPersona). We give a condition under which a response preserves within-group prior odds, a baseline that meets it, and an attack evaluation with controls on six tasks, with three generators and a second, stronger attacker.


We hope the new results and clarifications address your concerns, and we would be grateful if you would consider raising your score. We are happy to answer any further questions.

AuthorsOpenReview ↗

Follow-up before the discussion closes

Thank you again for your careful review. As the discussion period ends today, we wanted to gently follow up. We hope that our response addresses your concerns about how close GrRAG is to the conditions of the two theorems and about the sensitivity of the conclusions to the attacker, with new experiments on both in the revised paper. We look forward to any further questions or comments you may have.

Reviewer_3xQbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies the trade-off between personalization and identity leakage in LLM-generated responses, proposing group-level personalization as an intermediate alternative to generic and individual personalization. It formalizes leakage through posterior identification gain and derives conditional anonymity guarantees and leakage–utility bounds for idealized group mechanisms. Motivated by this analysis, it introduces GrRAG, which clusters users based on profile summaries and generates responses using a shared group persona and retrieved documents from the group’s histories. Experiments on six LaMP and LongLaMP tasks compare personalization granularities using reference-based metrics, LLM-judged personal-voice similarity, and five-candidate user-identification attacks. The reported results suggest that group personalization can reduce measured attribution relative to individual personalization, with task-dependent effects on output quality and personal-voice similarity.

Critical strengths
  1. The paper addresses an important problem: how to retain the benefits of personalization while limiting identity exposure. Exploring group-level context as an intermediate option between generic and fully individualized generation is a useful direction for studying this trade-off.
Critical weaknesses
  1. The evaluation and threat model do not convincingly establish the claimed utility–privacy trade-off. The main “Util” metric measures similarity to a user’s writing style, which does not establish usefulness, correctness, or user preference. The privacy evaluation assumes that the attacker possesses histories or summaries for five candidates, with the target guaranteed to be included, without sufficiently grounding this substantial auxiliary knowledge and reliable shortlist in a concrete deployment scenario. Sensitivity to candidate count is not established, and the forced-choice protocol does not evaluate false attribution when the target is absent. Moreover, the privacy metric measures a fixed LLM attacker’s increase in top-1 identification accuracy, rather than the theoretical posterior leakage. Low measured leakage can reflect attacker limitations or distraction, while unchanged predictions can conceal increased confidence. The generic-response control illustrates this ambiguity: adding uninformative text reduces identification accuracy from 56% to 26% on Review Writing (Table 21). The limited second-attacker evaluation does not resolve these measurement concerns. Finally, substantive positioning against DP-ICL and PromptPATE remains missing: although Duan et al. is cited, the paper provides neither an appropriate experimental comparison nor sufficient explanation of differences in applicability and protection guarantees. Together, these issues leave the practical significance of the reported trade-off insufficiently established.

  2. The theory does not directly justify the implemented method. Theorem 2 analyzes a mixture of individual response distributions, whereas GrRAG generates from shared group context, these mechanisms are not equivalent. The paper also does not establish that clustering user-summary embeddings produces small within-group utility differences, which are needed for Eq. (7) to provide a small utility-change bound. Although the authors acknowledge these limitations, the theorem offers motivation rather than a guarantee of GrRAG’s privacy–utility benefits

Questions for authors
  1. Utility metrics validity. What evidence connects the Util score to actual usefulness or user preference? Could you validate the claimed benefits through user judgments or task-specific measures beyond stylistic similarity?
  2. Privacy metric validity. How do you validate identification-accuracy gain as a measure of privacy leakage? A small gain could reflect either little identifying information in the response or the attacker’s inability to use that information. What evidence distinguishes these explanations and supports interpreting GrRAG’s lower score as a privacy improvement?
  3. Threat model and candidate selection. Who is the intended attacker, and how would they obtain candidate histories or summaries and a shortlist guaranteed to contain the target? Why use five candidates, and how sensitive are the conclusions to candidate count and similarity? How would the attack handle cases where the target is absent?
  4. Connection to the theory. Section 3 acknowledges that Theorem 2 does not certify GrRAG. Could you compare GrRAG with the distribution-averaging mechanism in Eq. (5)? Does clustering user-summary embeddings produce groups with small within-group utility differences, as defined in Section 3, and what evidence supports this?
  5. Comparison with prior approaches. How does the protected information and threat model differ from DP-ICL and PromptPATE? Could you provide a comparison on compatible tasks, or explain precisely why these approaches cannot serve as suitable baselines?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 3xQb [1/2]

Thank you for recognizing the importance of preserving useful personalization while limiting identity exposure. Tables R1 and R2 are in our general response.

Response to Q1


"What evidence connects the Util score to actual usefulness or user preference? Could you validate the claimed benefits through user judgments or task-specific measures beyond stylistic similarity?"

We report the standard task-specific metrics and add Util as a second measure, validated with controls.

  • Task-specific measures. ROUGE-1/L and METEOR are the standard metrics of LaMP and LongLaMP. The reference is the user's own text for the same query, so they compare a response with the user's own choice. GrRAG improves METEOR over No Pers. in all nine task–generator comparisons on LongLaMP (Table 2).
  • Util is specific to the user. It scores personal voice against the user's writing and passes the profile-discrimination control on five of six tasks (Appendix G). In a new run on these five tasks, a Full Pers. response scores 0.27–1.31 points higher (1–5 scale) against the writing of its intended user than against users from other groups.
  • The paper reads utility together with attribution. From K=10K=10 to K=200K=200 on Topic Writing, Leak rises by 14.29 percentage points while Util stays flat (Section 4.3), so utility alone would hide the cost of finer personalization.

Response to Q2


"A small gain could reflect either little identifying information in the response or the attacker’s inability to use that information. What evidence distinguishes these explanations and supports interpreting GrRAG’s lower score as a privacy improvement?"

Thank you for this insightful question. We have added a stronger attacker (GPT-6 Luna), which further confirms that GrRAG's lower Leak reflects less identity information.

What we did. GPT-6 Luna with reasoning attacked the same 300 items per task as the Qwen attacker, with the same candidates, and is more accurate on every task (Appendix K.2 of the revised paper). G2 in our general response adds 20 candidates (Table R2).

Table R5. Top-1 identification accuracy (%), shown as Qwen / GPT-6 Luna attacker, four other candidates from the target's group (Table 33 in Appendix K of the revised paper). Generator Qwen3.5-9B, summary evidence.

TaskQuery-onlyGrRAGFull Pers.
News Headline31.9 / 49.335.4 / 42.744.7 / 60.0
Scholarly Title61.3 / 70.060.5 / 67.061.8 / 70.7
Tweet Paraphrase31.7 / 47.330.2 / 34.758.8 / 69.3
Abstract Generation55.8 / 60.756.5 / 58.770.0 / 72.0
Review Writing30.3 / 38.025.8 / 32.060.4 / 72.3
Topic Writing37.2 / 51.337.2 / 41.086.7 / 93.3

Results. Our conclusions remain under the stronger GPT-6 Luna attacker, and two results separate the two explanations.

  • The attackers use identity information when it is present. A Full Pers. response raises accuracy over query-only by 13–49 points for Qwen and 11–42 for GPT-6 Luna on five tasks.
  • A GrRAG response adds none. For both attackers it gives no reliable increase over the query-only level, and it is never significantly above a No Pers. response (−8.7 to +1.7 points for GPT-6 Luna), which has the same format and no user information.

Remarks.

  • Theorem 1 predicts this result. For a fixed query, GrRAG gives each group member the same prompt, so the response contains no information that separates group members beyond the query (Eq. 3).
  • Population candidates. Under GPT-6 Luna, Full Pers. also exceeds GrRAG when the other candidates come from the population, on the same three tasks as for Qwen in Table R6.
  • Confidence. With Full Pers. the Qwen attacker's probability for the intended user rises by 3.6–23.2 points, more than with GrRAG on every task.

"adding uninformative text reduces identification accuracy from 56% to 26% on Review Writing (Table 21)"

This shift appears only with candidates from the population. With four candidates from the target's group, the same generic response changes accuracy by +1 point. Relative to the No Pers. review for the same query, Full Pers. raises identification by more than twice as much as GrRAG (Table R3 in our response to Reviewer 4jbF).

AuthorsOpenReview ↗

Response to Reviewer 3xQb [2/2]

Response to Q3


"Who is the intended attacker, and how would they obtain candidate histories or summaries and a shortlist guaranteed to contain the target?"

A reader of a posted response who holds public writing by possible authors. The tasks produce public text, and candidate histories are earlier texts by users of the same platform. A shortlist that contains the target favors the attacker, as in [1]. Eq. (3) holds for any prior over users, so it holds for any shortlist.

"Why use five candidates, and how sensitive are the conclusions to candidate count and similarity?"

Five candidates follow [1]. The conclusions hold from 2 to 20 candidates, and the difference is larger when the other candidates are similar users (members of the target's group).

Table R6. Full Pers. minus GrRAG in top-1 identification accuracy (percentage points), by source of the other candidates: population / target's group (Table 35 in Appendix K). 300 items per task, generator Qwen3.5-9B, summary evidence. Bold: significant after Holm correction at the new counts (2, 10, 20).

Task251020
News Headline−2.2 / +2.0−0.7 / +9.2+2.6 / +8.9+2.0 / +8.7
Scholarly Title−0.2 / +0.8−1.0 / +1.3+0.3 / +3.7+1.7 / +3.3
Tweet Paraphrase+13.2 / +17.5+22.1 / +28.7+19.8 / +24.1+19.2 / +21.2
Abstract Generation+0.5 / +9.7+1.0 / +13.5+2.1 / +13.8+4.3 / +10.7
Review Writing+5.2 / +23.8+14.2 / +34.6+14.8 / +30.2+21.3 / +31.1
Topic Writing+10.2 / +32.3+19.2 / +49.5+21.3 / +43.2+22.8 / +52.5

Results. With candidates from the population, Full Pers. exceeds GrRAG at every count on Tweet Paraphrase and Topic Writing, and at 10 and 20 on Review Writing. With candidates from the target's group, GrRAG shows no significant increase over query-only at any count.

"How would the attack handle cases where the target is absent?"

The forced-choice attack always names a candidate. Its accuracy with the target present is an upper bound on correct identification when the target may be absent.

Response to Q4


"Could you compare GrRAG with the distribution-averaging mechanism in Eq. (5)?"

Both give all members of a group one response distribution, so Theorem 1 holds for both. GrRAG has higher Util on Tweet Paraphrase and is within 0.11 points on the other five.

  • Identity. For a fixed query, GrRAG gives each group member the same prompt, so Theorem 1 holds without averaging. This is a guarantee, and the attack results agree (G1).
  • Utility. We ran group averaging (Eq. 5) directly under a uniform conditional prior (G1, Table R1, Appendix K.1).

"Does clustering user-summary embeddings produce groups with small within-group utility differences, as defined in Section 3, and what evidence supports this?"

We measured the utility change that Eq. (7) bounds, as Full Pers. minus group averaging. It is 0.23–0.60 points on five tasks and near zero on Scholarly Title. A GrRAG response also has higher Util for users from the same group than from other groups on three tasks.

Response to Q5


"How does the protected information and threat model differ from DP-ICL and PromptPATE? Could you provide a comparison on compatible tasks, or explain precisely why these approaches cannot serve as suitable baselines?"

DP-ICL [2] and PromptPATE [3] protect each example in a private set of demonstrations. We study who a response was written for. Their guarantee is differential privacy with respect to one example. Ours is that the response preserves within-group prior odds (Eq. 3).

Why they are not suitable baselines:

  • All examples come from one user. In personalization, the prompt holds the intended user's own documents. A bound for one document covers the whole context only at the privacy budget multiplied by the number of documents. A strong bound on the whole context makes the response nearly independent of the user.
  • PromptPATE labels public inputs by a noisy vote and releases a student prompt. This applies to classification, and our tasks are open-ended generation.
  • DP-ICL is complementary. Applied to the group pool, it bounds the influence of each document. The response distribution still depends only on the query and the group, so Eq. (3) holds.

We have added this distinction to Section 2.


We hope the new results and clarifications address your concerns, and we would be grateful if you would consider raising your score.

References

[1] Salemi and Zamani. Improving User Privacy in Personalized Generation: Client-Side Retrieval-Augmented Modification of Server-Side Generated Speculations. SIGIR 2026.

[2] Wu et al. Privacy-Preserving In-Context Learning for Large Language Models. ICLR 2024.

[3] Duan et al. Flocks of Stochastic Parrots: Differentially Private Prompt Learning for Large Language Models. NeurIPS 2023.

Reviewer_4jbFOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper investigates how the granularity of personalization contexts (e.g., group vs. individual) affects both downstream utility and the leakage of intended-user identity. The authors introduce a query-conditioned posterior identification gain to formalize this leakage, theoretically connecting it to information and prediction risk. They prove conditional anonymity and leakage-reduction bounds for an idealized group mechanism. To validate this, they introduce a GrRAG baseline (group personas + pooled retrieval) and conduct extensive experiments on LaMP and LongLaMP tasks across multiple generators. Notably, a significant portion of the work rigorously audits the evaluation instruments themselves, separating reference overlap, judged personal voice, and actual attacker identification increments. The paper concludes that the privacy-utility tradeoff is highly task- and attacker-dependent.

Critical strengths

1.A rigorous separation of identity information from personalization utility. The theory explicitly conditions on the query and distinguishes idealized response mixing from context summarization. The proofs and counterexamples in Appendix I (e.g., showing how the quadratic leakage measure can increase under nonuniform priors) are highly valuable. They prevent common overinterpretations and provide solid conceptual grounding, even without offering a direct privacy guarantee for the GrRAG baseline.

2.A highly commendable audit of the evaluation instruments. The authors do not just report standard metrics; they subject them to rigorous controls (planted own/decoy/generic responses, matched query-only comparisons, independent-attacker replays). This uncovers critical weaknesses that raw AUC or ROUGE scores would easily conceal, providing useful methodological lessons for the broader personalization-privacy community.

Critical weaknesses

1.Some attribution comparisons remain vulnerable to attacker behavior and candidate construction. While the generic-response controls are a great addition, Table 21 shows a -0.30 generic-response increment on Review Writing and -0.14 on Topic Writing. This indicates that even uninformative responses can significantly alter the attacker's decision rule. Subtracting a query-only baseline may not fully eliminate this effect, as these shifts may not cancel out across response styles. Furthermore, while the frozen-input replay in Table 30 strongly supports the Topic Writing result, the conclusions for News and Review remain inconclusive. Thus, broad cross-task claims about privacy reduction are less secure than the main directional comparisons might suggest.

2.The theory offers limited guidance for the implemented utility-privacy choice. There is a disconnect between the theoretical framework and the practical baseline. The GrRAG implementation uses context summarization and retrieval, which does not actually implement the response mixture in Theorem 2. Consequently, neither the within-group utility similarity nor the proximity to the idealized mixture is established. Additionally, "personal-voice Utility" serves as a distinct proxy rather than a calibrated measure of user benefit, and group-arm scores are unavailable for two main generators (Llama and Gemma). The analysis therefore explains possible tradeoffs but does not yet establish when the implemented grouping preserves useful personalization.

Questions for authors

1.Empirical-validity clarification: Under completely fixed candidate identities, evidence, and order, which main cross-task group-versus-individual contrasts remain supported after strictly accounting for generic-response effects? Specifically, can the recorded data distinguish a genuine identification reduction from an attacker response-format effect on Review Writing? Please also clarify the population represented by the selected replay sample and define exactly which broader claims it can support.

2.Contribution-scope clarification: What empirical evidence links the implemented GrRAG groups to the utility-similarity condition required by Theorem 2? If this link is not established, I strongly suggest explicitly delimiting the conclusion to measured personal voice/reference overlap and attacker-specific attribution. Identifying which specific comparisons demonstrate useful group personalization beyond the No-Pers. baseline would help determine the strength of your practical guidance.

4 comments
AuthorsOpenReview ↗

Response to Reviewer 4jbF [1/2]

Thank you for your careful review, and for recognizing the separation of identity information from personalization utility and the audit of the evaluation instruments. New experiments are marked "(new)".

Response to Q1 (and W1): Empirical-validity clarification


"Under completely fixed candidate identities, evidence, and order, which main cross-task group-versus-individual contrasts remain supported after strictly accounting for generic-response effects?"

The group-versus-individual contrast remains supported on four of six tasks, for all three generators: Tweet Paraphrase, Abstract Generation, Review Writing, and Topic Writing. To account for generic-response effects, we compare each personalized response with the No Pers. response to the same query, which has the same format and contains no user information. Candidate identities, summary evidence, and order are identical for GrRAG, Full Pers., and No Pers. on every item. We have added Table R3 to the paper as Table 36 (Appendix K.4).

Table R3. Change in top-1 identification accuracy relative to No Pers., shown as GrRAG / Full Pers. Values are in percentage points. The four other candidates are from the population, and the attacker sees a personal summary of each candidate. Bold marks the larger increase. The difference is significant after Holm correction in every cell.

TaskQwen3.5-9BLlama-3.1-8BGemma-3-12B
Tweet Paraphrase−2.4 / +20.2−2.3 / +7.1−2.3 / +13.6
Abstract Generation+4.1 / +6.4+0.4 / +2.4+1.0 / +2.9
Review Writing+8.9 / +20.2+3.3 / +10.7+5.9 / +15.0
Topic Writing+16.5 / +34.3+5.6 / +26.2+9.2 / +30.1

Results. Full Pers. raises identification above the No Pers. control in all 12 comparisons, and by more than GrRAG in every one. The probability assigned to the intended user and a length-adjusted comparison agree. With candidates from the target's group, the contrast extends to News Headline (Table 12 of the paper and G1 in our general response).

"Specifically, can the recorded data distinguish a genuine identification reduction from an attacker response-format effect on Review Writing?"

Yes. On Review Writing the reduction remains when the response format is matched, for three generators and two attackers. A response-format effect cannot produce this.

  • Same format, three generators. GrRAG, Full Pers., and No Pers. each produce a review for the same query, so a format effect shifts all three alike. Relative to the No Pers. review, Full Pers. raises identification by more than twice as much as GrRAG for every generator (Table R3).
  • Same format, candidates from the target's group, two attackers (new, Appendix K.2). Relative to the No Pers. review, a GrRAG review changes accuracy by at most 1.7 points and a Full Pers. review raises it by 32 to 42 points, for both the Qwen attacker and GPT-6 Luna. The GrRAG increase in Table R3 is information about the group, which Eq. (3) allows, and it disappears when the other candidates are from the same group.
  • The −0.30 generic-response increment in Table 21 is specific to boilerplate text on 50 items. A No Pers. review is also uninformative, and on the full test sets it changes accuracy by at most 0.3 percentage points relative to query-only (Table 2, Leak rows).

"Please also clarify the population represented by the selected replay sample and define exactly which broader claims it can support."

The replay sample represents the Qwen3.5-9B test sets of News Headline, Review Writing, and Topic Writing. We chose these three tasks before scoring the replay, which is what "selected using existing evidence" in Appendix J.4 refers to: News Headline as the short-output task, Review Writing because of Table 21, and Topic Writing for its large recorded difference. Within each task we drew 150 items uniformly at random. The replay (Table 30) compares own-history retrieval with group retrieval, the retrieval-only versions of Full Pers. and GrRAG, on frozen inputs for two attackers.

The broader claim it supports is that the reduction from group retrieval on Topic Writing persists under an independent attacker. With document evidence, group retrieval lowers attribution by 19.0 and 17.3 percentage points for the Qwen and Llama attackers, and both survive Holm correction. On News Headline and Review Writing, the replay agrees with the full test sets.

For the cross-task comparison of GrRAG and Full Pers., we rely on the full recorded data and on a stronger attacker (new, Appendix K.2). The recorded data cover 39,948 items and three generators with fixed candidates. With GPT-6 Luna, Full Pers. exceeds GrRAG on five of six tasks, including News Headline and Review Writing, in all four conditions (G2 in our general response, Table R2). We have clarified the retrieval arms and sampling in Section 4.4.

AuthorsOpenReview ↗

Response to Reviewer 4jbF [2/2]

Response to Q2 (and W2): Contribution-scope clarification


"What empirical evidence links the implemented GrRAG groups to the utility-similarity condition required by Theorem 2?"

We ran the response mixture of Theorem 2 directly and measured the utility change that the utility-similarity condition bounds (new, G1 in our general response, Table R1). The mixture is group averaging (Eq. 5), and the utility change is Util under Full Pers. minus Util under group averaging, the left side of Eq. (7), under a uniform conditional prior. It is 0.23–0.60 points on five tasks and near zero on Scholarly Title. This is the trade-off that Theorem 2 describes: group averaging gives up Util where group members differ. We have added this result to Section 4.3 and Appendix K.1.

On proximity to the mixture, GrRAG matches or exceeds the Util of group averaging. It is higher on Tweet Paraphrase and within 0.11 points on the other five tasks.

Theorem 1 holds for GrRAG without the response mixture. For a fixed query, every group member receives the same prompt, so GrRAG meets its condition by construction, and neither the Qwen attacker nor GPT-6 Luna gains a significant increase from a GrRAG response within a group (G1 and G2).

"If this link is not established, I strongly suggest explicitly delimiting the conclusion to measured personal voice/reference overlap and attacker-specific attribution."

We have adopted this wording. The revised conclusion and contribution list name what was measured: higher reference overlap (METEOR) than No Pers., lower attribution by a fixed LLM attacker than Full Pers., and personal-voice Util that varies by task. Util is specific to the user: in a new check on five tasks, a Full Pers. response scores 0.27–1.31 points higher against the writing of its intended user than against users from other groups.

"Identifying which specific comparisons demonstrate useful group personalization beyond the No-Pers. baseline would help determine the strength of your practical guidance."

METEOR: all nine task–generator comparisons of Table 2. Util: Tweet Paraphrase and News Headline for Qwen3.5-9B, Abstract Generation for all three generators, and Topic Writing for Llama.

  • Tweet Paraphrase is the clearest case. GrRAG raises Util from 1.21 to 2.96, the same as Full Pers. (Table 8), and gives no increase in identification over No Pers. (Table R3).
  • The group-arm scores for Llama and Gemma are now available (new). We scored the six missing cells with the same LLM judge for both arms and added them to Table 2.

Table R4. Util (personal voice) for GrRAG minus No Pers. Llama and Gemma use the new API scores for both arms (Table 2 keeps the submitted local score for No Pers.). Qwen uses the submitted local scores. Bold indicates an improvement after Holm correction.

TaskQwen3.5-9BLlama-3.1-8BGemma-3-12B
Abstract Generation+0.132+0.145+0.106
Review Writing−0.128−0.049−0.183
Topic Writing+0.027+0.062+0.017

Results. Our practical guidance is by task. Group-level personalization raises personal-voice Util on Tweet Paraphrase, News Headline, and Abstract Generation, and it raises reference overlap on Review Writing and Topic Writing.


We hope the new results and clarifications address your concerns, and we would be grateful if you would consider raising your score. We are happy to answer any further questions.

AuthorsOpenReview ↗

Follow-up before the discussion closes

Thank you again for your careful review. As the discussion period ends today, we wanted to gently follow up. We hope that our response addresses your concerns about generic-response effects in the attribution comparisons and about the link between GrRAG and the theory, with new experiments on both in the revised paper. We look forward to any further questions or comments you may have.

The conclusion that GrRAG preserves personalized utility still needs more support. A close average style score alone cannot fully validate the theoretical conditions. The empirical gains also vary by task. The improvement on Topic Writing remains unclear. On Review Writing, the lower style scores and higher identity recognition rates suggest that its practical advantage over non-personalized methods is still unclear.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response to All Reviewers

We thank all reviewers for their careful reading and constructive feedback. Here, we answer two questions shared across the reviews.

G1: How close is the implemented GrRAG to the conditions of Theorems 1 and 2?


(Addressing 4jbF W2 and Q2, 3xQb W2 and Q4, and ygdQ W2 and Q1)

GrRAG satisfies the condition of Theorem 1 exactly: the response difference between members of the same group is zero. For a fixed query, every member of a group receives the same prompt, because the group persona and document pool are fixed and retrieval depends only on the query. Under the same generation settings their response distributions are therefore identical, and by Eq. (3) the response preserves within-group prior odds. We have stated this link in Section 4.1 of the revised paper.

Theorem 1 holds for GrRAG without the response mixture of Theorem 2. Eq. (5) averages the individual response distributions, whereas GrRAG generates from shared group context. Theorem 1 needs only identical response distributions within a group, and both provide them.

The attack results agree: a GrRAG response gives no significant increase in identification within a group, while a Full Pers. response does on five of six tasks. When the Qwen attacker chooses among the intended user and four other candidates from the target's group, GrRAG changes top-1 accuracy by −3.9 to +1.7 percentage points relative to the query-only control, whereas Full Pers. raises it by 9.3–24.9 points on five tasks. These are the full-test-set results in Table 12 of the paper (rows "documents, within cohort"). GrRAG's one-sided 95% upper bound is at most +3.4 points on every task. A stronger attacker gives the same result (Table R2).

For Theorem 2, we ran group averaging (Eq. 5) directly, and GrRAG matches or exceeds its Util. Group averaging is the idealized mechanism that Theorem 2 analyzes: it samples a group member from the conditional prior and uses that member's individual response distribution. For each query, we generated Full Pers. responses from the contexts of two other group members (the user's own response has weight 1/kG1/k_G) and scored their Util for the intended user (296–300 items per task, Qwen3.5-9B, uniform conditional prior). GrRAG is higher on Tweet Paraphrase and within 0.11 points on the other five tasks.

Table R1. Util (1–5 scale) for the intended user (Table 31 in Appendix K of the revised paper). Differences from group averaging are in parentheses. For Full Pers., this is the utility change bounded by Eq. (7). Bold marks values in parentheses that are significantly above zero after Holm correction.

TaskGroup averaging (Eq. 5)GrRAGFull Pers.
News Headline2.362.47 (+0.11)2.96 (+0.60)
Scholarly Title4.574.65 (+0.08)4.51 (−0.07)
Tweet Paraphrase2.563.03 (+0.47)3.11 (+0.56)
Abstract Generation2.522.63 (+0.11)2.88 (+0.36)
Review Writing1.171.16 (−0.01)1.40 (+0.23)
Topic Writing1.281.26 (−0.01)1.54 (+0.26)

G2: How sensitive are the conclusions to the attacker?


(Addressing 4jbF W1, 3xQb Q2, and ygdQ Q2)

The conclusions hold under a stronger attacker. We repeated the attack with GPT-6 Luna on the same prompts as the Qwen attacker: 300 items per task, 5 and 20 candidates from the target's group, with and without reasoning. Luna is the more accurate attacker in every task and condition. Even so, in all four conditions a GrRAG response gives no significant increase in identification on any task, while a Full Pers. response does on five of six. The same holds with the No Pers. response as the control, which accounts for generic-response effects: GrRAG is never significantly above No Pers., and Full Pers. is above it on five of six tasks. Table R2 shows the two conditions with reasoning.

Table R2. GPT-6 Luna attacker with reasoning: top-1 identification accuracy (%) among the intended user and the other candidates from the target's group (Table 32 in Appendix K). Changes relative to query-only are in parentheses, in percentage points. Responses are from Qwen3.5-9B, and the attacker sees a personal summary of each candidate. Bold as in Table R1.

TaskCandidatesQuery-onlyGrRAGFull Pers.
News Headline549.342.7 (−6.7)60.0 (+10.7)
2021.320.3 (−1.0)29.0 (+7.7)
Scholarly Title570.067.0 (−3.0)70.7 (+0.7)
2047.047.7 (+0.7)50.0 (+3.0)
Tweet Paraphrase547.334.7 (−12.7)69.3 (+22.0)
2026.320.3 (−6.0)51.7 (+25.3)
Abstract Generation560.758.7 (−2.0)72.0 (+11.3)
2043.344.7 (+1.3)56.0 (+12.7)
Review Writing538.032.0 (−6.0)72.3 (+34.3)
2017.013.0 (−4.0)57.0 (+40.0)
Topic Writing551.341.0 (−10.3)93.3 (+42.0)
2026.321.3 (−5.0)82.0 (+55.7)

Full Pers. also exceeds GrRAG after Holm correction in all four conditions on every task except Scholarly Title.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.