acceptodds
Under review as a conference paper at ICLR 2027

Leak Less, Solve More: Reinforced Anonymization for Privacy-Preserving Local-Cloud LLM Collaboration

Abstract

Privacy concerns have driven users to deploy trusted small language models (SLMs) locally. However, such SLMs may struggle with complex queries and therefore consult a cloud LLM before composing the final answer locally, forming a local–cloud–local collaboration. Although the forwarded query can be anonymized, a cloud LLM may still infer private attributes such as age, location, or occupation from semantic cues. Yet aggressively removing such cues can also eliminate information essential for solving the task. This creates a fundamental challenge: anonymization needs to suppress privacy leakage while preserving task-critical information. Existing anonymization methods either fail the inference attack or optimize anonymized text in isolation without verifying whether the downstream task remains solvable. Moreover, no existing benchmark jointly evaluates privacy leakage and task utility on the same query. To address these gaps, we first construct a benchmark of 2200 privacy-bearing queries drawn from four QA datasets. Each query is paired with four-option privacy probes that measure an LLM attacker's attribute inference accuracy (AIA), as well as a verifiable answer for measuring the end-to-end utility of the collaborative pipeline. We then introduce ReAnon, a reinforced anonymization framework that trains a local SLM to rewrite queries into task-equivalent representations against inference attacks while preserving downstream utility. The SLM is trained with supervised fine-tuning (SFT) and multi-objective GRPO using joint utility and entropy-based privacy reward. With Qwen3.5-4B as the local SLM and Qwen3.5-27B as the cloud LLM, the resulting SLM reduces average AIA by 14.9 points (from 79.76 to 64.82) while improving utility by 1.7 points (from 61.76 to 63.50) over its zero-shot counterpart, achieving the lowest AIA, the highest utility, and the best average rank among all evaluated protection methods.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Privacy

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UUiUOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper studies developing small language models in a local-cloud-local collaboration setting where local small models interact with more capable models in cloud to handle complex queries in a privacy-aware manner. For evaluation, the authors introduce a benchmark that consists of QA samples for assessing privacy leakage and task success for each query. The benchmark with 2,200 privacy-bearing queries is used to measure privacy leakage with attribute inference accuracy with an LLM attacker. For training, a two-step approach -- SFT initialization to learn basic anonymization followed by multi-objective GRPO -- designed to more explicitly consider privacy-utility trade-off is used. Experiments show favorable results against inference (adversarial anonymization) and training (AgentStealth) baselines.

Critical strengths
  1. Benchmark for assessing privacy-utility trade-offs in local-cloud-local interaction settings across multiple QA domains. The authors carefully curate QA data across legal, medical, and open-ended domains used for their main evaluations. Compared to prior benchmarks, it is comparable in size and supports attribute inference accuracy and local-cloud collaboration performance.

  2. The proposed training method is fairly straightforward and conceptually sound. The two-stage method where the first SFT step is used to learn basic anonymization followed by multi-objective RL to balance between privacy and utility is conceptually simple and straightforward to implement.

Critical weaknesses
  1. Technical contribution in training and its impact on performance. Both the local-cloud-local setting and training that involves distillation and RL have been studied in prior work. It appears that the multi-objective GRPO that balances between privacy and utility is one of the main method contributions, but the methodological "delta" compared to prior work and its impact on performance should be more clearly discussed and evaluated.

  2. While generalization to an unseen cloud LLM is evaluated with GPT-5-mini, it is unclear how the findings hold for much stronger unseen LLM attackers. The mini variant is considered by some to be fairly small (smaller than 27B), so it's possible that the local models trained against the 27B model might be sufficiently good for the mini model.

Questions for authors

Q. Have the authors evaluated the local models against stronger unseen LLMs?

2 comments
AuthorsOpenReview ↗

Author Response (W1, W2/Q)

We thank the reviewer for the positive assessment and the two precise questions.

W1 — What is the technical delta of the training method over prior work, and how much does each component contribute?

Table R24. The three training components and their measured effect

ComponentWhat it replacesEvidence
Reward from the end-to-end collaboration (task success of the final locally composed answer after the cloud response)text-retention / readability rewards (AgentStealth)Indirect (no isolated ablation of this component): no significant utility loss against any baseline, and significantly higher utility than AgentStealth (+3.1, p=0.002); Fig. 5(a) and Table R11
Graded privacy reward 1 − p* + attacker entropy (1 − p* is the Rao–Blackwellized binary reward, Prop. 2; the entropy term is affine in normalized MI under Assumptions 1–2, Prop. 3)binary hit reward, which gives no privacy signal on a probe with probability ≈0.43 at κ=0.9 (A.8)Fig. 5(a) (+SFT+MO-GRPO vs +entropy privacy reward) and Fig. 3(b) (mean attacker posterior on the true value 0.81 → 0.74 in training rollouts)
Per-objective normalization (MO-GRPO, Ichihara et al., 2025), applied to a utility that mixes judge and EM scores and to a probe-averaged privacy reward (Prop. 1, Theorem 1)a scalarized reward whose balance depends on the relative scales of judge and EM scoresFig. 5(a) (+SFT+GRPO vs +SFT+MO-GRPO); the λ sweep gives a controllable monotone trade-off (Table R25)

Table R25. λ sweep of the privacy weight

λAvg AIAUtility
0.966.663.8
1.0 (paper)64.863.5
1.161.461.7

Measured impact of the stages. In paper Fig. 5(a) (4B), SFT gives the largest privacy gain at a utility cost, GRPO / MO-GRPO recover utility while giving part of the privacy back, and the full model (SFT + MO-GRPO with the entropy privacy reward) is significantly more private than the untrained AA loop (−6.7 AIA, p≤0.002) at comparable utility (+1.0, n.s.) with the same budget. The prompt-aligned baseline (72.3) and the rounds sweep (one trained round, 65.8, beats five untrained rounds, 71.5; reply to Cbtz W4) locate the gain in training rather than prompting; within training, the privacy gain comes from SFT, and RL trades part of it for utility. The λ sweep shows that the MO-GRPO balance is controllable. We will report the Fig. 5(a) values as a table in the main text.

Table R26. Calibration of the quantity used by the privacy reward (E[p_true]) vs attack accuracy (same log-probability run as Table R14; micro-averaged over the 2,003 probes; |Δ| computed before rounding)

RowAccuracyE[p_true]|Δ|
Base prompt81.281.00.2
AA72.771.21.5
AgentStealth64.763.80.9
Ours63.762.90.8
Oracle redaction48.046.81.3

W2 / Q — Does the privacy advantage hold against attackers much stronger than GPT-5-mini?

We re-attacked the stored rewrites of all paper Table 3 methods with GPT-5.4, with a defense-aware GPT-5.4 (six original / rewrite / ground-truth examples), and, for Ours, with a white-box attacker holding the rewriter.

Table R27. Stronger and defense-aware attackers

RowQwen3.5-27B (paper)Qwen3.5-27B defense-aware (new)GPT-5.4 (new)GPT-5.4 defense-aware (new)
Ours64.863.869.670.3
AgentStealth65.465.072.171.3
TRACE-RPS69.369.574.074.2
AA71.570.976.275.9
Base prompt79.881.084.083.6

GPT-5.4 is stronger by 4–7 pts (its gain is largest on oracle-redacted text, +8.4, negligible on raw text, +0.1, and only +1.1 prior-only, i.e. it extracts more from the residual context). The ranking is unchanged and Ours keeps the lowest AIA among the five Table 3 methods (−14.4 vs base, −6.6 vs AA, −4.4 vs TRACE-RPS, −2.4 vs AgentStealth; the last gap was not significance-tested, so we claim no privacy advantage over AgentStealth). Defense awareness adds no meaningful leakage for any method (27B within ±1.3, GPT-5.4 within ±0.8), and the white-box attacker reaches 66.0 (reply to mJT6 W2). The ordering relative to base, AA and TRACE-RPS is therefore preserved under a substantially stronger, defense-aware attacker.

AuthorsOpenReview ↗

Gentle Follow-Up

Dear Reviewer UUiU,

Thank you again for your detailed review. As the discussion period ends on November 18, we wanted to check whether our response, with additional baselines, ablation experiments and explanation address your concerns. We would be grateful for any feedback, and happy to clarify further. If our response addresses your concerns, we would appreciate it if you would consider updating your assessment.

Best regards,

The Authors

Reviewer_mJT6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper looks at local–cloud–local LLM collaboration. A small local model anonymizes the user query, sends it to a cloud LLM for advice, then writes the final answer on-device with the original query. The issue is that the cloud can still infer private attributes (age, location, job, etc.) from semantic cues, but removing too much kills task utility.

Critical strengths

The problem is important and timely. As local data sanitization with LLMs becomes more common, the need for anonymization that also preserves task utility keeps growing.

The motivation is clear and well grounded. Building on Staab et al., the paper argues convincingly that anonymization needs more than redaction, since LLMs can infer private attributes from semantic cues even after explicit identifiers are removed.

The privacy–utility tension is framed well.

The local–cloud–local setting is realistic and practically relevant. The local model keeps the original query and composes the final answer, which reflects how on-device assistants would actually delegate to cloud models.

Critical weaknesses
  • *The statement that "existing defenses either do not consider inference adversary or do not evaluate downstream task success" overstates things. There are many existing benchmarks, and utility can be measured easily through perplexity with teacher forcing on the in-context document, LLM judges, or downstream task accuracy. Privacy can be measured by mounting attacks. DP-Fusion (arXiv 2507.04531), for example, does both: MCQ-based downstream utility (A.23) and MIA-style attacks for privacy.

The paper's own Table 1 also marks several benchmarks as measuring both task utility and attribute inference. I think the real novelty is narrower, scoring both axes on the same query inside a full local–cloud–local pipeline, and the claims should be scoped to that.

  • AIA is a nice way to look at privacy leakage, but why not probabilistic attacks? The attacker observes the output and models the probability of observing it given a particular PII value in the input. The four-option probes already give a candidate set, so a LOSS or Min-K% style attack using the local rewriter would be easy to run.

  • Removing PII lowers utility, but other metrics capture text quality. Existing methods, especially DP ones, can produce garbage text, so general utility (e.g., perplexity or a fluency judge on the rewrite) should also be reported. This matters more here because the local model writes the final answer with the original query, so task utility can stay reasonable even when the transmitted rewrite is poor.

  • The probes are generated by LLM annotators (as per my understanding of the paper). These can be biased, and the only audit described is that the task question does not reveal the answer. Human validation of a sample, with inter-annotator agreement, should be reported. The oracle-redaction AIA of 45.83 (vs. 25% chance) suggests that priors or weak distractors may be inflating AIA. No confidence intervals, seeds, or significance tests are reported, even though two datasets have only 100 test queries and the gap over AgentStealth is 0.6 AIA points.

  • What stops the attacker LLM from already knowing the contexts? All four datasets are public, so the attacker may have trained on them and could answer probes from memory rather than inference. This should be checked, e.g., with a memorization test or a held-out subset.

  • DP for local LLMs has now worked out, e.g., DP-Fusion (arXiv 2507.04531), which runs locally with formal guarantees. The authors should benchmark against it properly, not just against empirical methods. The annotated privacy spans could serve directly as the tagged groups.

Questions for authors
  1. Why are the samples so small (only 100 test queries for GerlayQA and HealthCareMagic), and are the gaps between methods statistically significant?
  2. What differentiates this work from existing methods and benchmarks that already evaluate both privacy and utility?
  3. Why not compare against DP methods (e.g., DP-Fusion), not just empirical baselines?
4 comments
AuthorsOpenReview ↗

Author Response (W1/Q2, W2)

We thank the reviewer for the expert review and the pointers to DP-Fusion and probabilistic attacks; we ran both, plus the requested text-quality and memorization checks.

W1 / Q2 — What is genuinely new compared with prior work that already evaluates privacy and utility?

We accept the point and will narrow the claim. Prior work measures privacy and utility on different data or sub-tasks, or measures utility as retention / readability / perplexity rather than end-to-end task success. Our contribution is scoring both axes on the same query inside the full local–cloud–local pipeline and training against that signal. We will revise the introduction and abstract, add DP-Fusion to related work, and split paper Table 1's "Task utility" column into "text retention / readability" and "downstream task success". In our pipeline, DP-Fusion reaches the lowest AIA (40.0) but 7.7 pts lower utility and low retention (2.71 / 5; Table R22), which illustrates the privacy–utility tension our benchmark is designed to expose.

W2 — Would a probabilistic (LOSS / Min-K%) attack reveal more leakage than black-box AIA?

These attacks require the rewriter's weights, which the cloud does not observe; AIA models an attacker that sees only x″. Since the base weights are public and a released adapter would be too, we treat white-box access as a stronger but plausible setting: the attacker holds Ours' 4B rewriter and prompt, substitutes each candidate value into the context, and picks the candidate maximizing the teacher-forced log p(x″ | prompt, x_candidate) (all probed spans are literal substrings; the stored rewrite requests were reproduced byte-identically).

Table R15. White-box likelihood attack vs black-box attack on Ours' rewrites

AttackerGerHCMMedQACosmosAvg AIA
Qwen3.5-27B, black-box (paper)59.361.959.378.964.8
Local rewriter, white-box likelihood (new)61.168.557.077.566.0
StatisticValue
Probes where both attackers give the same answer55.9%
Probes with white-box posterior > 0.939.6%
Accuracy on those probes97.1%

The likelihood attack reaches an average AIA of 66.0 (black-box: 64.8) and is highly confident on 39.6% of probes (97.1% accurate there), so an adversary with the rewriter weights obtains a strong signal for some attributes; we will list this as a limitation of the threat model. The two attackers agree on only 55.9% of probes, so a combined attack could do better. This attack was run on Ours only; the revision will add the baselines.

AuthorsOpenReview ↗

Author Response (W3, W4/Q1)

W3 — Is the transmitted rewrite itself usable text, or does the local model rescue poor rewrites?

We measured the quality of x″ for all paper Table 3 methods, and the utility when the cloud answers from x″ alone (no local composition).

Table R16. Quality of the transmitted text x″ (1,000 test queries)

MethodFluency (0–5)Info retention (0–5)PPL, corpusPPL, mean per text
Raw text4.484.959.616.8
Base prompt4.514.769.816.4
AA4.564.6511.417.0
AgentStealth4.574.4912.718.2
TRACE-RPS4.594.6210.916.8
Ours4.884.5711.014.1
Oracle redaction3.894.0615.632.0

The fluency judge (GPT-5.4) sees only x″; the retention judge sees the original, the question and x″ and does not penalize removed private attributes; PPL is from an independent Qwen3.5-9B.

Table R17. Utility when the cloud answers from x″ alone vs the full pipeline (4B; Δ computed before rounding; oracle-redaction utility is new, the paper reports none)

MethodCloud-only utilityFull-pipeline utilityΔ (cloud-only − full)Cloud-only judge Ger / HCM (1–5)Cloud-only EM MedQA / Cosmos
Base prompt62.261.8+0.42.32 / 2.4576.0 / 77.3
AA63.062.5+0.62.37 / 2.4078.8 / 78.0
AgentStealth60.560.4+0.12.38 / 2.1076.0 / 76.3
TRACE-RPS61.962.1−0.22.36 / 2.2877.8 / 77.0
Ours63.263.5−0.32.31 / 2.2083.0 / 79.8
Oracle redaction55.356.3−1.02.23 / 1.9264.5 / 73.8
DP-Fusion β=0.0152.755.8−3.22.01 / 1.9064.5 / 68.0
DP-Fusion β=0.152.555.5−3.02.10 / 1.8763.0 / 67.8

Ours' rewrites are the most fluent, with retention comparable to the baselines and perplexity on par with the raw text; they average 85 tokens (64% of the original). For Ours, cloud-only utility is within 0.3 pts of the full pipeline, so 4B utility does not hinge on the local model rescuing a poor rewrite. With 4B, local composition changes utility only slightly for the protection methods (−0.6 to +0.3 pts); the gain reported in paper Sec. 4.3 was measured with 9B, and the revision will state that it depends on the local model.

W4 / Q1 — Are the LLM-generated probes valid, is AIA (oracle 45.83 vs 25%) inflated by priors, and are the gaps significant with 100 queries?

Probe validation. Two human annotators checked a stratified sample of 40 probes (10 per dataset).

Table R18. Probe validation, 40 probes

CheckAnnotator 1 / 2 passBoth pass
(a) correct option is supported by the context39 / 3939 / 40
(b) not answerable from the question alone36 / 3836 / 40
(c) distractors same type and plausible38 / 3737 / 40

34 / 40 probes pass all checks for both annotators; raw agreement is 95–100%. Failing probes will be fixed or removed in the release. (b) as phrased to annotators also counts probes whose answer is guessable from the options alone (consistent with the 39.1% prior-only accuracy), which the audit does not target.

Oracle AIA of 45.8% and an option-free attack. The options are not equally plausible a priori, so the chance level exceeds 25% (prior-only: 39.1%), and oracle redaction keeps the rest of the context. To remove priors and distractors entirely, we drop the options: the attacker answers freely and GPT-5-mini judges the match (over-general answers and "unknown" count as misses).

Table R19. Open-ended (no options) vs multiple-choice attack on Ours' rewrites

Attack formatGerHCMMedQACosmosAvg AIA
MCQ (paper)59.361.959.378.964.8
Open-ended + LLM match judge (new)42.635.932.557.742.2

Without options, the attacker recovers 42.2% of attributes, 22.6 pts below MCQ; MCQ is thus the conservative choice, and part of its level reflects the option prior. The revision will report this attack for all Table 3 rows and the raw / redacted bounds.

CIs, seeds, significance, 0.6-pt gap. Full tables are in our reply to Reviewer Cbtz (Tables R8, R9); key rows:

Table R20. Paired differences and seed replication

ComparisonAIA Δ [95% CI], pUtility Δ [95% CI], p
4B Ours vs AgentStealth−0.58 [−3.23, +2.02], p=0.69 (n.s.)+3.09 [+1.05, +5.02], p=0.002
4B Ours vs base / AA / TRACE-RPS−14.9 / −6.7 / −4.5, all p≤0.002+1.7 / +1.0 / +1.4, n.s.
4B Ours, second RL seed (paper seed in parentheses)65.0 (64.8)63.3 (63.5)

The revision will state that Ours matches AgentStealth on privacy and exceeds it on utility, with CIs in every table. The 100-query size of the open-ended sets follows from the triple filter and judge cost; the CIs show which gaps it can resolve.

AuthorsOpenReview ↗

Author Response (W5, W6/Q3, Ethics review )

W5 — Does the attacker infer attributes, or recall them from memorized public data?

Table R21. Memorization checks

CheckResultWhat it rules out
Prior-only attacker (question + options, no text)39.1 (GPT-5.4: 40.2)recalling contexts from the probe alone
Oracle redaction (the probed spans removed)45.8 (raw text: 99.8)retrieving the span from memory of the public text
Gain of GPT-5.4 over 27B: raw text / oracle+0.1 / +8.4— (reported for completeness; not a memorization test)
Attack on GPT-5.4 paraphrases of the raw contexts (same attributes, 3.9% shared 8-grams) (new)99.65 vs 99.84 on the originals; 99.9% identical predictionsdependence of AIA on the verbatim surface form (not a memorization test on its own, since the attribute remains in the text)
Verbatim-continuation test: 27B continues the first half of each raw context (new)longest verbatim run 1.7 words (chance 1.2); 0 / 1,000 contexts with ≥20 verbatim words; ROUGE-L 0.13 vs chance 0.08memorized text (a memorized context would be reproduced)

These checks indicate that verbatim memorization is unlikely to drive AIA for the 27B attacker: with the probed spans removed, AIA is only 6.7 pts above the prior-only level, and the attacker cannot reproduce the benchmark texts. The continuation test was run only with 27B, so we draw no such conclusion for GPT-5.4.

W6 / Q3 — How does ReAnon compare with a formal DP method (DP-Fusion) in the same pipeline?

We implemented DP-Fusion with the same local Qwen3.5-4B, using our annotated spans as the private groups, inside the identical pipeline at two budgets.

Table R22. DP-Fusion inside the same pipeline

MethodNeeds gold spansAIAUtilityJudge Ger (1–5)Judge HCM (1–5)EM MedQAEM CosmosFluency (0–5)Retention (0–5)
DP-Fusion β=0.01 (ε≈11)yes40.055.82.042.2064.374.33.962.71
DP-Fusion β=0.1 (ε≈46)yes42.155.52.122.1963.572.33.932.75
Oracle redactionyes45.856.32.112.1465.375.03.894.06
AgentStealthno65.460.42.422.1775.874.04.574.49
Oursno64.863.52.332.2282.880.34.884.57

DP-Fusion sits at the privacy end, but its utility is 7.7 pts below Ours (largest gaps on MedQA and CosmosQA; on par on HealthCareMagic) and its retention is the lowest in our study (2.71 / 5). We will add DP-Fusion to paper Table 3 (and Table 4 once the 9B run completes), note that it uses gold span annotations whereas Ours needs none, and state in the limitations that Ours offers no formal guarantee; DP bounds span distinguishability while AIA measures semantic inference, so the two are complementary. The ε values follow DP-Fusion's accounting and are loose at these operating points, so this is an empirical comparison rather than one of guarantees.

Ethics review — Is any real PII exposed, and do the dataset licenses permit this use?

Table R23. Data sources and licenses

DatasetSourceLicenseUsage restrictionRelease form
GerlayQABüttner & Habernal, EACL 2024Apache-2.0 (code) + non-commercial research-use notice (data)research onlyIDs + annotations + rebuild script
HealthCareMagic-100kChatDoctor, Li et al. 2023no explicit data licenseacademic research onlyIDs + annotations + rebuild script
MedQAJin et al. 2021MIT—texts + annotations
CosmosQAHuang et al. 2019 (allenai/cosmos_qa)CC BY 4.0attributiontexts + annotations

We will extend the Ethics Statement to state that this work exposes no personal information beyond what the public source datasets already contain, and that these texts were processed by commercial APIs (GPT-5.4, GPT-5-mini) for annotation and evaluation. We collected no new user data and introduced no identifiers; the probes target attributes already present in the public texts; the one name-like span will be masked; attack outputs were used only for aggregate metrics. For GerlayQA and HealthCareMagic we will release only IDs, annotations and a rebuild script, and will align the supplementary material and the examples in paper Table 6 accordingly.

AuthorsOpenReview ↗

Gentle Follow-Up

Dear Reviewer mJT6,

Thank you again for your detailed review. As the discussion period ends on November 18, we wanted to check whether our response, with additional baselines, ablation experiments and explanation address your concerns. We would be grateful for any feedback, and happy to clarify further. If our response addresses your concerns, we would appreciate it if you would consider updating your assessment.

Best regards, The Authors

Reviewer_CbtzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates "task-aware anonymization" within a "local–cloud–local" question-answering (QA) workflow. We construct a benchmark dataset comprising 2,200 queries—derived from four existing QA datasets—in which task answers are paired with four-choice probing questions designed to infer sensitive attributes. The proposed REANON framework employs "attack-guided supervised trajectories" and a "multi-objective Group Relative Policy Optimization (GRPO)" algorithm to train a local rewriter. This training process balances the utility of the final task against a "privacy reward," which is calculated based on an attacker’s prediction accuracy and the entropy of their predictions regarding sensitive attributes. During deployment, the local model generates rewritten queries through a cycle of "self-attacks" and "optimization iterations" before sending them to the cloud; it then synthesizes the cloud’s response with the original context to formulate the final answer. Experimental results demonstrate that this approach reduces the inference accuracy for sensitive attributes while simultaneously enhancing overall task utility—a benefit observed even with a 4B-parameter model. Furthermore, the study evaluates the method's transferability to unseen cloud-based models.

Critical strengths
  1. Sensitive information can sometimes serve as a valid basis for completing a task; therefore, evaluating utility solely based on text retention or the extent of surface-level de-identification is insufficient. Assessing the final, collaboratively generated answer—rather than evaluating the rewritten output in isolation—represents a more meaningful improvement.

  2. This benchmark pairs each query with a "ground truth" answer and a set of attribute probes, with a clear division between a training set (1,200 entries) and a test set (1,000 entries). It spans multiple domains—including legal, medical, and narrative contexts—and encompasses both open-ended questions and multiple-choice tasks.

  3. Compared to a binary outcome based solely on whether an "attacker" succeeded, a "graded privacy reward" provides richer information. By normalizing both the utility reward and the privacy reward, the issue of inconsistent scales between the two is effectively addressed. Furthermore, the paper appropriately highlights the relationship between the MO-GRPO method and prior research. The introduction of entropy regularization is also a sound approach, designed to prevent the model from generating overly confident "decoy" information when faced with a fixed attacker.

Critical weaknesses
  1. The results of the Anonymized Utility Assessment (AIA) depend on the phrasing of the probing questions, the nature of the distractors, the prior probabilities of the attributes, and the attacker's ability to exploit a given candidate set. A more accurate interpretation of the evaluation scores could be achieved by incorporating "context-free/prior-only" attackers, performing option-permutation tests, accounting for open-ended inference, or introducing more sophisticated adaptive attackers (including those with retrieval-augmented capabilities). Furthermore, Appendix A.4 treats unparseable attacker outputs as errors; papers should report the frequency of such occurrences and how they are handled, to ensure that "spurious" gains—arising from the corruption of the output format—are excluded. While testing models on previously unseen datasets is valuable, it is insufficient to demonstrate robustness against "defense-aware, adaptive" attacks.

  2. For the 4B-parameter model, although overall utility on multiple-choice tasks improved, utility on open-ended tasks—as evaluated by a Large Language Model (LLM)—actually declined from 2.358 to 2.275. For the 9B-parameter model, the average AIA score shifted from 71.06% to 54.76%, yet the overall utility dropped from 62.93 to 62.02. This indicates that simultaneous improvements in privacy protection and utility are not universally guaranteed. The "overall utility" metric in this study is a weighted combination of "LLM-based scores (divided by 5)" and "exact-match percentages." While this calculation method is transparent, it does not constitute a calibrated, general-purpose utility metric. Consequently, when interpreting the results, one should prioritize domain-specific metrics and "privacy-utility" trade-off curves.

  3. The tables lack confidence intervals, and the study does not provide sufficiently detailed results from repeated training experiments. With the 4B model configuration, REANON outperforms AgentStealth by only 0.58 percentage points on the AIA metric; therefore, a paired uncertainty analysis is essential before claiming a robust advantage in privacy protection. In the open-ended task domain, each category contains only 100 test queries, and utility evaluation relies on a Large Language Model (LLM). Incorporating human spot-checks to assess the consistency of the LLM’s scoring would significantly enhance the credibility of the results. Furthermore, more detailed operational specifics regarding the difficulty-based filtering mechanism, as well as the probe generation and auditing processes, are required to determine whether any selection bias or annotation errors might be present. While medical examination scenarios serve as effective test cases for sensitive attributes, they should not be categorically equated to data leaks involving real, identifiable users.

  4. The "base-prompt" baseline involves only a single rewrite without relevance-based decomposition, whereas REANON integrates a trained model and task-specific instructions, undergoing five rounds of "local self-attack/optimization" cycles—as described in Appendix A.4. Although the ablation studies during the training phase touch upon the issue of attribution—and stronger anonymization baselines also employ similar iterative mechanisms—REANON’s 14.9-percentage-point improvement over the "base-prompt" baseline on core metrics should not be attributed solely to reinforcement learning. Providing a comparison where the prompt structure and optimization budget are strictly aligned—complemented by data on local latency, token generation volume, and memory overhead—would provide stronger support for the claim that the method is "suitable for on-device deployment."

Questions for authors
  1. In an AIA (Agent-Invisibile Attack) scenario, can the attacker—while obtaining the rewritten content—also access the task question and all available task options? If not, how do the results change when the input is fully "visible to the cloud"? Furthermore, does the relative ranking of the various methods remain consistent?

  2. How is the concept of "privacy-reward mutual information (MI)" defined, given the observed probe values ​​and candidate values? Is the attacker’s calibration evaluated post-training? How should the theoretical formulation be interpreted when the fixed Large Language Model (LLM) is not Bayes-optimal?

  3. Are data provided for probe accuracy, output parsing failure rates, and sensitivity to the ordering of options under "prior-only" or "no-context" conditions? How can one prevent a strategy from lowering the measured AIA value by interfering with the attack interface, rather than by eliminating evidence of the underlying attribute?

  4. When an untrained rewriter is compared against a trained rewriter—using the same correlation-based instructions and optimization budget—what is the magnitude of the performance gain? What are the token consumption, latency, and memory usage metrics across the five-round deployment cycle?

  5. Could the authors provide pairwise confidence intervals for the primary comparative results (specifically regarding the 0.58-point AIA difference compared to AgentStealth)? Additionally, could they specify the random seeds used for repeated training, and provide details on the human verification process for the open-ended evaluation/probe labeling?

5 comments
AuthorsOpenReview ↗

Author Response (W1/Q3)

We thank the reviewer for the detailed review. "Ours" denotes ReAnon; unless stated otherwise, all new numbers use the same 2,003 probes / 1,000 queries as the submission.

W1 / Q3 — Does AIA reflect real leakage, rather than priors, option order, parse failures, the MCQ format or a non-adaptive attacker?

Prior-only attacker. The attacker sees only the probe question and options (no context), which gives the empirical chance level.

Table R1. Prior-only attacker vs the 4B rows of paper Table 3

RowAIA
Prior-only attacker, 27B (new)39.1
Prior-only attacker, GPT-5.4 (new)40.2
Oracle redaction (span-redaction reference)45.8
Base prompt79.8
AA71.5
TRACE-RPS69.3
AgentStealth65.4
Ours64.8
Raw text (upper bound)99.8

The chance level is ≈39–40%, not 25%, because distractors are not equally plausible. Net leakage above it (AIA − 39.1) is 40.7 (base), 32.4 (AA), 30.2 (TRACE-RPS), 26.3 (AgentStealth), 25.7 (Ours) and 6.7 (oracle). We will add this view to paper Table 3.

Option permutation. We re-ran the 27B attack on Ours' rewrites with randomly re-ordered options.

Table R2. Option-order sensitivity

SettingAIA
Original order64.82
Permuted options65.07

As we can see in this table, AIA is insensitive to option position in aggregate (+0.25 pts).

Parse failures. The attacker decodes a single letter under a JSON-schema constraint, so its output is always parseable.

Table R3. Attacker-output parse failures and non-OK probes

RowParse errorsNon-OK probes (upstream anonymizer failure / network)
Raw text0 / 20030 / 2003
Base prompt0 / 200318 / 2003
AA0 / 20032 / 2003
TRACE-RPS0 / 200316 / 2003
AgentStealth0 / 200311 / 2003
Ours0 / 20030 / 2003
Oracle redaction0 / 20030 / 2003

An unparseable attacker answer counts as a miss, so the attacker's decoding is constrained; As shown above, all rows have 0 parse errors. The few non-OK probes (≤18 / 2,003, under 1%) are upstream anonymizer or network failures, never counted as successful protection, and too few to change the ordering of the methods.

Open-ended inference. The attacker answers without options; GPT-5-mini judges the match (over-general answers and "unknown" count as misses).

Table R4. Open-ended vs multiple-choice attack on Ours' rewrites (Qwen3.5-27B attacker)

Attack formatGerHCMMedQACosmosAvg AIA
MCQ (paper)59.361.959.378.964.8
Open-ended + LLM match judge (new)42.635.932.557.742.2

Most MCQ hits that are open-ended misses are correct only at a coarser granularity, so MCQ is the more conservative (attacker-favorable) metric.

Defense-aware attacker. The attacker is told the text was anonymized (generalization, deletion, possible substitutions) and given six in-context (original, rewrite, true attribute) examples of that defense from other test samples.

Table R5. Defense-aware attacker vs standard attacker (same probes; Δ computed before rounding)

Defense27B standard27B defense-awareΔGPT-5.4 standardGPT-5.4 defense-awareΔ
Base prompt79.881.0+1.384.083.6−0.4
AA71.570.9−0.576.275.9−0.4
TRACE-RPS69.369.5+0.274.074.2+0.2
AgentStealth65.465.0−0.472.171.3−0.8
Ours64.863.8−1.069.670.3+0.6

All changes are within ±1.3 pts for both attackers and all methods (Ours per dataset, 27B: 56.8 / 61.3 / 58.6 / 78.6 vs 59.3 / 61.9 / 59.3 / 78.9).

Retrieval-augmented attackers. Our threat model is an honest-but-curious cloud that sees only the transmitted text; external retrieval is out of scope. The probes target personal attributes, not direct identifiers, and the only name-like span in the sources will be masked in the release. We will state this in the Limitations.

AuthorsOpenReview ↗

Author Response (W2, W3/Q5)

W2 — Privacy and utility do not always improve together: what does the trade-off look like?

Utility per task family. We agree, and the revision will say so. With 4B, the utility gain comes from the exact-match tasks (MedQA 82.8, CosmosQA 80.3), while the open-ended judge score is slightly lower than with the base prompt (2.275 vs 2.358). With 9B, AIA drops by 16.3 pts while utility changes by −0.9 (n.s., p=0.44). For 4B, the utility gains over base / AA / TRACE-RPS (+1.7 / +1.0 / +1.4) are also not significant (Table R20). The revision will report per-domain metrics as primary and the composite as secondary, and the abstract and Sec. 4.2 will describe the results as a significant AIA reduction without significant utility loss, with a significant utility gain only over AgentStealth (+3.1).

Trade-off curve. We re-trained the MO-GRPO stage with λ ∈ {0.9, 1.0, 1.1}, all else identical.

Table R7. Privacy–utility curve from the λ sweep (Ours, 4B)

λGerHCMMedQACosmosAvg AIAUtility
0.958.665.259.883.066.663.8
1.0 (paper)59.361.959.378.964.863.5
1.155.655.358.276.661.461.7

The sweep gives a monotone trade-off; the revision will report ReAnon as this curve rather than a single point.

W3 / Q5 — Is the 0.58-pt gap robust (CIs, seeds), and are the LLM judge and benchmark construction reliable?

Paired CIs. Per-probe (AIA) and per-query (utility) bootstrap, B = 10,000, with shared resample indices.

Table R8. 95% CIs for the 4B methods (incl. DP-Fusion)

RowAIA [95% CI]Utility [95% CI]
Base prompt79.76 [77.30, 82.11]61.76 [59.77, 63.65]
AA71.48 [68.75, 74.14]62.48 [60.54, 64.40]
TRACE-RPS69.34 [66.48, 72.06]62.07 [60.00, 64.08]
AgentStealth65.40 [62.56, 68.21]60.41 [58.49, 62.41]
Ours64.82 [61.96, 67.64]63.50 [61.68, 65.33]
DP-Fusion β=0.0140.01 [37.19, 42.80]55.83 [53.72, 57.77]

As the reviewer anticipated, the 0.58-pt AIA gap to AgentStealth is not significant (−0.58 [−3.23, +2.02], p=0.69), while the utility gain is (+3.09 [+1.05, +5.02], p=0.002; EM alone +6.6, p<0.001). The AIA gaps to base, AA and TRACE-RPS are significant (p≤0.002). The revision will say "privacy on par with AgentStealth, utility significantly higher", and all tables will carry CIs.

Repeated training. A second RL seed trained from the same SFT checkpoint, all else identical.

Table R9. Seed replication of the RL stage (Ours, 4B)

SeedGerHCMMedQACosmosAvg AIAUtility
1 (paper)59.361.959.378.964.863.5
258.658.061.482.165.063.3

Seed 2 reproduces the paper row (AIA 65.0 vs 64.8, utility 63.3 vs 63.5; per-dataset AIA within ±4 pts). Seeds and hyper-parameters will be listed in A.6.

LLM judge vs human. A human annotator re-scored 40 random final answers (20 per open-ended dataset) with the same protocol and 1–5 scale.

Table R10. LLM judge vs human annotation

DatasetnMean score (paper)Mean score (human)Spearman ρ
GerlayQA202.502.350.91
HealthCareMagic202.152.250.67

Human and judge means agree (2.30 vs 2.33), with ρ=0.81 overall; 78% of scores are identical and all are within ±1.

Sample size. The open-ended sets have 100 test queries each because samples must contain a real private attribute, have a verifiable answer and pass calibration, and judging is costly; the paired CIs show which gaps this size can resolve.

Benchmark construction. A new appendix will give the full procedure; in brief:

(i) Difficulty calibration. As described in Sec. 2.2, we remove samples that are too easy or too hard for utility differences among defenses to be measurable; the new appendix will document the criterion, thresholds and the number of records kept per dataset.

(ii) Probe generation. GPT-5.4 generated spans and four-option probes under a strict JSON schema (exact-substring spans, one span per probe, same-type options, non-overlapping numeric ranges, no "unknown" option, direct identifiers never probed). Local validators rejected invalid spans or options, with up to three regenerations (>96% of records succeeded). Options were shuffled locally, so the correct position is uniform.

(iii) Audits. A GPT-5.4 audit (source text + probe question, no options) removed every probe whose target is the same as, disclosed by, presupposed by or related to the task question (44.8% of candidates). Probes whose target string still appeared in the task question were regenerated and re-audited until all passed; the released files validate with 0 errors. The two-annotator check (Table R18) is an additional post-hoc audit.

AuthorsOpenReview ↗

Author Response (W4/Q4, Q1)

W4 / Q4 — How much of the 14.9-pt gain comes from training rather than prompt and loop budget, and what does deployment cost?

AA already uses the same untrained 4B in the same 5-round loop (A.4), so the budget-aligned AIA reduction is 6.7 pts, not 14.9. We also added a prompt-aligned baseline that is identical to Ours (same prompt incl. relevance decomposition, 5 rounds, same chain) except for the untrained weights.

Table R11. Decomposition of the gain over the base prompt (4B, 27B attacker)

RewriterRoundsTrainedGerHCMMedQACosmosAvg AIAUtility
Base prompt0no76.576.279.786.679.861.8
AA5no61.170.271.083.671.562.5
Prompt-aligned (new)5no65.470.270.483.272.362.4
Ours5yes59.361.959.378.964.863.5

The 14.9-pt gap thus splits evenly (≈7.5 pts each) between the prompt plus loop (79.8 → 72.3) and training (72.3 → 64.8, at comparable utility, +1.1). Within training, paper Fig. 5(a) shows that the privacy gain comes from SFT (AIA 58.5 after SFT), while RL gives part of it back (→ 64.8) for utility. We therefore do not attribute the privacy gain to RL; its role is to restore utility and set the trade-off point (Table R7).

Table R12. Rounds sweep (same protocol, 1 / 3 / 5 self-attack rounds)

RewriterRoundsGerHCMMedQACosmosAvg AIAUtility
Base prompt076.576.279.786.679.861.8
AA (untrained)165.472.474.187.174.763.6
AA (untrained)364.270.271.285.972.963.5
AA (untrained)5 (paper)61.170.271.083.671.562.5
Ours (trained)160.563.059.879.865.863.7
Ours (trained)360.560.259.278.464.663.4
Ours (trained)5 (paper)59.361.959.378.964.863.5

Extra rounds help the untrained model (≈−1.6 AIA per two rounds) but leave Ours essentially flat (within ≈1 pt). Ours with one round (65.8) is 5.7 pts below AA with five rounds (71.5) at comparable utility (63.7 vs 62.5), so a single round suffices for deployment.

Table R13. Deployment cost per query of the local 4B rewriter (one RTX A6000, vLLM, single-stream)

RewriterRoundsLocal tokens (in + out)Latency mean (median)Peak GPU memory
Ours58.8k21.0 s (18.0 s)13.9 GB incl. pre-allocated KV (weights 8 GB bf16)
AA59.3k24.9 s—
AgentStealth59.1k24.8 s—
TRACE-RPS517.3k101.9 s15.9 GB

Tokens are averaged over all 1,000 queries; latencies are single-stream on 20 queries per dataset (T=0); "—" means not measured. Each extra round adds one self-attack and one rewrite call (≈1.8 s each); the cloud call does not depend on rounds. AA and AgentStealth share ReAnon's call structure, so at equal rounds ReAnon is ≈15% faster with 3–5% fewer tokens (shorter rewrites); TRACE-RPS needs 2× the tokens and ≈5× the latency (attention capture and suffix search). Ours' 13.9 GB peak includes vLLM's pre-allocated KV cache, so it fits a 16 GB GPU. These are workstation measurements, not evidence of on-device feasibility, which remains future work. The 1-round configuration (6.7 s, 2.9k tokens) is 3.7× faster than 5-round AA with 5.7 pts lower AIA.

Q1 — Do AIA and the ranking change if the attacker also sees the task question and options?

In the submission the attacker reads only x″; the task question and options are sent in the clear, and our audit is designed to ensure they reveal no probe answer. We re-ran the attack with x″ + question + options.

Table R6. "Fully visible to the cloud" attacker (Qwen3.5-27B; Δ computed before rounding)

RowPaper (x″ only)Fully visible (x″ + question + options)Δ
Base prompt79.880.0+0.3
AA71.571.1−0.3
TRACE-RPS69.370.1+0.7
AgentStealth65.465.3−0.1
Ours64.865.0+0.2

AIA changes by less than 1 pt for every method and the ordering of the methods by AIA is unchanged, so the question and options add almost no attribute information and paper Table 3 closely reflects this setting. Ours and AgentStealth remain on par (65.0 vs 65.3). We will report this in the appendix.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

We thank all reviewers for their thoughtful feedback. It helped us scope our claims more precisely, make the evaluation more rigorous, and strengthen the experimental evidence. We summarize the revisions below and indicate the corresponding reviewer comments for each substantive change.

    1. Claim the contribution in the Abstract and Introduction to scoring privacy and utility on the same query within the full local–cloud–local pipeline. We also split the "Task utility" column of Table 1 into "text retention / readability" and "downstream task success" and added DP-Fusion to the related work. In response to Reviewer mJT6 W1 & Q2 and Reviewer UUiU W1.
    1. Added validity checks for the AIA metric: a prior-only (no-context) attacker with a net-leakage view in Table 3, an option-permutation test, parse-failure statistics, an open-ended (option-free) attack, and a "fully visible to the cloud" attacker that also sees the task question and options. In response to Reviewer Cbtz W1, Q1 & Q3 and Reviewer mJT6 W4.
    1. Evaluated stronger and adaptive attackers: GPT-5.4, defense-aware attackers (Qwen3.5-27B and GPT-5.4), and a white-box likelihood attack that uses the rewriter's weights. We also expanded the threat-model discussion in the Limitations. In response to Reviewer Cbtz W1, Reviewer mJT6 W2, and Reviewer UUiU W2 & Q.
    1. Added paired bootstrap 95% CIs and significance tests to all main tables, a second RL-seed replication, and a list of seeds and hyper-parameters in Appendix A.6. Claims are reworded accordingly: privacy is on par with AgentStealth and utility is significantly higher. In response to Reviewer Cbtz W3 & Q5 and Reviewer mJT6 W4 & Q1.
    1. Reframed the privacy–utility results. Per-domain metrics are now the primary results and the composite utility is secondary. We also added a privacy–utility curve from a λ sweep and revised the wording in the Abstract and Section 4.2. In response to Reviewer Cbtz W2 and Reviewer UUiU W1.
    1. Separated the contribution of training from that of the prompt and the iteration loop. We added a prompt-aligned untrained baseline, a sweep over the number of self-attack rounds, and per-query deployment costs (tokens, latency, GPU memory). Fig. 5(a) now also appears as a table in the main text, and the on-device claim is clarified. In response to Reviewer Cbtz W4 & Q4 and Reviewer UUiU W1.
    1. Clarified the methodological changes relative to prior work and what each training component contributes. We moved the MI interpretation of the privacy reward into the main text and added an analysis of attacker calibration and of non-Bayes-optimal attackers in Appendix A.8. In response to Reviewer Cbtz Q2 and Reviewer UUiU W1.
    1. Added quality metrics for the transmitted rewrite (fluency, information retention, perplexity) and utility when the cloud answers from the rewrite alone. We also clarified in Section 4.3 that the benefit of local composition depends on the size of the local model. In response to Reviewer mJT6 W3.
    1. Added DP-Fusion as a formal-DP baseline at two privacy budgets in Table 3, using the same local model and pipeline. In response to Reviewer mJT6 W6 & Q3.
    1. Added a new appendix on benchmark construction covering difficulty calibration, probe generation, and audits. It also includes a two-annotator human validation of probes, a comparison of the LLM judge against human scoring, and memorization checks. In response to Reviewer Cbtz W3 & Q5 and Reviewer mJT6 W4 & W5.
    1. Extended the Ethics Statement with data sources, licenses, release form (IDs, annotations, and a rebuild script for GerlayQA and HealthCareMagic), and a confirmation that no new personal information is exposed. In response to Reviewer mJT6's ethics review.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.