acceptodds
Under review as a conference paper at ICLR 2027

PrivMeSA: Privacy-Aware Self-Evolving Multi-Agent System for Medicine via Local-Remote LLM Collaboration

Abstract

Clinical large language model (LLM) agents deployed locally can consult more capable remote models, but doing so risks exposing patient information. Privacy-conscious delegation places disclosure decisions with a local agent, yet removing explicit identifiers is insufficient: quasi-identifiers can accumulate across multi-turn consultations and repeated patient visits to enable re-identification. We introduce PrivMeSA, a privacy-aware self-evolving multi-agent system that learns to control disclosure and retains remote expertise for local reuse. A local agent manages each encounter and consults remote specialists that may request additional information. Reinforcement learning balances task accuracy against direct disclosure and registry-based re-identification risk, with privacy evaluated over the complete outbound transcript of each encounter. A local lesson memory distills completed consultations into generalized clinical guidance and retrieves relevant lessons before transmission, allowing subsequent cases to reuse expertise without another remote exchange. Memory grows without additional outcome labels or parameter updates. On an emergency-department benchmark built from MIMIC-IV-ED records, PrivMeSA improves mean task accuracy over delegation by up to 15.8 percentage points. In the same setting, PrivMeSA reduces the disclosure of personal details from 98.0% to 0.2% of cases and the share of cases in which the patient can be narrowed to ten or fewer registry patients from 74% to 0%.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
privacy, LLM agents, multi-agent systems, delegation, self-evolving agents, memory, reinforcement learning, k-anonymity, healthcare, clinical decision support

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_WxRHOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PrivMeSA, a privacy-aware multi-agent framework for clinical decision support that enables a local LLM to consult more capable remote specialist agents while minimizing patient information disclosure.

The framework combines two key components. First, a privacy-aware reinforcement learning objective trains the local agent to balance clinical task accuracy, direct disclosure of personal information, and re-identification risk measured through registry-based k-anonymity over multi-turn consultation transcripts. Second, a self-evolving local lesson memory distills completed remote consultations into reusable guidance, allowing subsequent cases to benefit from previously acquired expertise without additional remote communication.

The authors introduce a multi-turn emergency-department benchmark constructed from MIMIC-IV-ED, including repeated patient visits and specialist-initiated follow-up questions. Experiments with Gemma 4 12B and Granite 4.2 8B demonstrate improved task accuracy and substantially reduced disclosure compared with delegation and PAPILLON baselines. Additional ablations investigate the training objective, consultation behavior, and benefits of test-time memory accumulation.

Critical strengths

1. Important and well-motivated problem formulation for privacy-aware multi-agent collaboration.

The paper extends privacy-conscious delegation beyond single-message sanitization by explicitly considering the accumulation of quasi-identifiers across multi-turn consultations and repeated patient visits. Evaluating disclosure over complete outbound transcripts and linking patient encounters provides a more realistic privacy assessment than examining individual messages in isolation. The formulation is particularly relevant to healthcare applications where local systems may need assistance from remote models without exposing sensitive patient information.

2. Strong empirical results and meaningful analysis of privacy-utility trade-offs.

The proposed approach achieves a favorable balance between task accuracy and privacy across two local model backbones. The improvements are especially pronounced for Granite 4.2 8B, where mean accuracy increases from 0.425 under delegation to 0.583, while direct disclosure falls from 98.0% to 0.2%. The paper also includes useful ablations of privacy-aware training and consultation behavior. The test-time memory experiments provide evidence that previously acquired knowledge can reduce remote interactions without degrading predictive accuracy in the evaluated setting.

Critical weaknesses

1. The privacy guarantees are limited by the attacker-dependent evaluation protocol.

Although the paper reports that no patient is re-identifiable under PrivMeSA, this conclusion is based on a particular registry-matching procedure and an LLM-based quasi-identifier extractor. Appendix B.2 states that visits are excluded from re-identification statistics when extraction errors produce attributes inconsistent with the patient's actual record. This exclusion rule could lead to optimistic estimates of privacy, particularly if extraction errors differ across methods.

Furthermore, the evaluation focuses on a predefined set of quasi-identifiers and uses k ≤ 10 as the re-identification threshold. While the comparison with an independent attacker backbone is helpful, it does not establish robustness to broader auxiliary information, different linkage strategies, or alternative attacker models. The reported zero re-identification rate should therefore be interpreted as a result under the specified evaluation protocol rather than a general privacy guarantee.

2. The experimental comparisons do not fully isolate the contributions of privacy-aware RL and self-evolving memory.

The main baselines, Delegation and PAPILLON, use untrained local models, whereas PrivMeSA benefits from task-specific GRPO training and an accumulated lesson memory. This makes it difficult to determine how much of the accuracy improvement arises from privacy-aware optimization versus general task-specific fine-tuning or memory reuse.

The local-only RL baseline and the reward ablations partially address this issue, but the reward ablations use a simplified configuration with a single remote specialist and shorter training. A more direct comparison using an accuracy-only trained delegation policy with identical specialist access and training budget would better establish the contribution of the privacy-aware objective.

Additionally, the test-time memory experiment reports a reduction of approximately 15% in remote messages, but the associated accuracy gains are relatively small and vary across evaluation quarters. More detailed analysis of memory retrieval correctness, lesson quality, and failure cases would strengthen the claim that the memory safely transfers expertise across patients.

Questions for authors

Q1. Robustness of re-identification evaluation: Appendix B.2 excludes visits when the attacker extracts attributes that do not match the patient's record. What fraction of visits is excluded for each method, and how would the privacy results change if these visits were retained or treated conservatively? Could the authors additionally evaluate robustness to alternative quasi-identifier extraction and matching strategies?

Q2. Attribution of gains from privacy-aware RL: Could the authors compare PrivMeSA against an accuracy-only GRPO-trained delegation baseline with the same local model, remote specialists, training budget, and memory configuration? This would help isolate the benefits of the privacy-aware reward from those of task-specific policy optimization.

Q3. Memory quality and potential negative transfer: How often does the lesson memory retrieve clinically incorrect or inapplicable guidance, and how frequently does the local policy accept such lessons? Could the authors analyze whether memory accumulation introduces errors or negative transfer across clinically similar but distinct patient cases?

Q4. Generalization of privacy behavior: How well does the learned disclosure policy generalize to different remote specialists, different follow-up-question strategies, and unseen quasi-identifier combinations? The current results demonstrate strong performance within the constructed consultation environment, but the robustness of the policy to changes in the interaction and attacker distributions remains unclear.

3 comments
AuthorsOpenReview ↗

Response to Reviewer WxRH (part 1 of 2)

Thank you for the detailed review and for finding our multi-turn, linked-visit evaluation more realistic. We have run all four requested analyses and address them below, in two parts.

W1, Q1. Robustness of re-identification to the attacker's protocol

Excluded visits (Appendix B.2), kept or treated conservatively

A visit is excluded when an extracted value contradicts the patient's record, so the true patient is not among the k matches. As requested, we have measured how many visits are excluded and how the result changes when they are kept or counted as re-identified.

Table R1: Share of visits re-identified (k ≤ 10) under three ways of handling excluded visits, where the true patient is not among the attacker's k matches. Dropped: excluded visits are removed, as in the paper. Scored by k: excluded visits stay in and count as re-identified only if k ≤ 10. All re-identified: every excluded visit counts as re-identified, the most conservative choice.

Local model and methodVisits excludedDropped (paper) ↓Scored by k ↓All re-identified ↓
Gemma Delegation21.2%48.5%53.9%59.4%
Gemma PAPILLON36.4%1.0%1.8%37.1%
Gemma PrivMeSA (ours)21.0%0.0%0.2%21.0%
Granite Delegation7.7%74.4%75.5%76.4%
Granite PAPILLON21.6%41.7%42.2%54.3%
Granite PrivMeSA (ours)16.3%0.0%0.0%16.3%
  • PrivMeSA has the lowest rate under every treatment for both local models.
  • Even when every excluded visit counts as re-identified, the most conservative choice, PrivMeSA has 21.0% with Gemma and 16.3% with Granite, against at least 37.1% and 54.3% for the baselines.

Alternative extraction and matching strategies

Our attacker counts only explicitly stated values, so every match rests on information that was actually sent. We have added an extended attacker that also reads ranges and descriptive terms ("in her 70s", "elderly", "febrile") and matches the chief complaint by meaning, with a threshold set on Delegation only. PrivMeSA was not trained against it.

Table R2: Accuracy and re-identification under the extended attacker, counting only matches that include the true patient.

Local model and methodMean task accuracy ↑Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Gemma Delegation0.56451.6%98.0%
Gemma PAPILLON0.5262.3%55.7%
Gemma PrivMeSA (ours)0.5921.5%44.6%
Granite Delegation0.42575.5%100%
Granite PAPILLON0.42043.6%99.0%
Granite PrivMeSA (ours)0.5830.0%6.7%
  • PrivMeSA is the most accurate and the least re-identified with both local models.
  • We use k ≤ 10 because public health data releases commonly hide any group of 10 or fewer people, since such small groups make a person easy to single out. The ranking of methods is the same at every threshold from k ≤ 1 to k ≤ 100, and we will report these in the paper.
  • We agree that zero re-identification holds for the tested dataset and attacker, not as a guarantee, and will say so in the paper.

W2, Q2. Accuracy-only GRPO with the same specialists, budget and memory

Appendix E.2 used a shorter run with one remote specialist. As requested, we have now trained the full PrivMeSA system with both privacy weights at 0 and nothing else changed: the same local model, three remote specialists, memory and 200 steps.

Table R3: Accuracy-only GRPO against PrivMeSA, same setup (Gemma 4 12B, 453 cases). Changes from PrivMeSA are in points.

MethodMean task accuracy (change) ↑Direct disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Accuracy-only GRPO0.567 (−2.6)90.3%31.4%92.0%
PrivMeSA (ours)0.5920.9%0.0%0.0%
  • Without the privacy terms, the policy does not learn to withhold (90.3% direct disclosure, close to Delegation's 94.3%).
  • The privacy reward terms do not lower accuracy. Compared with the accuracy-only policy, PrivMeSA changes mean accuracy by +2.6 points (95% CI +0.2 to +4.9, a paired bootstrap interval over the 453 cases).
  • The privacy terms make the policy ask short, focused questions instead of sending the record. It asks more of them (5.3 vs 4.4 per case, at 19 vs 41 tokens each) and receives more guidance from the specialists (441 vs 368 tokens per case), which helps most on Primary diagnosis (0.380 vs 0.342).
AuthorsOpenReview ↗

Response to Reviewer WxRH (part 2 of 2)

W2, Q3. Quality of memory lessons and negative transfer

On the 897-case test stream, the memory offered 419 lessons, and the policy answered 384 of these tasks (92%) from the memory lesson instead of asking a remote specialist. We have added an LLM judge (GPT-5.6-Luna) to rate each lesson. We have also compared each answer with the answer to the same task in an empty-memory run, where the policy had to ask a remote specialist.

Table R4: Accuracy of answers taken from memory, against asking a remote specialist for the same task (Gemma 4 12B, 897-case test stream).

Lesson typeShare of the 384Accuracy from the lesson ↑Accuracy asking a remote specialist ↑Difference (points)
All100%0.5910.612−2.1
From same patient, earlier visit17%0.5000.516−1.6
From another patient76%0.6080.632−2.4
Judged clinically unsound4%0.2670.400−13.3
  • The judge rated 64% of used lessons a full fit to the patient, 36% a partial fit and none inapplicable. Clinically unsound lessons are rare (15 of 384).
  • Answering from a lesson is only slightly less accurate than asking a remote specialist. Over the whole stream, memory keeps accuracy similar to an empty memory (0.591 vs 0.588) with 15% fewer messages.
  • Lessons from other patients lose about as much as those from the same patient (−2.4 vs −1.6 points), so we see no negative transfer between similar but distinct patients.

Q4. Generalization to other specialists and follow-up strategies

Different remote specialists

We have replaced GPT-5.6-Luna with Claude Haiku 5.5 as the remote specialists, without retraining.

Table R5: Results with Claude Haiku 5.5 as the remote specialists (local model Gemma 4 12B, 453 cases).

MethodMean task accuracy ↑Direct disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Delegation0.54288.7%44.4%91.0%
PAPILLON0.49319.9%1.8%11.0%
PrivMeSA (ours)0.5531.1%0.0%1.0%
  • Every method loses 2 to 4 accuracy points compared with GPT-5.6-Luna (Table 1).
  • PrivMeSA stays the most accurate and most private, with almost no change in direct disclosure (1.1% vs 0.9%).

Different follow-up questions

We have tested a new follow-up strategy, where the remote specialists ask for personal details with new reasons PrivMeSA has not seen in training. For example, the diagnostician asked for the full name, address and next of kin "because the record cannot be filed without them", and the clinician asked for the job title and employer for "return-to-work planning". PrivMeSA declined all 1,096 new requests without disclosing anything.

Unseen patients and quasi-identifier combinations

All test patients are held out from training (Sec. 4.1), so every result above is on quasi-identifier combinations the policy has never seen. We have also evaluated PrivMeSA without retraining on MC-MED (Kansal et al., 2025): 448 cases (100 patients) from the Stanford emergency department, a different hospital than MIMIC-IV-ED. Its registry holds all 70,545 MC-MED patients.

Table R6: MC-MED results (448 cases, Gemma 4 12B, no retraining). Task columns give accuracy.

MethodMean task accuracy ↑Disposition ↑Primary diagnosis ↑Direct disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Delegation0.5160.6850.34793.5%78.4%98.0%
PAPILLON0.4820.6810.28419.4%6.1%24.0%
PrivMeSA (ours)0.4970.6790.3151.8%1.1%15.0%
  • PrivMeSA has the lowest disclosure and re-identification at the new hospital, without retraining.
  • Re-identification is higher than on MIMIC for every method, because the registry is a third the size.
  • PrivMeSA's mean accuracy is within 2 points of Delegation and above PAPILLON.

We hope these results address your concerns, and we are glad to answer any further questions.

Thank you again for your detailed review and your positive assessment of our work. We have uploaded a revised paper, with all new results from the discussion in blue.

As promised in our reply, the paper now reports re-identification at every threshold from k ≤ 1 to k ≤ 100 (Appendix B.5) and states that our re-identification results hold for our registry and attacker rather than as a formal guarantee (Section 4.2). It also adds the excluded visits (Appendix B.4), the accuracy-only GRPO run (Table 4 and Appendix E.2), MC-MED (Appendix A.4), lesson quality (Appendix E.6), and the results with Claude Haiku 5.5 and new requests (Appendix E.7).


If you have any remaining concerns or questions, please let us know and we will try our best to answer and address them in the remaining time. If our responses have addressed your concerns, we would be grateful if you would consider raising your score.

Reviewer_fvK8OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper presents a local-global collaborative framework to tackle emergency department question answering, featuring competitive accuracy while minimizing the leakage of personally identifiable information. The authors use a balanced RL reward that includes accuracy, the percentage of retained personal details, and the percentage of patients in the cohort who are re-identifiable. The work also uses a memory that accumulates knowledge over time to reduce external queries.

Critical strengths
  • The need for privacy preservation and real-time responsiveness is clinically relevant.

  • Compared against two baselines, the accuracies across three categories of tasks do not come at the cost of privacy leakage.

Critical weaknesses
  • Critical ablations are missing, such as per-role ablation. When these studies are added, it would be beneficial to show the number of tokens used besides listed performance.
  • Some design choices are not sufficiently justified. The balancing coefficients lambda d and lambda a are not analyzed in terms of sensitivity.
  • Papillon as a baseline may not be appropriate because it does not involve explicit role playing and collaboration. On the other hand, there is at least one work before 2026 on a similar topic (https://arxiv.org/pdf/2504.00299 published in AAAI 2026). While its applicability to MIMIC dataset is not immediately clear, the authors could at least discuss the work qualitatively.
  • The memory is continuously built up during training and in the “memory on” setup during evaluation in Table 3. In this case, the order where cases are seen may change the training and inference trajectories. Such an effect needs to be quantified.
Questions for authors

As above

4 comments
AuthorsOpenReview ↗

Response to Reviewer fvK8 (part 1 of 2)

Thank you for noting the clinical relevance of the setting and that our accuracy does not come at a cost in privacy. As suggested, we have added the analyses below, in two parts.

W1. Per-role ablation, with the number of tokens used

Thank you for suggesting this ablation. We have added a per-role ablation, removing one remote specialist at a time from both the team and the local model's prompt. We evaluated the trained Gemma 4 12B policy without retraining on the 453 test cases (100 patients).

Table R1: Task accuracy with one remote specialist removed, averaged over the 453 cases (Gemma 4 12B). Changes from the full team are in points.

TeamMean (change) ↑Disposition ↑Primary diagnosis ↑Operative procedure ↑
Full team (PrivMeSA)0.5920.6920.3800.705
No clinician0.569 (−2.3)0.6690.3620.677
No diagnostician0.562 (−3.0)0.6880.3440.655
No interpreter0.580 (−1.2)0.7040.3640.673

Table R2: Privacy with one remote specialist removed (Gemma 4 12B, 453 cases).

TeamDirect disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Full team (PrivMeSA)0.9%0.0%0.0%
No clinician4.9%0.0%1.0%
No diagnostician0.0%0.0%0.0%
No interpreter0.9%0.0%0.0%
  • Removing the diagnostician changes mean accuracy by −3.0 points (95% CI −5.1 to −0.9), mostly on Primary diagnosis and Operative procedure.
  • Removing the clinician changes it by −2.3 points (95% CI −4.4 to −0.3), mostly on Disposition and Operative procedure. It also raises direct disclosure from 0.9% to 4.9%.
  • Both intervals are 95% paired bootstrap confidence intervals over the 453 cases, and both exclude zero.
  • Removing the interpreter does not significantly change accuracy.

We have also counted the tokens each method uses per case, on the remote side and on the local model.

Table R3: Average tokens per case on the 453 test cases (Gemma 4 12B). Local tokens are the text the local model writes, including PAPILLON's rewrites and aggregates.

MethodMessages to remote specialists ↓Tokens sent to remote specialists ↓Tokens received from remote specialists ↓Tokens generated locally ↓Total tokens ↓
Delegation7.143886051,0882,081
PAPILLON6.086238612,3393,822
PrivMeSA (ours), full team5.331044419211,466
PrivMeSA, no clinician5.22964638951,454
PrivMeSA, no diagnostician5.06843988681,351
PrivMeSA, no interpreter5.451004589151,473
  • PrivMeSA uses the fewest tokens of the three methods: 1,466 per case on average, against 2,081 for Delegation and 3,822 for PAPILLON.
  • PrivMeSA sends only 104 tokens per case to the remote specialists, against 388 for Delegation and 623 for PAPILLON. With Granite, the numbers are 98, 766 and 1,108.
  • Removing a specialist changes the total by at most 115 tokens per case.

W2. Sensitivity to λd and λa, and an accuracy-only reward

Sensitivity to λd and λa

We have trained two further settings. With λd = 0.6 and λa = 0.3, accuracy stays similar (0.595 vs 0.592) and Direct disclosure stays low (2.6%), while Re-identification after linking visits rises to 9%, as expected from the lower λa. With λd = 0.4 and λa = 0.5, Re-identification stays at 0% with an even higher median k, while Direct disclosure rises, as expected from the lower λd. Each coefficient therefore controls its own risk, and the paper's setting (λd = 0.5, λa = 0.4) keeps both risks low.

Accuracy-only reward (λd = λa = 0)

Appendix E.2 compares the full reward with an accuracy-only reward, in a shorter run with one remote specialist. For this reply, we have also trained it with the full team of remote specialists, the same memory and the paper's exact settings.

Table R4: GRPO with λd = λa = 0 (accuracy only) against PrivMeSA, same setup (Gemma 4 12B, 453 cases).

MethodMean task accuracy ↑Direct disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Accuracy-only GRPO0.56790.3%31.4%92.0%
PrivMeSA (ours)0.5920.9%0.0%0.0%
  • Setting both coefficients to 0 removes privacy almost entirely. The policy leaks a personal detail in 90.3% of cases, close to Delegation's 94.3%.
  • The privacy terms do not lower accuracy. Compared with the accuracy-only policy, PrivMeSA changes mean accuracy by +2.6 points (95% CI +0.2 to +4.9). They make the policy ask short, focused questions instead of sending the record, which helps most on Primary diagnosis (0.380 vs 0.342).
AuthorsOpenReview ↗

Response to Reviewer fvK8 (part 2 of 2)

W3. PAPILLON as a baseline, and the related work of Zhang et al. (AAAI 2026)

PAPILLON in a role-playing, multi-turn setting

Thank you for raising this point. PAPILLON is designed for local-remote LLM collaboration, the same setting as PrivMeSA. It protects privacy by rewriting each query to remove private information. Role-play and multiple turns do not conflict with this method. In the paper, PAPILLON follows its original design with one remote model: its rewriter processes every message the local model sends, in all turns, to a single remote specialist that also asks for personal details (Appendix D). Our Delegation baseline is the three-specialist team without protection, so it is the role-play collaboration baseline.

For this reply, we have also modified and run PAPILLON on the team of three remote specialists. Each message is rewritten with PAPILLON's released prompts and sent to the specialist the local model addressed. Everything else follows the paper's PAPILLON setup.

Table R5: PAPILLON with one remote specialist (paper) and with the three-specialist team (Gemma 4 12B, 453 cases).

MethodMean task accuracy ↑Direct disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Delegation (three specialists)0.56494.3%48.5%92.0%
PAPILLON (one remote specialist, paper)0.52621.2%1.0%8.0%
PAPILLON (three specialists)0.53829.1%0.7%2.0%
PrivMeSA (ours, three specialists)0.5920.9%0.0%0.0%
  • On the three-specialist team, PAPILLON's accuracy rises slightly (0.538 vs 0.526) but stays 5.4 points below PrivMeSA.
  • It leaks personal details more often with the team (29.1% vs 21.2%), and linking visits still re-identifies 2.0% of patients, against 0% for PrivMeSA.
  • With the same three remote specialists, PrivMeSA remains the most accurate and the most private method.

Accuracy-only GRPO and Dynamic Cheatsheet baselines

We have also added two new baselines. GRPO trained for accuracy only (Table R4) reaches 0.567 accuracy but leaks in 90.3% of cases. Dynamic Cheatsheet, a self-evolving memory (Table R6), matches our memory's accuracy on the 897-case test stream (0.584 vs 0.591) but has 16.8% direct disclosure instead of 0.7%.

Table R6: Dynamic Cheatsheet against our memory with the trained PrivMeSA policy (Gemma 4 12B, 897-case test stream).

MemoryMean task accuracy ↑Direct disclosure (any leak) ↓Re-identification (linked visits): k ≤ 10 ↓Messages per case ↓
PrivMeSA memory (ours)0.5910.7%0.5%5.96
Dynamic Cheatsheet0.58416.8%37.0%7.13
No memory0.5880.4%1.0%7.03

Discussion of Zhang et al. (AAAI 2026)

Thank you for pointing us to this work. Its code is not released, so we could not add it as a baseline. We will discuss it in the related work section.

  • Zhang et al. protect a query by changing its topic. The remote model receives a synthesized query on a different topic with the same reasoning structure, and the answer is reconstructed locally with tools.
  • This does not carry over to clinical consultation, because the topic is the patient's condition, and the remote specialist needs it to answer.
  • Their method also does not address the problem we study. It handles each query on its own, does not consider linkage across turns or visits, and does not learn a disclosure policy.

W4. Effect of case order on the growing memory

During training, patients are reshuffled on every pass, so training does not depend on one order.

We have rerun the memory evaluation on the same 897 test cases in two new random patient orders. Each patient's visits stay in time order.

Table R7: Memory evaluation in three patient orders (Gemma 4 12B). Reduction columns give the percentage drop in messages per case with the growing memory, against an empty memory (7.03 messages per case).

Patient orderMean task accuracy ↑Messages per case ↓Reduction: all cases ↑Reduction: first third ↑Reduction: second third ↑Reduction: last third ↑
1 (paper)0.5915.9615.2%11%16%18%
20.5975.8516.7%15%16%19%
30.5915.8616.6%13%19%18%
  • Accuracy varies by at most 0.6 points across orders. The message reduction varies by at most 1.5 points.
  • In every order, the reduction grows after the first third, as the memory fills. The paper's order gives the smallest reduction.

We hope these results address your concerns, and we are glad to answer any further questions.

Thank you for the detailed response and the clarifications. All my concerns are fully addressed. I decide to keep my initial rating.

Thank you for confirming that all your concerns are fully addressed. We have uploaded a revised paper, with all new results from the discussion in blue. We also have some new updates relevant to your points that you may find interesting, all included in the revised paper.

W3. Claude Haiku 5.5 as the remote models

PrivMeSA works with remote specialists that play clinical roles. To check that it does not depend on one remote model, we replaced GPT-5.6-Luna with Claude Haiku 5.5 as every remote model, without retraining.

Table F1: Results with Claude Haiku 5.5 as the remote models (Gemma 4 12B, 453 cases).

MethodMean task accuracy ↑Direct disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Delegation0.54288.7%44.4%91.0%
PAPILLON0.49319.9%1.8%11.0%
PrivMeSA (ours)0.5531.1%0.0%1.0%

PrivMeSA stays the most accurate and the most private method. Its Direct disclosure barely changes (1.1% vs 0.9% with GPT-5.6-Luna).

W3. New requests for personal details

In the role play, the specialists ask for personal details. To check that PrivMeSA does not only decline the requests it saw in training, we gave the specialists new requests with new reasons.

Table F2: The new requests from the remote specialists.

SpecialistRequestReason given
DiagnosticianFull name, home address and next of kin"The record cannot be filed without them"
ClinicianJob title and employerReturn-to-work planning

PrivMeSA declined all 1,096 such requests and disclosed nothing (Appendix E.7).

W4. Accuracy of lessons from earlier patients

The order of the cases decides which lessons are in memory when a patient arrives. To check whether lessons from earlier patients can mislead the policy, we compared each answer taken from a lesson with asking a remote specialist for the same task.

Table F3: Accuracy of answers taken from a memory lesson, against asking a remote specialist (Gemma 4 12B, 897-case test stream).

Lesson originShare of the 384 answersAccuracy from the lesson ↑Accuracy asking a remote specialist ↑Difference (points)
All100%0.5910.612−2.1
Same patient, earlier visit17%0.5000.516−1.6
Another patient76%0.6080.632−2.4

Lessons from other patients are about as reliable as lessons from the same patient's earlier visits, so lessons left by earlier patients do not mislead the policy (Appendix E.6).


Thank you again for your time and helpful feedback. If these updates address your concerns, we would be grateful if you would consider raising your score. If anything still keeps the score at its current level, or if you have any further questions, please let us know and we will try our best to answer and address them in the remaining time.

Reviewer_YbLdOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates continuous self-evolution through collaboration between local and remote LLMs under privacy constraints and proposes a method called PrivMeSA. By learning to control information disclosure and reuse expert knowledge, the method protects private information while continually improving the local model’s capabilities through collaboration. Experiments on MIMIC-IV-ED show that the proposed method improves task accuracy while reducing personal information leakage.

Critical strengths
  1. Data privacy remains a key concern in specialized domains such as healthcare. Research in this area can help accelerate the adoption of AI across different domains.

  2. The paper is clearly structured, and the proposed approach is well explained. The local self-evolution mechanism also enables continued improvement.

Critical weaknesses
  1. The experimental comparisons are insufficient. In particular, comparisons with other self-evolving approaches for local models are missing, leaving the effectiveness of the self-evolution training algorithm insufficiently validated. Furthermore, the evaluation is limited to a single dataset.

  2. The definition and detection of private information in the experiments have limitations, and the reported statistics may miss some disclosures. For example, semantically equivalent expressions with different wording may evade the detection of sensitive information.

Questions for authors

Although protecting medical information is important, failing to provide complete and accurate information when consulting an external model could compromise the accuracy of medical decisions. Should privacy protection instead be enforced through mechanisms at the network layer or other infrastructure layers?

4 comments
AuthorsOpenReview ↗

Response to Reviewer YbLd (part 1 of 2)

Thank you for the detailed review and for finding the paper clearly structured. We address each point below, in two parts.

W1. Comparison with other self-evolving methods for local models

We have added Dynamic Cheatsheet (DC) by Suzgun et al. (2025), a label-free self-evolving memory, as a new baseline. DC keeps one cheatsheet in the policy's prompt. After each case, the local model rewrites it with DC's curator prompt, adapted to our tasks. Our memory instead stores separate lessons, and the policy answers from a matching lesson without sending a message.

In this evaluation, both DC and our memory start empty and learn from the same consultations. The policy's privacy rules are the same in every run. We ran each memory with both the trained and the untrained PrivMeSA policy, on the 897-case test stream (200 patients).

Table R1: Self-evolving evaluation on the 897-case test stream, with the new Dynamic Cheatsheet baseline (Gemma 4 12B).

PolicyMemoryMean task accuracy ↑Direct disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓Messages per case ↓
Trained PrivMeSAPrivMeSA memory (ours)0.5910.7%0.0%0.5%5.96
TrainedDynamic Cheatsheet0.58416.8%8.0%37.0%7.13
TrainedNo memory0.5880.4%0.1%1.0%7.03
UntrainedPrivMeSA memory0.56584.4%33.5%86.0%6.11
UntrainedDynamic Cheatsheet0.56696.9%52.3%92.0%7.36
UntrainedNo memory0.56089.6%35.9%90.0%7.17
  • Accuracy is similar with both memory systems. With the trained policy, DC is 0.6 points below ours.
  • DC leaks far more: 16.8% direct disclosure with the trained and 96.9% with the untrained policy. It writes rules that make private details look useful, such as "admission is likely if home supervision is unavailable", so the policy sends them.
  • Re-identification follows the same pattern. With the trained policy, DC narrows 37.0% of patients to k ≤ 10 after linking visits, against 0.5% with our memory.
  • Only our memory reduces messages to the remote specialists: by 15% with both the trained and the untrained policy.

W1. Evaluation beyond a single dataset

Thank you for this suggestion. We have added MC-MED (Kansal et al., 2025), with 118,385 adult visits to the Stanford emergency department from 2020 to 2022. It comes from a different hospital than MIMIC-IV-ED, with a similar record format and repeat visits. We sampled 448 cases (100 patients with 3 to 10 visits each), as in our MIMIC test set. Its registry holds all 70,545 MC-MED patients. We used the MIMIC-trained policies and memory without retraining. MC-MED has no procedure coding, so we report two tasks.

Table R2: MC-MED results (448 cases, Gemma 4 12B, no retraining). Task columns give accuracy.

MethodMean task accuracy ↑Disposition ↑Primary diagnosis ↑Direct disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Delegation0.5160.6850.34793.5%78.4%98.0%
PAPILLON0.4820.6810.28419.4%6.1%24.0%
PrivMeSA (ours)0.4970.6790.3151.8%1.1%15.0%
  • PrivMeSA still has the lowest disclosure and re-identification at a new hospital, without retraining.
  • Re-identification is higher than on MIMIC for every method, because the registry is a third the size (70,545 vs 205,504 patients).
  • PrivMeSA's mean accuracy is within 2 points of Delegation and above PAPILLON.
AuthorsOpenReview ↗

Response to Reviewer YbLd (part 2 of 2)

W2. Detecting private information expressed in different wording

Re-identification

Re-identification is measured on values rather than wording. The LLM attacker reads each stated quasi-identifier and records its value, so different phrasings of the same fact give the same match. For example, "64 y/o gentleman, arrived via EMS" and "male, 64, brought by ambulance" both become age 64, male, ambulance.

Direct disclosure

In the paper, direct disclosure used verbatim matching. We have added a check by meaning. An LLM lists every personal detail in the messages. A judge then decides whether each one matches an injected detail, allowing synonyms, spelling variants and paraphrases.

Table R3: Direct disclosure (any leak), verbatim as in the paper, and with the new check, which counts details found verbatim or by meaning.

MethodGemma, verbatim (paper) ↓Gemma, by meaning ↓Granite, verbatim (paper) ↓Granite, by meaning ↓
Delegation94.3%94.3%98.0%98.0%
PAPILLON21.2%25.4%79.9%80.4%
PrivMeSA (ours)0.9%0.9%0.2%0.2%
  • PrivMeSA's rate does not change with the new check.
  • Only PAPILLON leaks more by meaning, because its rewriter often rewords the details it sends.

Q1. Why not enforce privacy at the network or infrastructure layer?

Thank you for this question. Network-layer protection is indeed valuable and can be combined with PrivMeSA, but it addresses a different threat.

  • Network-layer measures such as encryption and access control protect messages from third parties. They do not hide the content from the remote model and its provider, who must read a message to answer it.
  • In our paper, the threat model is this LLM provider, so what to send must be decided before the message leaves the hospital.
  • On MIMIC, withholding identifying details does not lower accuracy. PrivMeSA is more accurate than Delegation, which shares personal details in 94.3% of cases (0.592 vs 0.564, Table 1), and than the same system trained for accuracy only in a new run (0.567).

We hope these results address your concerns, and we are glad to answer any further questions.

Thank you for the authors’ response. However, after reviewing the responses and the comments from the other reviewers, I choose to maintain my original score.

Thank you for reading our replies. We have uploaded a revised paper, with all new results from the discussion in blue. We also have some new updates relevant to your points that you may find interesting, all included in the revised paper.

W1. Dynamic Cheatsheet re-identification by quarter

Our reply compared Dynamic Cheatsheet with our memory over the whole test stream. To show how each memory develops, we now also report re-identification by quarter of the stream.

Table Y1: Re-identification (linked visits) as median k by quarter of the 897-case test stream (Gemma 4 12B). Higher k is more private.

MemoryQ1 ↑Q2 ↑Q3 ↑Q4 ↑
Dynamic Cheatsheet136119244
PrivMeSA memory (ours)3,3633,5963,5483,584

With Dynamic Cheatsheet, a typical patient matches only 13 to 192 registry patients. With our memory, a typical patient matches more than 3,300 in every quarter.

W1. Memory in new patient orders

A growing memory may depend on the order in which patients arrive. To check this, we reran our memory evaluation in two new random patient orders.

Table Y2: Our memory in three random patient orders (Gemma 4 12B, 897-case test stream). Reduction is against an empty memory (7.03 messages per case).

Patient orderMean task accuracy ↑Messages per case ↓Reduction in messages ↑
1 (paper)0.5915.9615.2%
20.5975.8516.7%
30.5915.8616.6%

Accuracy changes by at most 0.6 points, and the memory reduces messages by 15% to 17% in every order.

W1. Accuracy of memory lessons

Self-evolution only helps if the lessons it stores are reliable. To check this, we compared each answer taken from a lesson with asking a remote specialist for the same task.

Table Y3: Accuracy of answers taken from a memory lesson, against asking a remote specialist (Gemma 4 12B, 897-case test stream).

Lesson originShare of the 384 answersAccuracy from the lesson ↑Accuracy asking a remote specialist ↑Difference (points)
All100%0.5910.612−2.1
Same patient, earlier visit17%0.5000.516−1.6
Another patient76%0.6080.632−2.4

Answering from a lesson is only 2.1 points less accurate than asking a specialist, and it sends no message. Lessons from other patients are about as reliable as lessons from the same patient's earlier visits (Appendix E.6).

W1. Claude Haiku 5.5 as the remote models

Our evaluation used GPT-5.6-Luna as the remote models. To check that the results do not depend on this model, we replaced it with Claude Haiku 5.5, without retraining.

Table Y4: Results with Claude Haiku 5.5 as the remote models (Gemma 4 12B, 453 cases).

MethodMean task accuracy ↑Direct disclosure (any leak) ↓Re-identification: k ≤ 10 ↓Re-identification (linked visits): k ≤ 10 ↓
Delegation0.54288.7%44.4%91.0%
PAPILLON0.49319.9%1.8%11.0%
PrivMeSA (ours)0.5531.1%0.0%1.0%

PrivMeSA stays the most accurate and the most private method. Its Direct disclosure barely changes (1.1% vs 0.9% with GPT-5.6-Luna) (Appendix E.7).


Thank you again for your time and helpful feedback. If these updates address your concerns, we would be grateful if you would consider raising your score. If anything still keeps the score at its current level, or if you have any further questions, please let us know and we will try our best to answer and address them in the remaining time.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of New Results and Revisions

We thank the AC and the three reviewers for their careful reviews. The reviewers found the clinical setting relevant (fvK8), the paper clearly structured (YbLd), and the multi-turn, linked-visit evaluation more realistic (WxRH). They also noted that PrivMeSA's accuracy does not come at a cost in privacy (fvK8).

After our replies, Reviewer fvK8 wrote: "All my concerns are fully addressed." Reviewer YbLd thanked us for the response and did not raise further concerns. Reviewer WxRH did not comment further, and their rating remains positive.

We ran the analyses the reviewers requested, and all new results support the paper's claims. We have uploaded a revised PDF with the new results in blue.

New results

Accuracy-only GRPO ablation (WxRH, fvK8)

  • Without the privacy terms, the policy has Direct disclosure in 90.3% of cases and 92.0% of patients re-identified when visits are linked, against 0.9% and 0% for PrivMeSA. This shows that PrivMeSA's privacy comes from the privacy terms of the reward.
  • The privacy terms do not reduce accuracy. PrivMeSA's mean task accuracy is 0.592, against 0.567 for the accuracy-only policy. (Table 4, App. E.2)

Dynamic Cheatsheet as a self-evolving baseline (YbLd)

  • Its mean task accuracy is similar to our memory's (0.584 vs 0.591), but it has Direct disclosure in 16.8% of cases, against 0.7% with our memory.
  • Only our memory reduces the messages to the remote specialists, by 15%. (Table 3, App. E.3)

Second dataset: MC-MED (YbLd, WxRH)

  • Without retraining, PrivMeSA has Direct disclosure in 1.8% of cases, against 93.5% for Delegation and 19.4% for PAPILLON. Only 1.1% of its visits are re-identified at k ≤ 10, against 78.4% and 6.1%.
  • PrivMeSA's mean task accuracy (0.497) is above PAPILLON's (0.482) and similar to Delegation's (0.516). (App. A.4)

Robustness to the attacker protocol (WxRH)

  • PrivMeSA has the lowest Re-identification under every treatment of excluded visits. Even when every excluded visit counts as re-identified, only 21.0% of its visits are re-identified, against at least 37.1% for the baselines. (App. B.4)
  • The ranking of the methods by Re-identification is the same at every threshold from k ≤ 1 to k ≤ 100. (App. B.5)

Specialist roles ablations and token use (fvK8)

  • Removing the diagnostician or the clinician lowers mean task accuracy by 3.0 and 2.3 points.
  • PrivMeSA uses the fewest tokens per case (1,466, against 2,081 for Delegation and 3,822 for PAPILLON) and sends only 104 tokens per case to the remote specialists. (Sec. 4.2, App. E.4)

Patient order for the growing memory (fvK8)

  • We reran the growing memory on the same 897 test cases in two new random patient orders. Mean task accuracy changes by at most 0.6 points.
  • The memory reduces the messages to the remote specialists by 15% to 17% in every order, against an empty memory. (App. E.5)

Quality of memory lessons (WxRH)

  • When the memory offers a lesson, the policy answers the task from it 92% of the time instead of asking a remote specialist.
  • Lessons from other patients are about as reliable as lessons from the same patient, so there is no negative transfer. (App. E.6)

Other remote models and unseen requests (WxRH)

  • With Claude Haiku 5.5 as the remote models, PrivMeSA has the highest mean task accuracy (0.553) and Direct disclosure in only 1.1% of cases, the fewest of all methods.
  • PrivMeSA declined all 1,096 follow-up requests with reasons not seen in training and disclosed nothing. (App. E.7)

PAPILLON with the three remote specialists (fvK8)

  • We ran PAPILLON's released prompts with the same three remote specialists, rewriting every message sent to each of them.
  • PrivMeSA has higher mean task accuracy (0.592 vs 0.538), Direct disclosure in fewer cases (0.9% vs 29.1%), and fewer patients re-identified when visits are linked (0% vs 2.0%).

Related work (fvK8)

  • The related work now discusses Zhang et al. (AAAI 2026), which sends the remote model a synthesized query on a different topic with the same reasoning structure. (Sec. 2)
  • This approach does not apply to clinical consultation, where the topic is the patient's condition itself.

Direct disclosure judged by meaning (YbLd)

  • We added a check that also counts personal details sent in different wording, such as synonyms and paraphrases.
  • With this check, PrivMeSA still has Direct disclosure in only 0.9% of cases with Gemma and 0.2% with Granite. PAPILLON has Direct disclosure in more cases (25.4% instead of 21.2%), because its rewriter often rewords the details it sends.

Together, these results show that PrivMeSA's privacy holds at a second hospital, with other remote models, under unseen requests and under stricter attacker protocols, and that the privacy terms of the reward do not reduce accuracy.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.