acceptodds
Under review as a conference paper at ICLR 2027

Don't Call It Privacy Until You Pick a Metric: Joint Auditing of Fine-Tuning and Privacy Metrics in Clinical LLMs

Abstract

Fine-tuning open-weight LLMs on clinical data improves task performance, but it can also leak sensitive information from the training data. There is no single agreed way to measure this leakage. This raises a practical question: for a given privacy budget, does full-parameter or parameter-efficient (QLoRA) fine-tuning give better utility? Answering this question is complicated by the fact that privacy itself has no single agreed-upon measurement, and three attack families have each been invoked in the literature to justify when a clinical LLM is safe, namely 1) verbatim extraction, 2) prompt-based identity probing, and 3) membership inference. We posit that these problems are not separable. The verdict on which fine-tuning regime supports the greatest amount of privacy depends on which attack one is concerned about. In this paper, we provide the first benchmark that varies both objectives jointly. We fine-tune six open-weight models on four clinical tasks. Each model is trained under regimes: full-parameter and QLoRA, each with two loss variants. We evaluate every resulting model with all three attack families and with task-specific utility metrics. We additionally study differentially private fine-tuning on a focused slice of the grid as a case study of how a privacy-preserving training method behaves with respect to various privacy attacks. In total, our study spans over 250 fine-tuned model configurations and more than 1,000 evaluation runs, including multi-seed replication, hyperparameter sensitivity sweeps, and decoding-controlled ablations. Our investigation yields two primary findings: (i) Privacy metrics can disagree, and the way they disagree is itself task-dependent e.g., verbatim extraction risk and membership-inference risk show negative correlations on one of our four tasks, with weaker or inconsistent relationships on the others and (ii) the fine-tuning regime interacts with these privacy metrics in a nontrivial, task-dependent manner. In other words, a fine-tuning regime may reduce privacy risk under one metric while increasing it under another. Privacy outcomes therefore depend on the specific combination of fine-tuning regime, attack, and clinical task. We release all code for a public benchmark that supports systematic evaluation of both privacy and utility in clinical LLM fine-tuning across multiple attacks and tasks.

Primary area:
datasets and benchmarks
Keywords:
privacy evaluation, clinical LLMs, memorization, membership inference, differential privacy, fine-tuning, benchmark

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_BBZ2OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies how different fine-tuning strategies affect utility and privacy leakage in clinical LLMs. The paper evaluates six open-weight models across four clinical tasks, comparing full-parameter fine-tuning and QLoRA under full-input and masked-input loss. Privacy is evaluated using three attack families: exact memorization, prompt-based identity attacks, and membership inference. The study also includes differentially private fine-tuning, hyperparameter sensitivity, and multi-seed experiments. The main finding is that privacy metrics do not consistently agree across tasks and configurations, and the preferred fine-tuning strategy depends on the privacy metric used.

Critical strengths
  1. Broad experimental evaluation: The paper evaluates six models across four clinical tasks using multiple fine-tuning strategies and privacy attacks. The inclusion of hyperparameter sensitivity, multiple seeds, and DP fine-tuning strengthens the empirical contribution and provides a useful basis for comparing privacy risks.
  2. Metric-dependent privacy assessment: The paper highlights a practical limitation of evaluating privacy using a single attack or metric. The analysis of utility-privacy trade-offs and changes in preferred fine-tuning configurations provides useful evidence that privacy evaluations should consider multiple attack families.
Critical weaknesses
  1. Validity of the privacy attack measurements: Some prompt-based attacks may not fully distinguish memorization from general prediction ability. For example, the MedQA attack retains the final medical question and answer options, so correct predictions may reflect general medical knowledge rather than identity-specific information leakage. Although the paper includes additional controls for ICD prediction, the interpretation of prompt-attack accuracy as a direct privacy-risk measure remains unclear. This matters because these measurements are used to compare fine-tuning configurations.
  2. Statistical support and sensitivity to attack implementation: Table 1 shows that only one of the 12 canonical cross-protocol correlations remains statistically significant after multiple-testing correction and the confidence-interval criterion. While this is consistent with a lack of a universal relationship, it does not necessarily establish reliable disagreement between the metrics. Furthermore, the stronger EZ-MIA attack detects membership information that the primary Random Forest attack often misses. Some masked-loss EZ-MIA results also have ROC-AUC values significantly below 0.5, which may indicate membership information in the reverse direction. These issues limit confidence in the reported privacy rankings and their interpretation.
Questions for authors
  1. Can the authors clarify how the prompt-based identity attacks distinguish patient-specific memorization from general task knowledge? For MedQA, would comparing predictions with shuffled or removed patient identifiers help isolate identity-dependent leakage?
  2. Given the substantial differences between RF-MIA and EZ-MIA, how stable are the main cross-metric conclusions when using the stronger attack? Also, have the authors considered reversing membership scores when ROC-AUC is significantly below 0.5?
  3. Can the authors quantify the statistical reliability of the preferred fine-tuning configurations? In particular, would the selected configurations remain the same after accounting for uncertainty in the utility and privacy measurements?
4 comments
AuthorsOpenReview ↗

Response to reviewer BBZ2's W1/Q1 and Q3

We thank the reviewer for their thoughtful feedback and constructive suggestions. We conducted additional experiments and statistical analysis to address the comments below.

W1/Q1:

For mortality and readmission, the attack prompt contains only the patient identifier, so no task knowledge is available to the model. MedQA however, keeps the final question sentence. So, we ran control experiments: we removed the question sentence and kept only the synthetic patient name and the answer options, so nothing in the prompt can be answered from medical knowledge.

MedQA PA accuracy with the question sentence removed (patient name and options only).

BackboneBaseQLoRA, fullQLoRA, maskedLift (masked − base)
Llama-3-8B-Instruct0.2910.3220.388+0.097
MedLLaMA-8B0.2850.2780.388+0.103
Meditron-7B0.2710.2900.364+0.093
Qwen3-0.6B0.2750.2740.342+0.067
Qwen3-1.7B0.3140.2920.360+0.046
Qwen3-4B0.2640.2740.358+0.094
  • All six base models fall to chance (0.26 to 0.31). Without a fine-tuned identity association, the prompt carries no answerable content.
  • Masked-loss fine-tuning lifts accuracy above base on every backbone (+0.05 to +0.10). Since the question is absent, this lift reflects patient-specific memorization rather than task knowledge.
  • The configuration ordering is preserved. For Llama-3-8B, QLoRA-masked is the most vulnerable configuration under both the original prompt (0.510) and the identity-only prompt (0.388).

Spearman ρ between EM/MIA metrics and MedQA PA accuracy under both prompts.

ProtocolMetric vs. PA accuracyOriginal PA (rho, p_perm, q_BH,10)Name + options PA (rho, p_perm, q_BH,10)
EMMem@10-0.272, 0.061, 0.202-0.098, 0.710, 0.789
EMROUGE-L-0.328, 0.013, 0.065-0.295, 0.224, 0.374
EMBERTScore-0.234, 0.009, 0.065-0.358, 0.137, 0.274
MIAROC-AUC+0.154, 0.387, 0.553-0.246, 0.547, 0.684
MIATPR@5%FPR+0.251, 0.119, 0.274-0.026, 0.906, 0.906

Because the PA measurements are used to compare fine-tuning configurations, we also checked whether the cross-protocol relationships change under different PA methods. As shown in the table above, they do not: all correlations remain weak and non-significant under both formulations.

We will add the identity-only to MedQA PA and discuss a shuffled-identifier variant as a further check.

Q3:

We thank the reviewer for highlighting the uncertainty of measurements of privacy and utility in selecting preferred fine-tuning configurations. This has also motivated our multi-seed experiments already included in Section 4.2 and Appendix G. We further quantify how such run-to-run variability affects preferred model selection here.

  • Method. Using four Llama-3/MedQA training seeds, we exhaustively evaluated all 44=2564^4=256 paired-seed bootstrap resamples. For each draw, we recomputed utility, privacy risk, min–max normalization, and the original equal-weight TOPSIS-style score (same weights for privacy and utility as example). And we show the bootstrap winner with the highest win frequency below.

  • Selection reliability.

Privacy metricWinner with seed=42 (main body)Four-seed bootstrap winnerWin frequencyScore Δ [95% CI]
EM Mem@10Q-maskedF-masked99.6%+.050 [.005, .260]
PA accuracyF-fullBase72.3%+.026 [-.270, .226]
RF-MIA AUCF-maskedF-masked100%+.671 [.554, .712]

The RF-MIA winner is reproduced, whereas the EM winner changes and the original PA winner (F-full) wins only 27.7% of resamples. Thus, not all nominal winners are stable in view of the variation/uncertainty of measurements. However, EM vs. PA and PA vs. RF-MIA select different winners in all 256 paired resamples, supporting persistent cross-family selection differences despite measurement variability.

  • Raw trade-off stability. Across four seeds, F-masked improves utility (+.0253±.0095) and reduces Mem@10 (-.1835±.0034) and RF-MIA AUC (-.2002±.1063) relative to F-full, but increases PA accuracy (+.0530±.0155). These differences show that the same training change can affect privacy metrics in opposite directions, which supports our conclusion that privacy metrics measurements could not be interchanged.

  • Implications. In clinical LLM auditing, uncertainty across training runs such as training seeds can affect which configuration appears preferable, making it important to report selection variability alongside utility and privacy measurements. We will extend our multi-seed analysis to discuss this uncertainty and distinguish stable from uncertain selections. These results further support our central finding that privacy assessments vary across metrics.

AuthorsOpenReview ↗

Response to reviewer BBZ2's W2/Q2

W2/Q2:

We address the statistical support for disagreement first, and then the differences between RF-MIA and EZ-MIA, including below-chance AUCs.

  • Statistical support We agree, and we do not claim a reliable negative correlation between metrics. By disagreement we mean that a fine-tuning configuration can be robust to one attack family and vulnerable to another. What our claim requires is that no pair shows a reliable positive relationship, so that no attack can stand in for another.

  • Stability of main conclusions between different MIA attack implementations We report cross-protocol correlation analysis using both RF-MIA and EZ-MIA. Section 4.2.1 summarizes the EZ-MIA robustness analysis, with detailed results provided in Appendix F.1 (Figures 30–33 and Tables 9–10). While the magnitude of MIA-involving correlations vary between RF-MIA and EZ-MIA, both analysis support our central finding: cross-protocol relationships are sensitive to the choice of attack implementation and do not exhibit consistent alignment.

We further assess the sensitivity of cross-protocol correlations to MIA implementation by comparing RF-MIA and EZ-MIA ROC-AUC across the full fine-tuning grid. As shown below, correlations vary in magnitude and sometimes reverse direction, indicating that the observed relationships depend on how membership inference is operationalized, which is not inherently identical.

ROC-AUC cross-family correlation sensitivity over the complete fine-tuning grid. 95% CIs are computed from 10,000 backbone cluster bootstrap; Δρ = ρ(EZ) − ρ(RF).

TaskComparatorρ(RF) [95% CI]ρ(EZ) [95% CI]Δρ [95% CI]
MedQAEM Mem@10+0.284 [-0.238, +0.684]+0.724 [+0.600, +0.817]+0.440 [-0.021, +1.036]
MedQAPA Accuracy+0.154 [-0.246, +0.573]-0.318 [-0.567, -0.107]-0.472 [-0.921, -0.031]
MedQAPA F1+0.242 [-0.164, +0.590]-0.248 [-0.476, -0.056]-0.490 [-0.902, -0.067]
MortalityEM Mem@10+0.006 [-0.443, +0.598]+0.049 [-0.204, +0.199]+0.043 [-0.772, +0.641]
MortalityPA Accuracy-0.299 [-0.765, +0.273]+0.128 [-0.203, +0.457]+0.427 [-0.354, +1.160]
MortalityPA F1-0.319 [-0.686, +0.148]+0.315 [+0.003, +0.623]+0.634 [-0.005, +1.209]
ReadmissionEM Mem@10-0.324 [-0.620, +0.013]+0.394 [+0.054, +0.719]+0.718 [+0.124, +1.283]
ReadmissionPA Accuracy-0.215 [-0.616, +0.038]+0.283 [+0.016, +0.712]+0.497 [+0.057, +1.054]
ReadmissionPA F1-0.125 [-0.545, +0.283]+0.293 [-0.018, +0.732]+0.418 [-0.048, +0.979]
ICDEM Mem@10+0.417 [-0.052, +0.733]+0.675 [+0.503, +0.851]+0.259 [-0.199, +0.922]
ICDPA Accuracy+0.092 [-0.296, +0.440]-0.285 [-0.719, +0.130]-0.377 [-1.133, +0.304]
  • ROC-AUC below 0.5 We agree that a ROC-AUC below 0.5 may indicate reversed direction membership information. We identified the cases and reversed the EZ-MIA ROC-AUC scores. Below is the result for the MedQA task.

Raw and Reversed ROC-AUC

TaskModelFine-tuningLossRaw AUCAUC*
MedQALlama-3Full-FTmasked0.4790.521
MedQAMedLLaMAFull-FTmasked0.4750.525
MedQAQwen3-0.6BFull-FTmasked0.4590.541
MedQAQwen3-0.6BQLoRAfull0.4880.512
MedQAQwen3-0.6BQLoRAmasked0.4940.506
MedQAQwen3-1.7BFull-FTmasked0.4180.582
MedQAQwen3-1.7BQLoRAmasked0.4910.509
MedQAQwen3-4BFull-FTmasked0.4730.527

Moreover, we recomputed the cross-protocol Spearman correlations after reversing the EZ-MIA ROC-AUC scores. As shown below, all correlation signs remain unchanged, although their magnitudes vary.

Effect of reversing the MIA scores on original correlation

TaskComparatorρ(EZ-raw)ρ(EZ-AUC*)Δρ [95% CI]Sign outcome
MedQAEM Mem@10+0.724+0.481-0.243 [-0.333, -0.119]Same
MedQAPA Accuracy-0.318-0.249+0.069 [-0.318, +0.285]Same
MedQAPA F1-0.248-0.203+0.045 [-0.305, +0.249]Same
MortalityEM Mem@10+0.049+0.042-0.007 [-0.391, +0.287]Same
MortalityPA Accuracy+0.128+0.297+0.169 [-0.087, +0.316]Same
MortalityPA F1+0.315+0.399+0.084 [-0.241, +0.382]Same
ReadmissionEM Mem@10+0.394+0.294-0.100 [-0.620, +0.213]Same
ReadmissionPA Accuracy+0.283+0.001-0.282 [-0.761, +0.000]Same
ReadmissionPA F1+0.293+0.026-0.267 [-0.741, +0.072]Same
ICDEM Mem@10+0.675+0.630-0.046 [-0.185, +0.138]Same
ICDPA Accuracy-0.285-0.534-0.248 [-0.394, -0.018]Same

Thank you for the responses and additional experiments. The MedQA control experiments help address my concern about distinguishing general task knowledge from identity-specific leakage, although a shuffled-identifier control would provide further evidence. The additional EZ-MIA analysis and bootstrap results are also helpful, particularly in clarifying the sensitivity of privacy measurements and the stability of model selection. However, some EZ-MIA correlations remain strongly positive, so I would suggest qualifying the broader claim about the absence of reliable relationships between privacy metrics.

Overall, the responses address several of my concerns and provide useful additional evidence. I appreciate the authors' efforts and will maintain my original rating.

We are glad that our MedQA control experiments, EZ-MIA analysis, and uncertainty analysis with bootstrap results have helped address several of the reviewer's concerns. We appreciate the constructive follow-up and helpful suggestions. We conducted the additional shuffled-identifier experiments and clarification of EZ-MIA correlations as below.

1. MedQA Shuffled-Identifier Control

We conducted this experiment across all six backbones using the original MedQA prompt, replacing each patient's name with another patient's name in the training set while keeping the question and options unchanged. We also include our earlier question-removal results, which retain only the patient name and answer options for comparison.

MedQA PA accuracy under three prompt formulations. Standard uses the original question and patient name; Shuffled names replaces the patient name while retaining the question and options; Question removed only retains the original patient name and answer options.

BackboneConfigurationStandardShuffled namesQuestion removed only
Llama-3-8BBase0.3960.4000.291
QLoRA full0.4880.4880.322
QLoRA masked0.5120.5360.388
MedLLaMABase0.1960.2200.285
QLoRA full0.2760.2600.278
QLoRA masked0.2720.2800.388
Meditron-7BBase0.2080.2160.271
QLoRA full0.2760.2880.290
QLoRA masked0.2880.3080.364
Qwen3-4BBase0.1960.2120.264
QLoRA full0.2800.2720.274
QLoRA masked0.4320.4360.358
Qwen3-1.7BBase0.1960.1880.314
QLoRA full0.3360.3440.292
QLoRA masked0.3680.3840.360
Qwen3-0.6BBase0.2480.2640.275
QLoRA full0.2640.2560.274
QLoRA masked0.3680.3600.342

Shuffling names has little effect on accuracy when the clinical question is retained (all absolute differences from standard ≤ 0.024), indicating that the remain clinical question senetece contribute substantially to PA performance potentially by allowing the model to draw on its original medical knowledge reasoning capabilities. After removing the question, all six base models perform near chance (0.264–0.314), while QLoRA-masked consistently outperforms Base (+0.046 to +0.103).

This supports the question-removed formulation from our earlier response (patient name and options only). In the camera-ready version, we will adopt the question-removed formulation as the primary MedQA prompt attack, apply the shuffled-identifier control to it as well, and report both controls in Appendix C.2. Our earlier correlation analysis also shows that the cross-protocol associations remain inconclusive under both prompt formulations, consistent with one of our broader findings that privacy assessments vary across metrics.

2. EZ-MIA Correlation Clarification

We thank the reviewer for pointing this out. We agree that some EZ-MIA correlations are significantly positive. However, these associations are limited to specific tasks and metric pairs as shown below.

Significant positive cross-protocol correlations involving EZ-MIA (Figures 30–33).

TaskPrivacy metric pairSpearman ρBH-adjusted significance
MedQAEM Mem@10 vs. EZ-MIA AUC+0.72q < 0.001
MedQAEM ROUGE-L vs. EZ-MIA AUC+0.62q < 0.05
MedQAEM BLEU vs. EZ-MIA AUC+0.65q < 0.01
ICDEM Mem@10 vs. EZ-MIA AUC+0.68q < 0.01
ICDEM BLEU vs. EZ-MIA AUC+0.62q < 0.05
ICDEM BERTScore vs. EZ-MIA AUC+0.53q < 0.05

Only 6 of 60 cross-protocol EZ-MIA comparisons show significant positive associations after BH correction. All six are between EM and EZ-MIA AUC on MedQA or ICD. No positive associations survive correction on mortality or readmission, or between PA and EZ-MIA across any task. Likewise, none of the 12 canonical RF-MIA cross-protocol comparisons in Table 1 shows a significant positive association.

While certain privacy metrics show significant positive correlations in specific settings in EZ-MIA cases, these relationships vary across clinical tasks and attack implementations. This is consistent with our central finding that privacy assessments are not consistently aligned across different privacy protocols. We will expand and revise this EZ-MIA correlation analysis and clarify this distinction in the camera-ready version.

These results do not contradict our central finding. While certain privacy metrics are positively correlated in specific settings, their relationships vary substantially across clinical tasks and attack implementations. Thus, our conclusion that different privacy attacks don't provide interchangeable assessments holds. We will expand and revise the EZ-MIA correlation discussion and analysis in the camera-ready version and clarify this distinction in conclusion.

Reviewer_JTMMOpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The authors first claim that privacy lacks a universally accepted quantitative metric, and note that the literature has instead operationalized privacy through three distinct families of attacks: 1) verbatim extraction, 2) prompt-based identity probing, and 3) membership inference. They subsequently present a benchmark that, to their knowledge, is the first to jointly vary both objectives. Six models were fine-tuned on four different clinical tasks, under full-parameter and QLoRA.Based on extensive experimental evaluation, the authors derive two principal findings. First, privacy metrics exhibit task dependence. Second, the fine-tuning regime may decrease privacy risk under one metric while concomitantly increasing it under another.

Critical strengths
  1. The authors address a timely and important topic. Given the EU AI Act’s phased enforcement schedule, the relevant timing renders this matter immediately operational rather than solely of academic interest. The authors present a comprehensive literature review that systematically contextualizes their study within the broader research domains of privacy auditing, clinical NLP, and regulatory compliance. This comprehensive review of related work not only situates the proposed framework within the existing literature but also exposes critical gaps in existing evaluation methodologies, thereby motivating the paper’s multi-attack, multi-regime design.

  2. The study is distinguished by the breadth of its empirical evaluation: it systematically assesses four fine-tuning regimes in combination with three distinct attack families across multiple clinical large language models (LLaMA-3-8B, Qwen3 variants, MedLLaMA-3, and Meditron) using the MIMIC-IV dataset. The breadth of the evaluation—encompassing the complete factorial set of model, training-regime, and attack-condition combinations—demonstrates the methodological rigor in privacy auditing and materially enhances the evidential strength and credibility of the resulting conclusions.

  3. Overall, the paper is clearly written and the experimental results are analyzed in a systematic and well-organized manner. By applying the TOPSIS multi-criteria decision-making method and delineating the privacy–utility Pareto frontier across all regime–model combinations, this approach transforms a complex, high-dimensional results table into empirically grounded and operationally actionable deployment guidance. Section 5 presents a cogent and evidence-driven discussion that enriches the reader’s understanding of the experimental findings.

Critical weaknesses

The writing would benefit from a thorough proofreading pass, as some minor errors are present throughout the manuscript.

  1. Although the authors claim in the abstract that the source code has been made publicly available, no repository link or accompanying code was found, which hinders reproducibility. An anonymous repository link could have been provided to address this concern while maintaining the double-blind review process.

  2. Greater care should be taken in the writing of each sentence. For instance, in lines 250–251, the authors state that a full hyperparameter sweep is reported in Appendix D.1; however, the figures therein only compare learning rates, falling short of the comprehensive analysis implied.

  3. This is not critical, but can elevate the quality of the manuscript. Some references appear to be missing throughout the manuscript. When a task or concept is originally introduced in prior work, the corresponding citation should be included to aid reader comprehension. (minor) QLoRA is used without a citation.

  4. Some minor errors are provided below.

  • In line 485, a closing quotation mark is incorrectly used in place of an opening quotation mark before safest, which should be corrected.
  • The resolution of Figure 6 appears insufficient and should be improved for clarity.
  • For consistency within the paragraph, the authors are encouraged to specify the model size of Meditron (e.g., 7B or 70B) as done for other models.
Questions for authors
  1. Where can I find the LiRA-style auditing convention in line 1133 from [1] ?

  2. How practically significant are the reported ΔAcc values in real-world clinical settings? Additionally, a more discussion is needed to clarify the conditions under which ΔAcc becomes negative. Finally, how competitive are the fine-tuned models relative to the latest state-of-the-art models, if any ?

  3. The dataset split for the 15-day readmission prediction task is stated to be provided in Appendix B; however, it could not be located. Could the authors clarify where this information appears? Is it divided chronologically?

[1] Carlini, Nicholas, et al. "Membership inference attacks from first principles." 2022 IEEE symposium on security and privacy (SP). IEEE, 2022.

6 comments
AuthorsOpenReview ↗

Response to Reviewer JTMM: Reproducibility and Clarifications

We thank the reviewer for recognizing the timeliness of the topic, the breadth of our evaluation, and the actionable guidance. We address the comments on reproducibility and presentation, and the questions on the MIA convention, utility gains, and data splits, point by point below.

  • W1: We will include the code link in the paper, update the manuscript's code availability and reproducibility statements. Code link: https://anonymous.4open.science/r/llm-privacy-audit-2029/README.md
  • W2: Hyperparameter study (lines 250-251).
    • The study varies more than the learning rate, but you are right that “full hyperparameter sweep” overstates it, and we will call it a "hyperparameter sensitivity study".
    • For QLoRA, we vary the learning rate, the LoRA rank r ({16, 32, 64} for MedQA; {16, 32} for mortality and readmission), and the scaling factor (alpha) in 2r, 4r. For full-parameter fine-tuning, we vary the learning rate. Every setting is trained under both full-input and masked-input loss. The rank and alpha of each run are reported in Tables 4, 6, and 8.
    • In Figures 11, 13, and 15, color encodes the learning rate, and rank and appear in the point labels. That is why the figures look like a learning-rate comparison.
    • We will revise the wording in Section 4.1 and Appendix D.1, label every point in the figures with its full (lr, r, alpha) setting, and list the explored combinations explicitly.
  • W3: Missing references: We will add the appropriate references for QLoRA (Dettmers et al., 2023 [1]), DP-SGD (Abadi et al., 2016 [2]), and review the manuscript for other missing citations.
  • W4: Presentation and formatting corrections: We will correct the quotation mark preceding "safest" in line 485, improve the resolution of Figure 6, and consistently identify the evaluated Meditron model as Meditron-7B. We will also conduct a proofreading pass to correct typographical and formatting inconsistencies throughout the manuscript.
  • Q1:
    • LiRA-style convention (line 1133). We use the evaluation convention of Carlini et al.,2022 [4], not the LiRA attack itself.
    • Carlini et al. argue that membership-inference attacks should be judged by the true-positive rate at a low false-positive.
    • We follow that recommendation, we use membership metrics TPR at 5% FPR, and we also report TPR at 1% FPR.
    • Our attack is a Random Forest over four likelihood-based features (Appendix C.3).
    • As an independent check, we also run EZ-MIA Ilić et al., 2026 [5], which needs no training step for the attacker and no shadow models.
    • A full LiRA evaluation would require shadow models for every backbone and task, which we leave to future work. We will reword line 1133 to name the convention.
  • Q2:
    • Significance. Fine-tuning gives consistent benchmark gains:
      • Utility improves in 77 of 96 backbone-task-configuration cells.
      • The best configuration per task gains 6.8-11.7 points on average (Fig. 17).
      • On MedQA, masked-loss QLoRA gains +9.6 points (95% CI [6.6, 13.1]).
      • A masked-loss configuration gives the largest gain in 20 of 24 backbone-task pairs.
      • Clinical benefit itself requires validation beyond benchmarks; as Singhal et al.,2025 [3] note, medical QA benchmarks do not capture the factuality, safety, and complexity of real clinical interactions. Within this scope, we aim to give practitioners a complete and transparent picture: every configuration is reported with its raw utility alongside all three privacy attacks, so utility gains are never judged without their privacy cost.
    • Negative ΔAcc. Regressions are concentrated in full-parameter fine-tuning on the binary MIMIC-IV tasks (mortality, readmission), most notably MedLLaMA on readmission and Qwen3-0.6B/1.7B on mortality. Masked-loss QLoRA regresses in only 2 of 24 cells. We keep these cases because our goal is to evaluate privacy and utility jointly
    • SOTA. Our best MedQA model (70.1%) is below Med-PaLM 2 (86.5%; Singhal et al., 2025 [3]), a far larger proprietary model. We study 0.6B-8B open-weight models that hospitals can fine-tune on-site, and our claims concern the relative ranking of configurations. -We will add this analysis to Appendix D.2 and Section 4.1.
  • Q3:
    • The general protocol is in Appendix I (Dataset Splits): an 80/10/10 train/validation/test split wherever dataset size permits.
    • For 15-day readmission, the data were split randomly, not chronologically. A random split is used to draw training members and held-out non-members from the same underlying data distribution, minimizing distribution-shift confounding in our membership-inference evaluation. This follows the standard same-distribution MIA evaluation setting, where the objective is to distinguish samples based on training membership rather than distributional differences. (Section 3.3).
    • We will state the random split in Appendix B and add a cross-reference to Appendix I.

References:

  • [1] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient Finetuning of Quantized LLMs. NeurIPS 2023. arXiv:2305.14314.
  • [2] Martín Abadi, Andy Chu, Ian Goodfellow, H. Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. Deep Learning with Differential Privacy. ACM CCS 2016. arXiv:1607.00133.
  • [3] Karan Singhal, Tao Tu, Juraj Gottweis, et al. Toward expert-level medical question answering with large language models. Nature Medicine, 31:943–950, 2025. doi:10.1038/s41591-024-03423-7.
  • [4] Carlini, Nicholas, et al. "Membership inference attacks from first principles." 2022 IEEE symposium on security and privacy (SP). IEEE, 2022 . arXiv:2112.03570.
  • [5] David Ilić, David Stanojević, and Kostadin Cvejoski. Powerful Training-Free Membership Inference Against Fine-Tuned Autoregressive Language Models. ACL 2026 (Long Papers), pages 14077–14093. doi:10.18653/v1/2026.acl-long.640.

Thank you for your thoughtful and detailed responses to the initial reviews. However, upon conducting a side-by-side comparison with [1], a new concern has arisen regarding whether the relationship between this paper and [1] has been sufficiently addressed, particularly in the context of my W3 concern. While [1] is cited in the paper, it appears to be referenced only for minor and general points, rather than for its core contributions that are most directly relevant to this work.

I would like to raise the following points again and kindly ask the authors to provide clear and direct responses:

  1. Potential unattributed use of the attack suite from [1]

The ICD prompt-based identity attack suite (Appendix B.6.1, Figure 7, and the primary protocol in Sec 3.3) appears to be nearly identical to the design presented in [1] — including the attack types (generation, multi-choice, binary, gender attack, and false-condition control) and even the specific prompt wording (see attached figure). However, this section contains no citation to [1], and the paper states “we design multiple complementary attack formulations,” which gives the impression that these formulations were originally conceived by the authors of this paper. Could the authors clarify whether and how this attack suite was developed independently of [1]?

  1. Missing citation in Related Work — Clinical-LLM Privacy

[1] appears to be the most directly relevant prior work in the Clinical-LLM privacy domain, yet it does not appear anywhere in the Related Work section. Could the authors explain this omission?

  1. Insufficient discussion of differentiation from [1]

The differences between this paper and [1] currently appear to be limited to the choice of base models and datasets. This is a significant concern for the paper’s claimed contributions. A more thorough discussion of how this work advances beyond [1] — methodologically, conceptually, or empirically — is needed.

  1. First finding in the Introduction lacks citation to [1]

The first finding highlighted in the Introduction — that privacy metrics from different attacks disagree — appears to have a close precedent in [1]. This claim is currently made without any reference to [1], which raises concerns about the originality of this finding.

As the above points are directly related to the paper’s core contributions and novelty, I look forward to the authors’ candid and specific responses.

[1] Yujin Kang, Eunsun Kim, and Yoon-Sik Cho. "Can Personal Health Information Be Secured in LLM? Privacy Attack and Defense in the Medical Domain." Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security. 2025.

Reviewer_uiReOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper audits privacy leakage in clinical LLM fine-tuning by jointly varying the fine-tuning regime and the privacy attack. Six open-weight models (Llama-3-8B-Instruct, MedLLaMA, Meditron, and Qwen3 at 0.6B/1.7B/4B) are fine-tuned on four clinical tasks (MedQA, ICD prediction, in-hospital mortality, and 15-day readmission from MIMIC-IV) under full-parameter and QLoRA training, each with full-input or masked-input loss, and are evaluated with three attack families: exact memorization, prompt-based identity attacks using synthetically injected patient identifiers, and membership inference. A DP-SGD case study (ε ∈ {1, 2, 10}) is included on a subset. The authors report that privacy metrics from different attack families do not consistently agree, that this pattern varies across tasks, and that the preferred fine-tuning configuration under a utility–privacy trade-off depends on which privacy metric is used.

Critical strengths

S1. Privacy claims about fine-tuned clinical LLMs are often based on a single attack, and evaluating whether different attack families lead to same verdict is a relevant question for practitioners and regulators.

S2. This study spans six LLM backbones, four tasks, four fine-tuning configurations, three attack families, and one DP case study, with additional robustness checks across seeds, hyper-parameters.

S3. Confidence intervals account for backbone-level clustering, permutation tests are applied within each backbone to exclude the impact of difference between backbones. Both rigorous settings make the reported numbers more convincing.

S4. Masked-input loss consistently reduces exact memorisation across seeds, which is a useful practical observation. The code is released for replication.

Critical weaknesses

W1. The paper should clearly define what it means by “disagreement.” The abstract motivates the claim with a negative correlation (mortality EM–MIA, ρ = −0.71), which leads readers to interpret disagreement as contradiction, i.e., two metrics ranking models in opposite orders. However, conclusion describes the metrics as “negatively or near-zero correlated,” which is a much weaker notion. Besides contradiction (a significant negative correlation), a lack of agreement can correspond to two other situations: a negligible association, where the confidence interval lies narrowly around zero, and or an inconclusive result, where the confidence interval is wide enough to include positive, zero, and negative values. Most results in Table 1 appear to fall into the inconclusive category: 11 of the 12 canonical correlations have confidence intervals spanning zero, many of them wide. Such results indicate insufficient evidence rather than disagreement, and the paper should make this distinction explicit to avoid overstating its central claim.

W2. The unadapted Llama-3-8B-Instruct base model, which has never seen these patients, reaches 0.406 PA accuracy (Table 2), well above the 0.25 chance level and comparable to fine-tuned models (0.37–0.51), while its accuracy on the full questions is 0.618. Therefore, it is unclear whether a high PA accuracy reflects genuine identity-linked leakage or simply the model’s inherent ability to reason about and answer the question.

W3. The question posed at the start of the abstract, whether full-parameter or QLoRA fine-tuning gives better utility for a given privacy budget, is never directly answered.

Questions for authors
  1. How do you define “disagreement” between privacy metrics? Could you distinguish, for each canonical pair in Table 1, whether the result reflects contradiction, a negligible association, or inconclusive result?

  2. For MedQA, given that the base model already reaches 0.406 for PA, how does this metric capture identity-linked leakage rather than task ability?

  3. Can you directly answer the question posed in the abstract: for a given privacy budget, does full-parameter or QLoRA fine-tuning yield better utility?

5 comments
AuthorsOpenReview ↗

Response to reviewer uiRe part 1

We thank the reviewer for the careful reading and constructive questions, which we address below.

W1/Q1:

We agree with the reviewer: the term was not clearly defined. We will revise the definition as follows. By disagreement we mean that a privacy verdict does not transfer across attack families: a configuration's risk under one family does not tell you its risk under another. We did not mean disagreement to be read as contradiction.

TaskPairρ95% CIq_BHCategory
MortalityEM–MIA-0.707[-0.862, -0.337]0.005Contradiction
ReadmissionEM–MIA-0.412[-0.731, -0.073]0.212Inconclusive
ICDEM–MIA+0.426[0.015, 0.683]0.147Inconclusive
MedQAEM–PA-0.272[-0.576, 0.162]0.157Inconclusive
MedQAEM–MIA+0.096[-0.375, 0.492]0.740Inconclusive
MedQAPA–MIA+0.251[-0.282, 0.710]0.251Inconclusive
MortalityEM–PA+0.163[-0.370, 0.542]0.619Inconclusive
MortalityPA–MIA-0.156[-0.589, 0.392]0.716Inconclusive
ReadmissionEM–PA+0.305[-0.028, 0.731]0.196Inconclusive
ReadmissionPA–MIA-0.400[-0.734, 0.031]0.131Inconclusive
ICDEM–PA-0.075[-0.571, 0.306]0.831Inconclusive
ICDPA–MIA-0.112[-0.495, 0.180]0.752Inconclusive
Table 1.a: Per-pair classification of the canonical correlations in Table 1

Inconclusive here means the pair fails the paper’s joint criterion (q < 0.05 after global BH over 163 tests, and a cluster CI excluding zero). Two pairs, readmission EM–MIA and ICD EM–MIA, have CIs that exclude zero but do not survive multiplicity correction (q = 0.21 and 0.15).

Inconclusive correlations support this claim as well: substituting one attack for another requires a demonstrated positive relationship between them, and for 11 of the 12 pairs (Table 1.a) no such relationship is established. No pair shows a reliable positive relationship either, so no attack can reliably stand in for another. The ranking-level evidence below shows the same thing directly: the configuration judged safest under one attack is judged least safe under another.

Table 1.b tracks individual configurations across the three canonical metrics. Within each row, the backbone, task, configuration and raw utility are fixed, so a change in rank comes from the privacy metric.

TaskBackboneConfigurationEM Mem@10PA accuracyMIA TPR@5% FPR
MedQALlama-3QLoRA, masked0.955 (best)0.293 (worst)0.608 (2nd)
MedQAQwen3-1.7BFull FT, masked1.000 (best)0.293 (worst)0.789 (best)
MedQALlama-3Full FT, full0.392 (3rd)0.842 (best)0.171 (worst)
MedQAMeditron-7BFull FT, full0.156 (3rd)0.5220.147 (worst)
ReadmissionLlama-3QLoRA, masked0.400 (2nd)0.3900.000 (worst)
ReadmissionMeditron-7BFull FT, masked0.997 (best)0.293 (worst)0.768 (best)

Table 1.b: Equal-weight TOPSIS-style utility–privacy scores (higher is better) under each canonical privacy metric (Tables 11, 13, 15). Ranks are among the four fine-tuned configurations of that backbone and task.

  • The same configuration can be rated best or worst depending on the attack. On MedQA, Llama-3 QLoRA-masked is rated best under exact memorization but worst under the prompt attack. On readmission, Meditron-7B Full FT-masked is rated best under exact memorization and membership inference but worst under the prompt attack.
  • Reversals occur between different pairs of attack families. Exact memorization and the prompt attack reverse (Llama-3 and Qwen3-1.7B on MedQA, Meditron-7B on readmission), and so do the prompt attack and membership inference (Llama-3 and Meditron-7B on MedQA, Llama-3 on readmission). Exact memorization and membership inference are negatively correlated on mortality (Table 1.a).
  • The reversals are not specific to one model or task. They appear across the backbones and tasks, so a single-attack audit can rate a configuration best when another attack rates it worst.
  • Non-transfer does not require a significant correlation. MedQA and readmission have only inconclusive correlations in Table 1.a, yet both show reversals. This is the sense of disagreement in our definition: a verdict reached under one attack cannot be carried over to another.
  • We do not claim the metrics always contradict each other. Our claim is that this agreement cannot be assumed, so the safest configuration must be identified under each attack separately.

We will define disagreement as above and we will update the abstract, Section 4.2.1, and conclusion.

AuthorsOpenReview ↗

Response to reviewer uiRe part 2

W2/Q2:

  • we ran an additional experiment to answer it directly. Unlike the mortality and readmission attacks, which expose only the patient identifier, the MedQA PA prompt keeps the final question sentence. We therefore examined whether this sentence contributes to PA accuracy independently of the identity.
  • The 0.406 base accuracy is specific to Llama-3-8B. Other base models score far lower on the same prompt (Qwen3-4B 0.30, Qwen3-1.7B 0.12; Figures 25 to 29). Llama-3-8B-Instruct is also the strongest base model on MedQA (0.618 on the full question), which suggests the retained question sentence may contribute to its PA accuracy. We test this directly below.
  • We re-ran the MedQA attack with the question sentence removed

Table 2.a: Llama-3 MedQA prompt-attack accuracy before and after removing the final question sentence

Llama-3 configurationOriginal PAName + optionsΔ
Base0.40600.2906-0.1154
Full FT / full loss0.37200.3220-0.0500
Full FT / masked loss0.40600.3180-0.0880
QLoRA / full loss0.42400.3220-0.1020
QLoRA / masked loss0.51000.3880-0.1220

Table 2.b: Raw MedQA PA accuracy for the name + options formulation.

BackboneBaseQLoRA / fullQLoRA / masked
Llama-3-8B-Instruct0.29060.32200.3880
Med-LLaMA-8B0.28480.27770.3880
Meditron-7B0.27130.29010.3642
Qwen3-0.6B0.27490.27400.3420
Qwen3-1.7B0.31400.29200.3600
Qwen3-4B0.26400.27400.3580

Table 2.c: Spearman correlations between privacy metrics and PA accuracy under the original and name + options prompt formulations.

ProtocolMetric vs. PA accuracyOriginal PA (ρ, p, q_BH)Name + options PA (ρ, p, q_BH)
EMMem@10-0.272, 0.061, 0.202-0.098, 0.710, 0.789
EMROUGE-L-0.328, 0.013, 0.065-0.295, 0.224, 0.374
EMBERTScore-0.234, 0.009, 0.065-0.358, 0.137, 0.274
MIAROC-AUC+0.154, 0.387, 0.553-0.246, 0.547, 0.684
MIATPR@5%FPR+0.251, 0.119, 0.274-0.026, 0.906, 0.906
  • All six base models fall to chance (0.26 to 0.31, compared to a 0.25 floor (Table 2.b)).
  • Masked-loss fine-tuning lifts accuracy above base on every backbone (+0.05 to +0.10).
  • Full-loss QLoRA shows little or no lift, consistent with our finding that masked loss changes the risk a model carries (Section 5).
  • Configuration ordering is preserved: QLoRA-masked remains the most vulnerable Llama-3 configuration under both prompts, and full-loss QLoRA shows little or no lift, consistent with Section 5.
  • Cross-protocol relationships have non-significant changes: (Table 2.c) all correlations remain weak and non-significant under both prompts.

Our conclusions therefore hold under the identity-only prompt. We will add this control to the camera-ready version.

AuthorsOpenReview ↗

Response to reviewer uiRe part 3

W3/Q3

The answer is determined by which metric defines the budget and how tight it is. Under a formal DP budget, Full FT gives better utility. Under empirical budgets, a tight prompt-attack budget favors Full FT for five of six backbones, while tight memorization and membership budgets split evenly, and the regime with the higher unconstrained utility wins once the budget is loose. We agree this should have been stated explicitly; it is the direct consequence of our central finding that a privacy budget is not well defined until the metric is chosen.

Table 3.a: Preferred fine-tuning regime on MedQA under tight and loose privacy budgets.

BackboneHighest utilityMem@10 (Tight / Loose)PA acc. (Tight / Loose)MIA TPR@5% (Tight / Loose)
Llama-3-8B-InstructFull FTQLoRA / Full FTFull FT / Full FTFull FT / Full FT
MedLLaMAQLoRAFull FT / QLoRAFull FT / QLoRAFull FT / QLoRA
Meditron-7BQLoRAQLoRA / QLoRAFull FT / QLoRAQLoRA / QLoRA
Qwen3-0.6BQLoRAQLoRA / QLoRAQLoRA / QLoRAQLoRA / QLoRA
Qwen3-1.7BFull FTFull FT / Full FTFull FT / Full FTQLoRA / Full FT
Qwen3-4BFull FTFull FT / Full FTFull FT / Full FTFull FT / Full FT
Tight budget: Full FT / QLoRA—3 / 35 / 13 / 3

Caption: Preferred fine-tuning regime on MedQA by privacy-budget metric, based on the utility–privacy Pareto landscapes (Figure 2 and Figures 25-29). A tight budget admits only the lowest-risk fine-tuned configuration, while a loose budget admits all configurations, making the highest-utility configuration preferred. Each entry reports the preferred regime under tight / loose budgets.

  • The regime flips with the budget for the same model: For Llama-3 under Mem@10, QLoRA-masked gives the highest utility at a threshold of 0.03 (0.699 at risk 0.024), while Full FT-masked does at 0.04 (0.701 at risk 0.040) (Table 2). MedLLaMA flips in the opposite direction under all three metrics.
  • The regime flips with the metric for the same budget level(Table 3.a): Under a tight budget, Meditron-7B favors QLoRA for memorization and membership inference but Full FT for the prompt attack; Qwen3-1.7B favors Full FT for memorization and the prompt attack but QLoRA for membership inference.
  • Formal DP budgets favor Full FT : For Llama-3, Full FT is ahead of QLoRA at every {1,2,10} on MedQA (accuracy 0.68 vs.\ 0.56 to 0.58) and mortality (0.84 vs.\ 0.41 to 0.51) (Figures 34 and 35), and DP-QLoRA fails to converge for Qwen3-4B on MedQA.
  • Across all four tasks, loss type matters more than regime: The masked-loss variant of either regime gives the highest utility gain in most backbone--task cells (Figure 17; average gain on MedQA +9.6% for QLoRA-masked vs.\ +5.9% for Full FT-masked, on readmission +6.8% for Full FT-masked vs.\ +2.8% for QLoRA-masked), and the two masked-loss configurations form the Pareto frontier in nearly every panel of Figures 2 and 25 to 29.

We will add this analysis to Section 4.3, state the answer directly in the abstract and discussion

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.