acceptodds
Under review as a conference paper at ICLR 2027

Privacy leakage or distribution shift? Post-hoc Privacy Auditing of foundation models

Abstract

Membership inference attacks (MIA) provide a theoretically grounded basis for auditing privacy leakage from machine learning models, but applying it in a post-hoc manner to pre-trained large language models (LLMs) is challenging. Indeed, MIAs quantify leakage by distinguishing data used in training (members) from non-members from the same distribution. However, in practice, auditors have to collect non-members from different sources, resulting in distribution shift that biases privacy leakage measurements. To solve this issue, we propose two general extensions to recently proposed debiased post-hoc privacy auditing techniques to make them practical for LLMs. First, we reduce the non-member distribution shift by rewriting non-members in the style of member data and filter members and non-members to focus on regions of greater empirical overlap. Second, we strengthen MIAs on LLMs by adding paraphrase-related features that compare model behaviour on suspected members with model behaviour on semantically similar paraphrases. A thorough experimental evaluation on text and vision-language benchmarks with temporal, formating and collection shifts, demonstrates that our approaches decrease distribution shift while retaining detectable membership signals, thus yielding meaningful privacy leakage estimates. Our results also show that distribution shifts resulting from naturally collected non-members does not preclude practical post-hoc privacy auditing.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Privacy, Auditing, Differential Privacy, Membership Inference Attack

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_2gYmOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes three practical mechanisms for post-hoc MIA-based privacy auditing of LLMs and VLMs. It reduces distributional shift between member and non-member examples by rewriting non-member texts and filtering both groups for greater empirical overlap, and improves MIA performance by adding paraphrase-related features from auxiliary LLMs. The experimental results across multiple datasets and models show effective privacy leakage estimates with reduced distributional shift and a retained MIA signal with these techniques.

Critical strengths
  1. Practical improvements to post-hoc privacy auditing: The paper proposes non-member rewriting, overlap-based filtering, and local-paraphrase-based features for MIA to reduce data-only distinguishability between member and non-member data while retaining MIA performance, providing a practical privacy auditing approach for LLMs and VLMs.
  2. Extensive experiments and ablation studies: The experiments cover a wide range of target models and datasets, providing evidence for the effectiveness and generalizability of the proposed techniques. The ablation studies also demonstrate the contribution of each component to the claimed improvements.
Critical weaknesses
  1. Residual distributional shift may be mistaken for privacy leakage: In this work, BERT models are used to encode text inputs for the discriminator, while the target models are LLMs/VLMs. The discriminator may miss distributional shift that the LLM-/VLM-based model can exploit. It remains unclear whether the performance gap reflects actual privacy leakage or the residual distributional shift that the discriminator fails to capture.
  2. Inconsistent result reporting: Some reported results seem to be inconsistent across tables and sections, leading some quantative comparisons and conclusions unclear. For example:
    1. Table 3 reports ϵ~causalq0.025\tilde{\epsilon}^{q_{0.025}}_\text{causal} values ranging from 0.42 to 0.60 for OpenClip models on LAION and cc12m, which are lower than the reported PANORAMIA estimates, while Section 3.1 states them as 1.74, 2.09, and 2.06, and are larger instead.
    2. Table 3 reports ϵ~panoramia\tilde{\epsilon}_\text{panoramia} as 1.72 for Gutenberg/OpenLLaMA-3B, while Section 3.1 states it as 1.71.
    3. Section 3.2 states that ϵ~causal\tilde{\epsilon}_\text{causal} is the largest for LAION benchmark when filtering at k=0.25k = 0.25, but Table 4 shows the largest value at k=0.5k = 0.5 (1.7040 at k=0.5k = 0.5 vs. 1.4776 at k=0.25k = 0.25).
    4. Table 3 reports TPR@10%FPR as 0.5994 for LAION/Vit-B-32, while the value is 0.5594 in Table 4 for the same setup.
Questions for authors
  1. Conflicting reported results: Which reported ϵ~causalq0.025\tilde{\epsilon}^{q_{0.025}}_\text{causal} are correct for the OpenClip models on LAION and cc12m? Could the authors clarify the discrepancy between Table 3 and Section 3.1? Would the correction change the interpretation of privacy leakage estimates in Section 3.1?
  2. WikiMIA auditing results: In Table 3, the MIA yields a lower TPR@k%FPR than the discriminator across all target models while providing positive privacy estimates. Section 3.1 attributes this to the better performance at very low FPR, which is absent in the table. Could the authors report these results to support the claim? Is there any explanation or intuition behind this performance trend?
  3. Effect of rewriting on MIA performance: Why does rewriting increase MIA's TPR@k%FPR for Temporal-Wiki/Pythia-2.8B in Table 4? Does this reflect better separation of member and non-member data, or artifacts introduced by rewriting?
3 comments

W1: “..The discriminator may miss distributional shift that the LLM-/VLM-based model can exploit..”

To directly investigate whether this difference in feature information explains the observed membership advantage, we conduct an additional experiment on Temporal-Wiki using GPT-2 as an independent reference model. GPT-2 was pretrained on WebText, which explicitly excluded Wikipedia pages, making it a suitable reference model with different training provenance from the audited Pythia models. We extract the same paraphrase-based z-score features from GPT-2 as those used in our target-model MIA and incorporate them into the data-only discriminator. This allows the discriminator to capture additional distributional differences detectable through autoregressive language-model features, without accessing the audited target model. As shown in Section 3.2 Table 7, the adjusted leakage estimate remains positive after adding these reference-model features. Moreover, the observed lower bound on the c_closeness remain almost identical for two of the reference model and comparable (well within one std) for the largest reference model. This provides evidence that the MIA's advantage is not simply due to its access to paraphrase-based likelihood features, but reflects additional membership information obtained specifically from the audited target model. For other benchmarks, identifying a suitable autoregressive reference model with documented non-overlap with the audited member records is considerably more difficult. A reference model that has itself trained on the audited members could introduce membership information into the discriminator, complicating its interpretation as a data-only baseline. We therefore retain the BERT-based discriminator for these settings and acknowledge this limitation. For the VLM experiments, our existing discriminator already incorporates features from a pretrained reference CLIP model, including cross-modal representations and similarity scores comparable to those used by the target-model MIA (Figure 3). The resulting leakage estimates remain positive, providing additional evidence that the observed membership advantage is not simply attributable to the target-model MIA having access to richer feature representations.

W2&Q1: “Inconsistent result reporting..”

We apologize for these errors in writing, which we have corrected in the paper. Table 3 contains the correct VLM results. We have updated Table 3 with the addition of standard deviations. We have corrected the numerical values and corresponding discussion in Section 3.1 to match Table 3. We note that it does not change our main conclusion: we do detect positive privacy leakage in these settings.

Q2: “WikiMIA auditing results..”

For that particular run of WikiMIA we report [email protected]%FPR scores for discriminator and MIA as follows: Discriminator: 0.0, Pythia2.8B/12B: 0.0625, Pythia6.9B: 0.0129. This distinction at smaller FPR rate is relevant for WikiMIA because of its small sample size. The benchmark contains only 388 members and non-members each, and is further reduced to 155 per group after filtering. At this size, a single false positive already corresponds to approximately 0.65% FPR, so the reported TPR@1%FPR provides limited resolution of the extreme tail. We have updated Table 7 with mean and standard deviation over 5 runs with random seed and the mean values behave more robustly as expected.

Q3: “Effect of rewriting on MIA performance..”

The increase in TPR after rewriting should be interpreted in the context of the changed non-member distribution, and we do not interpret the raw increase in MIA TPR as evidence of stronger membership leakage. Since TPR is measured at a fixed FPR with respect to the non-member data, rewriting changes the empirical data distribution of non-members and the learned classification boundary and affects TPR as well. In this specific case, we observe that rewriting shifts non-member MIA scores downward, and the decision threshold corresponding to 1% FPR also decreases, allowing more member data to be correctly classified and increasing TPR. Importantly, our discriminator is designed to capture these distributional differences observable from the data, including potential artifacts introduced by rewriting. Such differences are accounted for in our discriminator-adjusted audit.

Area_Chair_GWgmOpenReview ↗

Reviewer interaction needed

Dear reviewer,

The authors have replied to your comments and the discussion is close to the end. We encourage you to participate in the discussion to see whether your questions and concerns have been answered by the authors. Thank you!

Best, Your AC

Thank you for the clarifications and corrections to the experimental results. My concerns have been addressed: the additional GPT-2 experiment helps address my concern about the discriminator, the reporting inconsistencies have been corrected, and the explanations of the WikiMIA and rewriting results are helpful.

I will raise my score from 2 to 3. Thanks!

Reviewer_w2okOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses post-hoc privacy auditing of foundation models, where members and non-members come from different distributions and this shift inflates membership inference results. Building on PANORAMIA and causal auditing, the authors reduce the shift by rewriting non-members in the style of members and filtering both sets to their overlapping region. They also strengthen the MIA with paraphrase-based features. Experiments on several LLM and VLM benchmarks report positive privacy leakage estimates in most settings.

Critical strengths
  1. PANORAMIA and causal auditing had not produced meaningful estimates on real LLMs before, and this paper shows that rewriting and filtering can make them work on real foundation models.

  2. The method needs no shadow models and no access to training, and the same pipeline works for both text and image-caption data. The paraphrase-based reference is a cheap substitute for shadow models and could be useful beyond this paper.

Critical weaknesses
  1. On the experimental side, the paper never checks whether the pipeline gives near-zero leakage on a model that did not train on the members. Data-only differences alone can already separate members from non-members [1, 2], so without this negative control it is hard to tell real leakage from pipeline artifacts. This is the main reason I am not convinced by the central claim.

  2. On the method side, the non-members are rewritten by LLMs, and LLM-generated text has a distinctive likelihood profile under language models [3]. The paraphrase-based z-score is also very close to DetectGPT’s detection statistic [3]. A BERT-only discriminator cannot see these differences, and filtering keeps exactly the samples BERT fails to separate. Both could lower c_lb and inflate the leakage estimate.

[1] D. Das, J. Zhang, and F. Tramèr. Blind Baselines Beat Membership Inference Attacks for Foundation Models. IEEE Security and Privacy Workshops (DLSP), 2025. arXiv:2406.16201.

[2] J. Zhang, D. Das, G. Kamath, and F. Tramèr. Position: Membership Inference Attacks Cannot Prove that a Model Was Trained On Your Data. arXiv:2409.19798.

[3] E. Mitchell, Y. Lee, A. Khazatsky, C. D. Manning, and C. Finn. DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature. ICML 2023.

Questions for authors
  1. Can you run the pipeline on a model that never saw the members, for example an intermediate Pythia checkpoint evaluated on data it has not yet seen? Near-zero estimates would resolve my main concern.

  2. If the discriminator also gets likelihood features from a reference LM that did not see the members, do the leakage estimates stay positive?

  3. How was k chosen for each dataset? If it was chosen based on the final estimates, how is this selection accounted for?

  4. Section 3.1 reports bootstrap lower bounds of 1.74, 2.09 and 2.06 for the image-caption models, but Table 3 shows 0.42, 0.60 and 0.45. Which values are correct?

3 comments

W1&Q1: “..the paper never checks whether the pipeline gives near-zero leakage on a model that did not train on the members..”

We add negative-control experiments where both pseudo-members and pseudo-non-members are drawn from data that were not used to train the target model. We consider two settings: (1) randomly splitting non-members from the same benchmark into two groups, where little distribution shift is expected, and (2) constructing the two groups using non-members from different benchmarks, where we expect distribution shift. We designed these settings instead of using an intermediate Pythia checkpoint, which would require reconstructing checkpoint-specific training data and evaluating a partially trained model. Our design preserves the original fully trained target model and uses known non-members to isolate potential false leakage attribution. We report the results and discussion in Section 3.2 and Table 6. In summary, across these negative-control experiments, we obtain 0 or near-0 (0.03 and 0.02 within one std of zero) audited leakage estimates, suggesting that our pipeline does not spuriously attribute distributional shift to target-model membership leakage in the controlled settings where there is known no training evidence.

W2: “..LLM-generated text has a distinctive likelihood profile under language models..”

We agree that LLM rewriting can introduce generation-specific artifacts, and this is precisely one of the considerations motivating our use of multiple rewriting models (we mentioned this in the Mitigating rewriting artifacts paragraph in Section 2.2 and we have edited it to be more clear). Specifically, we generate rewrites using two different auxiliary LLMs, combine their outputs into a candidate pool, and retain the most member-like rewritten version per original non-member during the filtering phase. This design reduces dependence on the characteristic writing patterns of any single rewriting model and avoids retaining multiple related rewrites from the same original text. We do not claim that this would eliminate all differences between original and LLM-rewritten text, such as specific likelihood patterns. However, note that remaining signatures are captured by the discriminator as well, and removed from the MIA signal to compute the final audited leakage estimate. If anything, such signatures should reduce the power of our test (underestimating leakage), and not create mis-attribution.

W2: “..paraphrase-based z-score is also very close to DetectGPT’s detection statistic..”

We agree that our paraphrase-based z-score shares the general principle of comparing a text's likelihood with neighbouring perturbations. However, the two methods address different objectives and use different feature constructions. DetectGPT estimates a normalized log-probability discrepancy under local perturbations to detect machine-generated text without classifier training. Our method instead generates semantic paraphrases and computes standard and robust normalized scores for a collection of target-model statistics, including token-level loss distributions and Min-K++ features. These features are then combined with text representations in a supervised membership inference classifier. We clarify this in related work (Appendix A) and cite DetectGPT in the revised paper.

W2&Q2: “...BERT-only discriminator cannot see these differences, and filtering keeps exactly the samples BERT fails to separate…”, “If the discriminator also gets likelihood features from a reference LM that did not see the members, do the leakage estimates stay positive?”

To directly test this hypothesis, we conduct the suggested experiment on Temporal-Wiki using GPT-2 as a reference model. GPT-2 was pretrained on WebText, which explicitly excludes Wikipedia pages, making it a suitable reference model with substantial difference from the Pythia target models. We extract the same paraphrase-based z-score features from GPT-2 as those used in our target-model MIA, and add them to the data-only discriminator. As shown in Section 3.2 Table 7, the adjusted leakage estimate remains positive after adding these reference-model features. Moreover, the observed lower bound on the c_closeness remain almost identical for two of the reference model while we observe a modest improvement for the largest reference model. This provides evidence that the MIA's advantage is not simply due to its access to paraphrase-based likelihood features, but reflects additional membership information obtained specifically from the audited target model. For other datasets, identifying a comparable autoregressive reference model with documented non-overlap with the audited member records ismore difficult. Using a reference model that has itself trained on the audited members would introduce membership information into the baseline, confounding its interpretation as a data-only discriminator. We therefore retain the discriminator structure for these settings. For the VLM experiments as described in Section 2.4 and illustrated in Figure 3, the reported results already incorporate features from a pretrained reference CLIP model into the discriminator, providing further evidence that the observed membership advantage is not simply due to MIA having access to richer feature representations.

Q3: “How was k chosen for each dataset?...” The filtering proportion k is selected from {0.25, 0.5, 0.75} based on the empirical score distributions shown in Figure 4. The single k value is fixed before downstream discriminator and MIA evaluation, and we do not fine-tune many k values to maximize the audit results.

Q4: “Which values are correct?”

We apologize for these errors in writing, which we have corrected in the paper. Table 3 contains the correct VLM results. We have updated Table 3 with the addition of standard deviations. We have corrected the numerical values and corresponding discussion in Section 3.1 to match Table 3. We note that it does not change our main conclusion: we do detect positive privacy leakage in these settings.

We thank the authors for the additional experiments and clarifications. The negative controls and stronger discriminator largely resolve our main concerns. The authors also clarify the filtering procedure and correct the reported results. Further validation beyond Temporal-Wiki would strengthen the reference-model analysis. We raise our score from 2 to 3.

Reviewer_HbiqOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies a problem in post-hoc privacy auditing: a membership attack may detect source or time differences rather than whether a sample was used for training. To reduce this confusion, it rewrites non-members in the style of members and filters both groups toward greater overlap. It then strengthens data-only discriminators and adds paraphrase-relative MIA features before applying existing PANORAMIA and causal auditing methods. The contribution is a set of practical improvements to these auditing pipelines.

Critical strengths
  1. The motivation addresses a real barrier to privacy auditing. Source and time differences can make MIA success look like evidence of training membership. The paper targets the practical setting where an auditor cannot obtain clean, matched member and non-member data.
  2. The evaluation includes relevant data-only baselines and component tests. Table 7 compares TF-IDF and Bag-of-Words classifiers with stronger encoders, without accessing the audited model. Tables 4–5 separately test rewriting/filtering and MIA features, showing reduced data distinguishability and stronger attacks. These controls help assess the pipeline beyond reporting a larger privacy estimate.
Critical weaknesses
  1. Lower data distinguishability does not establish a more accurate audit. Table 4 compares original, rewritten, and filtered data, but provides no genuinely same-distribution member/non-member reference. It therefore does not show how closely the corrected results recover an audit without source or time confounding. The lexical and BERT baselines are useful, but a weaker discriminator signal does not prove that residual shift is removed. A full-pipeline control on a target that never trained on the audit records is also not reported.
  2. The text and table give opposite comparisons with PANORAMIA. For OpenCLIP ViT-B/32, page 7 reports a causal bootstrap estimate of 1.74, above PANORAMIA. Table 3 instead gives 0.4203, below PANORAMIA's 0.6826. The other two VLM comparisons also reverse. The closeness definition is inconsistent too: Appendix B states pG(x)≤ecpD(x)p_G(x)\le e^c p_D(x), while §2.1 states pD(x)≤ecpG(x)p_D(x)\le e^c p_G(x). These are different assumptions, so the authors must identify which one the analysis and code use.
Questions for authors
  1. Add a same-distribution reference. Using a controlled target with known training membership, please compare genuinely same-distribution members/non-members, a shifted audit set, and the corrected set. Report both data-only and target-model MIA TPR at the same FPR, audit estimates, and retained sample sizes. This would test correction accuracy rather than just reduced distinguishability.

  2. Measure the audit's false-positive rate. Please test the complete method on a controlled model that was not trained on any of the audit records, while keeping the source or time differences between the two groups. State the rule used to declare leakage, then report the fraction of repeated audit trials that incorrectly declare training-data leakage.

  3. Clarify how data are split and how rewritten samples are checked. Please state which samples are used to guide rewriting, train the filter and classifiers, choose settings, and evaluate the final audit. Explain whether an original record and all its rewritten versions stay in the same split. After changing entities or events, explain how you check that rewritten non-members are still absent from the target model's training data. Where this cannot be checked, state that limit.

  4. Correct the conflicting numbers and definitions. Please identify whether the VLM bootstrap values on page 7 or in Table 3 are correct, then update the comparison with PANORAMIA. Also identify which closeness inequality the analysis and code use: pD(x)≤ecpG(x)p_D(x)\le e^c p_G(x) or pG(x)≤ecpD(x)p_G(x)\le e^c p_D(x). Provide the result files and settings supporting these corrections.

6 comments

W1&Q1: “Lower data distinguishability does not establish a more accurate audit.”, “Add a same-distribution reference.”

Our goal is to improve the detection of membership information from a target model, under a realistic post-hoc auditing setting where distribution shift confounds the audit results. We do not aim to weaken the discriminator, but to construct an audit set in which non-members are closer to the members’ distribution, thereby increasing the power of membership detection tests. We do this with our rewriting and filtering pipeline, designed to increase the distributional overlap of naturally available non-member data. We still account for distribution shifts through a data-only discriminator, which we make as strong as possible for the discrimination task between members and re-written non-members. Prior work ([1], one of the auditing methods we use) has demonstrated the effectiveness of privacy measurement when in-distribution non-members are available: Section 5.2 compares audits using real members and in-distribution real non-members (RM;RN). The other auditing method we study [2] rely on propensity scores to account for distribution shift using propensity score and discount correct guesses on easy to distinguish (due to distribution shift) samples. Ablation results in Table 4 confirm that lowering distinguishability increases the measured audit value confirming the original paper assumption that propensity correction retains more informative guesses when member and non-member are similar. Our contribution in this work focuses on making these auditing [1,2] approaches applicable to trained models under data distribution shift. We still recognize the value of additional controls, especially to evaluate whether our re-writing pipeline introduces spurious membership evidence. We therefore conduct negative-control experiments, as described in our response to the next comment.

Q2: “..test the complete method on a controlled model that was not trained on any of the audit records..”

We add negative-control experiments using pseudo-members and pseudo-non-members that are both drawn from data that were not used to train the target model. We consider two settings: (1) randomly splitting non-members from the same benchmark into two groups, where little distribution shift is expected, and (2) constructing the two groups using non-members from different benchmarks, where we expect distribution shift. We run these experiments on the Pythia-2.8B model, which does not show evidence of real member leakage on both datasets (with ϵcausalq0.025=0\epsilon_ {causal}^{q0.025}=0 and ϵpan\epsilon_ {pan} ranges from 0.00 to 0.03, which are both significantly lower than the reported results in Table 3). We report and discuss the results in Section 3.2 and Table 6. In summary, across these negative-control experiments, we obtain 0 or near-0 (0.03 and 00.2 within one std of zero) audited leakage estimates, suggesting that our pipeline does not spuriously attribute distributional shifts to target-model membership leakage in the controlled settings where pseudo-members are not actually in the training set.

Q3: “Clarify how data are split and how rewritten samples are checked.”

We have added a detailed description of the data construction and evaluation pipeline in Appendix D.1 to clarify these points. To address the main concerns, we check for exact duplicates to ensure that rewritten non-members do not directly reproduce exact member text, and enforce valid train/test splits to prevent different rewrites of the same original non-member from appearing across splits. For BookMIA, we additionally split by book ID to avoid leakage between passages from the same book. The filtering proportion k is selected from {0.25, 0.5, 0.75} based on the empirical score distributions shown in Figure 4, before downstream discriminator and MIA evaluation, and we do not fine-tune many k values to maximize the audit results.

W2&Q4: “Correct the conflicting numbers and definitions.”

We apologize for these errors in writing, which we have corrected in the paper. Table 3 contains the correct VLM results. We have updated Table 3 with the addition of standard deviations. We have corrected the numerical values and corresponding discussion in Section 3.1 to match Table 3. We note that it does not change our main conclusion: we do detect positive privacy leakage in these settings. The inequality in Appendix B is a typo, and the correct expression is pG(x)≥e−cpD(x)p_\mathcal{G}(x)\geq e^{-c} p_\mathcal{D}(x), consistent with Section 2.1 and Definition 3 of PANORAMIA [1]. We emphasize that this error is limited to the written expression: our analysis uses the correct closeness assumption, and our experiments use the original PANORAMIA auditing implementation provided by [1]. Therefore, there is no inconsistency between the theoretical analysis and experimental implementation, and the correction does not affect any reported result.

[1] Kazmi et al. “PANORAMIA: Privacy Auditing of Machine Learning Models without Retraining.” NIPS2024.

[2] Cebere, Tudor, et al. "Privacy Auditing with Zero (0) Training Run." arXiv preprint arXiv:2605.14591 (2026).

Area_Chair_GWgmOpenReview ↗

Reviewer interaction needed

Dear reviewer,

The authors have replied to your comments and the discussion is close to the end. We encourage you to participate in the discussion to see whether your questions and concerns have been answered by the authors. Thank you!

Best, Your AC

Comments on the paper

1 comment

We thank the reviewers for their precious feedback.

First and foremost, we would like to apologise for discrepancies between table 3 and result discussion. The correct numbers were the one from Table 3. We have also updated Table 3 with the addition of standard deviation. We have also corrected typos in the c-closeness definition in Appendix B.

Following your recommendations, we have added two experiments to the paper. First, a negative control experiment with pseudo-members and pseudo-non-members both not seen by the model at training time. This experiment verifies that the Membership Inference Attack is not a better distribution shift detector than the baseline model. We measure audit values of zero (or near zero within one std) providing evidence that the main experiment audit values measure privacy leakage rather than distribution shift. Similarly in the text-only modality, the MIA has access to additional loss related features compared to the discriminator. We ran a control experiment adding to the discriminator features extracted from a reference GPT-2 model (not trained on neither member nor non-member data). We observe that the addition of those extra features does not close the gap between the discriminator and MIA performance. The observed privacy leakage can not be attributed to a weak by design discriminator.

Both experiments show expected results and address reviewers concerns.

All updates to the paper are highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.