acceptodds
Under review as a conference paper at ICLR 2027

Hierarchical Bayesian Membership Inference: A General Framework with Applications to Diffusion Models

Abstract

Membership inference attacks often reduce a model's response to a single score and treat the parameters of class-conditional signal distributions estimated from finitely many reference models as known, without propagating estimation uncertainty or separating variation across stochastic queries, trained models, data points, and groups. We propose a general hierarchical Bayesian framework that models these sources of variation under the member and non-member hypotheses, propagates uncertainty from finite reference-model data, and yields a posterior-predictive likelihood-ratio test. We instantiate the framework for denoising diffusion and flow matching models, whose randomized objectives yield repeated loss observations. For record-level inference, our attack outperforms state-of-the-art methods. For group-level inference, it infers an individual's presence without access to their training images and outperforms adapted baselines with a mixed DDPM and flow-matching reference pool, demonstrating robustness to model-type mismatch. Beyond diffusion, the framework applies to deterministic, repeated stochastic, and grouped observations.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
privacy, membership inference attacks, hierarchical Bayesian modeling, denoising diffusion models, flow matching models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 2 reviews, above 85% of the others.
Reviewer_zHJeOpenReview ↗
Rating
3(+1)
Confidence
2(−2)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a hierarchical Bayesian framework for membership inference that separates variation across stochastic queries, reference models, and data points. Its record-level instantiation models diffusion log losses with a Gaussian–Gamma hierarchy and computes a joint posterior-predictive likelihood ratio using Gibbs sampling and analytic marginalization. The group-level attack combines empirical-Bayes variance estimation with cross-fitted target-model adaptation to infer identity membership from potentially unseen images. Experiments cover DDPM and flow-matching models across four image datasets. The paper reports improved low-FPR attack performance and robustness when reference pools mix the two model families.

Critical strengths
  • Separating query randomness from variation across trained models is a sensible modeling choice. The conditionally conjugate construction and posterior-predictive derivations provide a concrete implementation of this idea.

  • The reported record-level improvements are substantial in several settings. For example, Table 1 reports 64.72% TPR at 0.1% FPR for flow matching on CIFAR10, compared with 51.45% for BASE and 48.94% for LiRA.

  • Inferring identity membership without querying training images is practically relevant. Table 2 explicitly separates this setting from one allowing a training image, and reports meaningful improvements over the adapted baselines.

Critical weaknesses
  • The experiments do not isolate the benefit of the proposed uncertainty modeling. The biggest concern for me is that the paper motivates its contribution through uncertainty propagation with finitely many reference models, but omits the most direct Bayesian comparison.

  • The explanation and scope of mismatch robustness are stronger than the evidence supports. Sections 4.1 and 5.3 explicitly treat the target as another draw from the same class-conditional population as the reference models. Integrating uncertainty within that population does not by itself establish robustness when the target comes from a different population.

  • The group-level uncertainty interpretation needs validation beyond ROC performance. The implemented group attack replaces the local predictive mixture with a moment-matched Gaussian and treats fitted factor quantities as fixed; Appendix D.6 appropriately acknowledges these approximations. It also assumes conditional independence between queried images after accounting for the global model coordinate. Consequently, the aggregate variance in Eq. (157) contains no residual covariance between images of the same person. Such dependence could affect the predictive tails, particularly when aggregating up to 200 images.

  • Important experimental choices are insufficiently specified in the manuscript. The attack fixes t∗t^* where the membership signal is strong, but neither its values nor the data used to select it are reported. This matters for comparing attacks without tuning on evaluation membership labels.

Questions for authors

See the weaknesses.

7 comments

Thank you for the review and for providing very relevant feedback for improving our work. Below we have addressed the questions and concerns raised, including a comparison with BaVarIA (eq. (8)), clarifications regarding the robustness under mixed reference model types including a new ablation experiment, a discussion of our group-level attack and the omission of residual covariance between images of the same individual with a new experiment that investigates this, and finally a clarification on some experimental parameter settings that were missing in the submission, including the loss query time t∗t_*.

Weakness 1.

We agree that our paper should include the most direct Bayesian comparison as a baseline, namely the BaVarIA attack (Eq. (8)). We have thus implemented its two variants, BaVarIA-t and BaVarIA-n, described in its original paper (Brännvall, Exponential-Family Membership Inference: From LiRA and RMIA to BaVarIA, 2026). We use κ₀ = 1 and α₀ = 2 for both variants, as suggested in their paper. The experimental settings are the same as described in our paper and we use the same log-loss samples and reference models for each attack.

All tabulated performance values are percentages, mean ± population standard deviation over ten targets. Bold marks the highest mean in each row. ImageNet is evaluated on a subset of 100k data points.

Table 1. Matched references: TPR at 0.1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online65.25 ± 6.5248.53 ± 4.8326.27 ± 2.7344.50 ± 2.8575.42 ± 4.05
DDPM / CIFAR10 / offline29.21 ± 3.8925.81 ± 3.8317.77 ± 2.2912.60 ± 1.4255.84 ± 6.37
FM / CIFAR10 / online51.45 ± 4.1938.56 ± 5.4122.98 ± 1.6138.88 ± 1.2964.72 ± 2.04
FM / CIFAR10 / offline21.80 ± 2.2118.85 ± 1.9313.62 ± 1.6111.19 ± 1.2946.00 ± 2.89
DDPM / CelebA / online15.30 ± 2.8613.61 ± 1.725.02 ± 0.615.42 ± 0.7017.03 ± 2.32
DDPM / CelebA / offline13.48 ± 1.7013.38 ± 1.705.16 ± 0.502.71 ± 0.1714.67 ± 2.11
FM / CelebA / online43.35 ± 3.9535.00 ± 1.3715.43 ± 0.9723.05 ± 1.6450.51 ± 3.46
FM / CelebA / offline26.54 ± 1.7925.53 ± 1.6012.22 ± 0.767.04 ± 0.2939.36 ± 4.03
FM / ImageNet / online0.94 ± 0.111.31 ± 0.180.98 ± 0.070.64 ± 0.062.03 ± 0.21
FM / ImageNet / offline1.29 ± 0.121.26 ± 0.120.86 ± 0.070.57 ± 0.051.49 ± 0.16

Table 2. Matched references: standardized pAUC at 1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online89.95 ± 2.3786.92 ± 1.6267.78 ± 1.5876.09 ± 1.5591.97 ± 1.55
DDPM / CIFAR10 / offline77.31 ± 2.4276.32 ± 2.3864.51 ± 1.0861.41 ± 0.4484.64 ± 2.73
FM / CIFAR10 / online83.95 ± 1.5181.81 ± 0.7966.48 ± 0.7072.02 ± 0.5986.89 ± 0.93
FM / CIFAR10 / offline72.63 ± 1.0171.57 ± 1.0362.63 ± 0.5960.29 ± 0.4179.69 ± 1.34
DDPM / CelebA / online63.91 ± 2.0062.86 ± 1.4854.23 ± 0.4254.56 ± 0.4664.18 ± 1.61
DDPM / CelebA / offline62.50 ± 1.4762.43 ± 1.4553.76 ± 0.2952.17 ± 0.0862.87 ± 1.58
FM / CelebA / online80.35 ± 2.1378.11 ± 1.0561.17 ± 0.8165.79 ± 1.1482.20 ± 1.71
FM / CelebA / offline73.01 ± 1.5372.60 ± 1.4958.94 ± 0.5155.63 ± 0.0877.01 ± 2.25
FM / ImageNet / online51.38 ± 0.1051.62 ± 0.0850.96 ± 0.0450.48 ± 0.0551.97 ± 0.09
FM / ImageNet / offline51.71 ± 0.1051.69 ± 0.1050.68 ± 0.0450.47 ± 0.0351.83 ± 0.09

Table 3. Mixed references: TPR at 0.1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online48.45 ± 7.2534.82 ± 5.2332.28 ± 3.6244.71 ± 3.6765.62 ± 3.74
DDPM / CIFAR10 / offline24.08 ± 3.7320.73 ± 2.9316.86 ± 2.6615.87 ± 2.0253.77 ± 5.15
DDPM / CelebA / online5.71 ± 1.095.16 ± 0.763.49 ± 0.503.41 ± 0.4813.88 ± 2.23
DDPM / CelebA / offline5.93 ± 0.944.91 ± 0.811.99 ± 0.261.79 ± 0.2513.77 ± 2.31
FM / CIFAR10 / online17.92 ± 3.5531.93 ± 3.8028.99 ± 1.9739.14 ± 3.0555.39 ± 2.75
FM / CelebA / online12.04 ± 2.3418.82 ± 1.5017.40 ± 1.1122.86 ± 1.4535.20 ± 2.34

Table 4. Mixed references: standardized pAUC at 1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online82.49 ± 2.9778.35 ± 2.7071.94 ± 2.0578.81 ± 1.6886.14 ± 1.65
DDPM / CIFAR10 / offline71.43 ± 2.7269.92 ± 2.6463.25 ± 1.3762.22 ± 1.0381.35 ± 2.23
DDPM / CelebA / online56.01 ± 0.8255.22 ± 0.6653.42 ± 0.4053.48 ± 0.4161.18 ± 1.34
DDPM / CelebA / offline55.66 ± 0.7354.94 ± 0.6551.96 ± 0.2151.77 ± 0.1961.83 ± 1.51
FM / CIFAR10 / online65.16 ± 2.2076.60 ± 1.0669.97 ± 0.7974.46 ± 0.7882.56 ± 1.33
FM / CelebA / online62.69 ± 1.8766.56 ± 1.1363.64 ± 0.8967.87 ± 1.1274.23 ± 1.57

Tables 1–4 demonstrate that the BaVarIA attacks are performing substantially worse than our hierarchical Bayesian attack: for example, in the online setting, mean TPR at 0.1% FPR is 75.42% for our attack versus 26.27% and 44.50% for BaVarIA-t and BaVarIA-n on DDPM/CIFAR10 with matched references, and 35.20% versus 17.40% and 22.86%, respectively, on FM/CelebA with mixed references. In the matched setting, BaVarIA is even worse than LiRA and BASE for the majority of benchmarks. However, in the mixed setting, BaVarIA closes the gap to BASE and LiRA and sometimes overtakes them, further suggesting that the Bayesian modelling could increase robustness to this type of mismatch in the reference model population.

Note that the LiRA performance here differs somewhat from that reported in the submitted version; we have used the class-specific global-variance version instead of the shared local-variance version since it has the highest average rank rather than the highest average TPR at 0.1% FPR. Average rank is arguably a better metric to determine which variance computation is overall best across the different datasets, and as a result the LiRA performance is higher on CelebA and ImageNet. The updated LiRA performance is in most cases better, yet still worse than our hierarchical attack.

Weakness 2.

While it is true that Bayesian marginalization does only account for uncertainty within the assumed model, using a mixture of reference model types does not necessarily violate the assumption that the target and reference models follow the same class-conditional distribution. The assumed target model distribution reflects the adversary's prior belief and side-knowledge; a heterogeneous mixture of reference models can naturally represent adversarial uncertainty about the target model. Consequently, a heterogeneous pool of reference models is not inherently inconsistent with our framework. However, it is true that the particular modelling assumptions, such as the Gaussian-Gamma hierarchy used in the record-level attack, may be less accurate when applied with a heterogeneous mixture of reference models. The mechanisms underlying the empirically observed robustness to heterogeneous pools of reference models differ for the record-level and group-level attacks, so we separate our discussion of the two cases below.

Record-level attack. We believe the predictive posterior marginalization may contribute to the observed robustness when using a mixture of reference model types, relative to the baselines. Our Bayesian method places a prior on the distributional parameters, updates their posterior distributions by reference model loss observations, and integrates over these posterior distributions when computing the target posterior-predictive likelihood. The latter integration allows parameter configurations that better explain the observed target loss values to contribute more to the posterior-predictive likelihood, which could provide more flexibility under a heterogeneous reference population. We contrast this with other baseline approaches such as LiRA which commit to a Gaussian distribution fitted by statistics from reference model observations, or BASE which simply compares the target loss to a soft minimum of the reference losses, neither of which has a mechanism similar to that of our Bayesian attack. Of course, this does not guarantee in any way greater robustness under a heterogeneous reference population including mismatched model types. With too large of a difference between target and reference models, the reference-conditioned posterior distribution could assign negligible probability to suitable configurations, rendering the predictive posterior integration ineffective. Rather, this should be understood as a plausible explanation for our empirical observations.

Group-level attack. Our group-level attack combines Bayesian marginalization with estimated plug-in parameters, relying more heavily on the latter compared to our record-level attack. Therefore, we opt for a different approach and explicitly design the attack for robustness under a heterogeneous reference population. Specifically, we introduce a rank-one factor component to capture systematic shifts between observations across models, which could arise from differences in model type. The associated target model coordinate is then inferred from observations on other (non-target) groups, allowing the predictive distribution to adapt to systematic differences between target and reference models. Thus, for the group-level attack, we are primarily attributing the observed robustness in the mixed reference experiments to this rank-one factor component, rather than uncertainty integration. We investigate this experimentally by comparing the performance of the group-level attack with and without the rank-one factor component.

Effect of the rank-one factor. To isolate the factor's contribution, we compared the full group attack with a version retaining the same local Bayesian hierarchy but disabling factor fitting, reference residual refitting, and target-coordinate adaptation. We compare them over the benchmarks included in the paper, with the same settings. The results are presented in Table 5.

Table 5. Group attack with and without the rank-one factor: TPR at 0.1% FPR. Values are percentages, mean ± population standard deviation over ten target models. Differences are the full attack minus the no-factor variant TPR in percentage points. Audit modes “0 train” and “1 train” allow zero and at most one target training record per audited group, respectively.

DatasetReferencesAudit modeTargetFull attackNo factorDifference
CelebAmatched0 trainDDPM14.00 ± 2.0114.86 ± 2.26−0.86
CelebAmatched0 trainFM9.32 ± 2.299.49 ± 2.74−0.16
CelebAmatched1 trainDDPM42.72 ± 4.9844.12 ± 6.94−1.41
CelebAmatched1 trainFM22.98 ± 4.9022.70 ± 5.06+0.28
CelebAmixed0 trainDDPM15.27 ± 1.866.62 ± 0.85+8.65
CelebAmixed0 trainFM9.55 ± 2.056.79 ± 1.17+2.76
CelebAmixed1 trainDDPM43.82 ± 5.8021.86 ± 3.54+21.96
CelebAmixed1 trainFM23.14 ± 3.8314.43 ± 2.14+8.71
VGGFace2matched0 trainFM8.60 ± 2.198.64 ± 2.07−0.04

The factor substantially improves performance with mixed references, while having little effect with matched references. Thus, it achieves its purpose and greatly improves robustness under mixed reference model types. However, we acknowledge that the group-level robustness claims are supported by limited evidence, with only one dataset and two model types due to resource constraints. We will include more results for the VGGFace2 dataset in the final version of the paper, if accepted. We will also clarify our discussion regarding robustness under a mixture of reference model types.

Weakness 3.

Our group-level attack should be seen as a modular semi-Bayesian approximation of a hierarchical model, with an additional rank-one factor component introduced to capture systematic shifts between models, arising from, e.g., mismatched reference models. Since the Bayesian marginalization is more restricted than in the record-level attack, the uncertainty integration qualities are more limited for this particular attack. However, our main goal is to make a strong and reasonably efficient attack, capable of inferring the training membership of individuals even without access to training records and outperforming prior attacks. Questions regarding how well probabilities are calibrated or if full covariance is represented become interesting only insofar as it helps us develop a stronger membership inference attack. Incidentally, this applies to the statements regarding full hierarchical Bayesian models about accounting for uncertainty due to limited queries or reference models; such qualities are important only to the extent to which they improve overall attack performance.

Reviewer_aj4ROpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a hierarchical Bayesian framework for membership inference. It develops record-level and group-level attacks for diffusion and flow-matching models, with empirical evidence suggesting improved detection in several settings.

Critical strengths

The proposed inference procedure is sound under its assumptions, and the group-level adaptation is well designed. The reported improvements on CIFAR10 and CelebA are meaningful.

Critical weaknesses
  1. Lacking baselines, why are we not comparing to BaVarIA?
  2. In the mixed-reference experiments, half of the reference models still belong to the target’s model family. Does the robustness advantage persist when none of the reference models match the target family?
Questions for authors
  1. Should Eq 12 be with τ/r\tau/ r instead of r?
  2. How was the diffusion timestep t∗t^\ast selected for each dataset and model family?
3 comments

Thank you for the review. We provide responses to each of the raised concerns and questions, including a comparison with BaVarIA, more discussion around the mixed-reference experiments, confirmation on the typo in Eq. (12), and a discussion on how t∗t_* was selected.

Weakness 1.

We agree that BaVarIA is a natural baseline and should be included in our paper and we have added its Student-t predictive variant, BaVarIA-t, and its Gaussian stabilized-variance variant, BaVarIA-n, to our baselines. We use κ₀ = 1 and α₀ = 2 for both variants, as suggested in the original paper. The settings are the same as described in the paper and we use the same log-loss samples and reference models for each attack.

All tabulated performance values are percentages, mean ± population standard deviation over ten targets. Bold marks the highest mean in each row. ImageNet is evaluated on a subset of 100k data points.

Table 1. Matched references: TPR at 0.1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online65.25 ± 6.5248.53 ± 4.8326.27 ± 2.7344.50 ± 2.8575.42 ± 4.05
DDPM / CIFAR10 / offline29.21 ± 3.8925.81 ± 3.8317.77 ± 2.2912.60 ± 1.4255.84 ± 6.37
FM / CIFAR10 / online51.45 ± 4.1938.56 ± 5.4122.98 ± 1.6138.88 ± 1.2964.72 ± 2.04
FM / CIFAR10 / offline21.80 ± 2.2118.85 ± 1.9313.62 ± 1.6111.19 ± 1.2946.00 ± 2.89
DDPM / CelebA / online15.30 ± 2.8613.61 ± 1.725.02 ± 0.615.42 ± 0.7017.03 ± 2.32
DDPM / CelebA / offline13.48 ± 1.7013.38 ± 1.705.16 ± 0.502.71 ± 0.1714.67 ± 2.11
FM / CelebA / online43.35 ± 3.9535.00 ± 1.3715.43 ± 0.9723.05 ± 1.6450.51 ± 3.46
FM / CelebA / offline26.54 ± 1.7925.53 ± 1.6012.22 ± 0.767.04 ± 0.2939.36 ± 4.03
FM / ImageNet / online0.94 ± 0.111.31 ± 0.180.98 ± 0.070.64 ± 0.062.03 ± 0.21
FM / ImageNet / offline1.29 ± 0.121.26 ± 0.120.86 ± 0.070.57 ± 0.051.49 ± 0.16

Table 2. Matched references: standardized pAUC at 1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online89.95 ± 2.3786.92 ± 1.6267.78 ± 1.5876.09 ± 1.5591.97 ± 1.55
DDPM / CIFAR10 / offline77.31 ± 2.4276.32 ± 2.3864.51 ± 1.0861.41 ± 0.4484.64 ± 2.73
FM / CIFAR10 / online83.95 ± 1.5181.81 ± 0.7966.48 ± 0.7072.02 ± 0.5986.89 ± 0.93
FM / CIFAR10 / offline72.63 ± 1.0171.57 ± 1.0362.63 ± 0.5960.29 ± 0.4179.69 ± 1.34
DDPM / CelebA / online63.91 ± 2.0062.86 ± 1.4854.23 ± 0.4254.56 ± 0.4664.18 ± 1.61
DDPM / CelebA / offline62.50 ± 1.4762.43 ± 1.4553.76 ± 0.2952.17 ± 0.0862.87 ± 1.58
FM / CelebA / online80.35 ± 2.1378.11 ± 1.0561.17 ± 0.8165.79 ± 1.1482.20 ± 1.71
FM / CelebA / offline73.01 ± 1.5372.60 ± 1.4958.94 ± 0.5155.63 ± 0.0877.01 ± 2.25
FM / ImageNet / online51.38 ± 0.1051.62 ± 0.0850.96 ± 0.0450.48 ± 0.0551.97 ± 0.09
FM / ImageNet / offline51.71 ± 0.1051.69 ± 0.1050.68 ± 0.0450.47 ± 0.0351.83 ± 0.09

Table 3. Mixed references: TPR at 0.1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online48.45 ± 7.2534.82 ± 5.2332.28 ± 3.6244.71 ± 3.6765.62 ± 3.74
DDPM / CIFAR10 / offline24.08 ± 3.7320.73 ± 2.9316.86 ± 2.6615.87 ± 2.0253.77 ± 5.15
DDPM / CelebA / online5.71 ± 1.095.16 ± 0.763.49 ± 0.503.41 ± 0.4813.88 ± 2.23
DDPM / CelebA / offline5.93 ± 0.944.91 ± 0.811.99 ± 0.261.79 ± 0.2513.77 ± 2.31
FM / CIFAR10 / online17.92 ± 3.5531.93 ± 3.8028.99 ± 1.9739.14 ± 3.0555.39 ± 2.75
FM / CelebA / online12.04 ± 2.3418.82 ± 1.5017.40 ± 1.1122.86 ± 1.4535.20 ± 2.34

Table 4. Mixed references: standardized pAUC at 1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online82.49 ± 2.9778.35 ± 2.7071.94 ± 2.0578.81 ± 1.6886.14 ± 1.65
DDPM / CIFAR10 / offline71.43 ± 2.7269.92 ± 2.6463.25 ± 1.3762.22 ± 1.0381.35 ± 2.23
DDPM / CelebA / online56.01 ± 0.8255.22 ± 0.6653.42 ± 0.4053.48 ± 0.4161.18 ± 1.34
DDPM / CelebA / offline55.66 ± 0.7354.94 ± 0.6551.96 ± 0.2151.77 ± 0.1961.83 ± 1.51
FM / CIFAR10 / online65.16 ± 2.2076.60 ± 1.0669.97 ± 0.7974.46 ± 0.7882.56 ± 1.33
FM / CelebA / online62.69 ± 1.8766.56 ± 1.1363.64 ± 0.8967.87 ± 1.1274.23 ± 1.57

Tables 1–4 demonstrate that the BaVarIA attacks are performing substantially worse than our hierarchical Bayesian attack: for example, in the online setting, mean TPR at 0.1% FPR is 75.42% for our attack versus 26.27% and 44.50% for BaVarIA-t and BaVarIA-n on DDPM/CIFAR10 with matched references, and 35.20% versus 17.40% and 22.86%, respectively, on FM/CelebA with mixed references. In the matched setting, BaVarIA is even worse than LiRA and BASE for the majority of benchmarks. However, in the mixed setting, BaVarIA closes the gap to BASE and LiRA and sometimes overtakes them, further suggesting that the Bayesian modelling could increase robustness to this type of mismatch in the reference model population.

Weakness 2.

Although we cannot test a mixture of reference model types that excludes the target type currently since it would require three different model types, we have no reason to believe the same effect would not apply. It is worthwhile to first clarify why we consider using a set of mixed type reference models.

The membership inference literature has primarily investigated the robustness of attacks against incorrect assumptions by the adversary regarding the target model distribution (e.g., due to assuming the wrong model architecture, hyperparameter settings, or training procedure) (Carlini et al., Membership Inference Attacks From First Principles, 2022; Lassila et al., Practical Bayes-Optimal Membership Inference Attacks, 2025). We argue that it is more relevant to investigate robustness or attack performance under adversarial uncertainty about the target model distribution. An adversary uncertain about, e.g., target model type, architecture, or training procedure, should arguably not rely on one type of reference model. In terms of Bayesian modelling, this represents being certain but wrong. Rather, the uncertainty should arguably be represented by widening the reference model population to include different alternatives weighted by their prior beliefs.

Now, does it matter that one of the model types matches the target model? It likely does, or at least that there are a few reference models that are reasonably close to represent the target model distribution fairly accurately. Likewise, if none of the reference models represent the target model well, the reference models might be of little use for membership inference.

Question 1.

We confirm that there is a typo in equation 12 and it should use κ=τ/r\kappa=\tau/r rather than τ\tau. We will update the paper and fix this.

Question 2.

We did forget to mention how we select t∗t_*. In the case of DDPM targets, we set t∗=0.1Tt_*=0.1T, where TT is the total number of steps in the noise chain such that t=0t=0 is no noise and t=Tt=T is pure noise. We motivate our choice by referring to Carlini et al., Extracting Training Data from Diffusion Models (2023, Figure 9): their LiRA experiments on CIFAR-10 found the strongest attacks at t∈[50,300]t\in[50,300] with T=1000T=1000, and they selected t=100t=100 for subsequent experiments. We likewise use t∗=0.1Tt_*=0.1T for all datasets when querying loss values from DDPM models, without tuning it.

For the flow-matching model, we are not aware of any prior work involving membership inference attacks on this type of model. Thus, we used a noise-level sweep on CelebA with the offline BASE attack to inform our choice. Performance was strongest around noise levels 0.30–0.40: AUC peaked at 0.30, TPR at 1% FPR at 0.35, and TPR at 0.1% FPR at 0.40. We chose 0.33≈1/30.33\approx 1/3 and used this value for all loss queries. In our flow-matching convention, noise level is 1−t∗1-t_*, so this corresponds to t∗=0.67t_*=0.67.

The table below shows selected levels from the sweep over {0,0.05,…,1}\{0,0.05,\ldots,1\}. Results are record-level attack metrics, reported as mean ± standard deviation across 10 FM CelebA targets; all metric values are percentages.

Noise levelAUCTPR at 1% FPRTPR at 0.1% FPR
0.0053.34 ± 0.191.42 ± 0.060.18 ± 0.03
0.1081.52 ± 1.3615.75 ± 1.874.04 ± 0.74
0.2088.82 ± 1.4530.75 ± 3.3910.46 ± 1.30
0.2590.03 ± 1.4334.66 ± 3.6712.90 ± 1.63
0.3090.47 ± 1.4336.33 ± 3.8414.02 ± 1.63
0.3590.35 ± 1.4436.96 ± 3.9614.29 ± 1.66
0.4089.77 ± 1.4835.99 ± 3.8414.50 ± 2.03
0.4588.72 ± 1.5234.13 ± 3.5213.51 ± 1.66
0.5087.18 ± 1.5831.31 ± 3.2712.28 ± 1.52
0.6081.92 ± 1.4923.13 ± 2.478.87 ± 1.07
0.7072.77 ± 1.3013.32 ± 1.414.77 ± 0.64
0.8060.29 ± 0.594.59 ± 0.431.29 ± 0.19
0.9051.29 ± 0.141.34 ± 0.050.18 ± 0.02
1.0050.24 ± 0.141.01 ± 0.050.11 ± 0.01

We will update the paper to include this explanation on how we select the query time t∗t_*.

Dear reviewer,

We have posted responses to all the points raised in the reviews. We believe we have addressed all of your comments and extend our experiments with the BaVarIA attack. Our attack substantially outperforms BaVarIA, showing the need for a carefully designed Bayesian attack model. As the discussion period ends on November 18, we would be grateful if you could let us know whether our responses adequately answers your questions and concerns. We would be happy to provide any further clarification or answer any additional questions.

Thank you for your time and consideration.

Comments on the paper

4 comments

Thank you for the careful assessment and for identifying relevant questions for us to address. We acknowledge that BaVarIA is a relevant baseline to include since it also considers a Bayesian inference approach to the membership inference problem. We have therefore evaluated both BaVarIA variants in the same experimental setting used to generate the results in our paper. We also explain the group-reference mask and the implementation specific details of the group-level attack.

Weakness 1.

We agree that BaVarIA (eq (8)) is directly relevant prior work and should be included in the paper. We have added its Student-t predictive variant, BaVarIA-t, and its Gaussian stabilized-variance variant, BaVarIA-n. We use κ₀ = 1 and α₀ = 2 for both variants, as suggested in the original paper. The settings are the same as described in the paper and we use the same log-loss samples and reference models for each attack.

All tabulated performance values are percentages, mean ± population standard deviation over ten targets. Bold marks the highest mean in each row. ImageNet is evaluated on a subset of 100k data points.

Table 1. Matched references: TPR at 0.1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online65.25 ± 6.5248.53 ± 4.8326.27 ± 2.7344.50 ± 2.8575.42 ± 4.05
DDPM / CIFAR10 / offline29.21 ± 3.8925.81 ± 3.8317.77 ± 2.2912.60 ± 1.4255.84 ± 6.37
FM / CIFAR10 / online51.45 ± 4.1938.56 ± 5.4122.98 ± 1.6138.88 ± 1.2964.72 ± 2.04
FM / CIFAR10 / offline21.80 ± 2.2118.85 ± 1.9313.62 ± 1.6111.19 ± 1.2946.00 ± 2.89
DDPM / CelebA / online15.30 ± 2.8613.61 ± 1.725.02 ± 0.615.42 ± 0.7017.03 ± 2.32
DDPM / CelebA / offline13.48 ± 1.7013.38 ± 1.705.16 ± 0.502.71 ± 0.1714.67 ± 2.11
FM / CelebA / online43.35 ± 3.9535.00 ± 1.3715.43 ± 0.9723.05 ± 1.6450.51 ± 3.46
FM / CelebA / offline26.54 ± 1.7925.53 ± 1.6012.22 ± 0.767.04 ± 0.2939.36 ± 4.03
FM / ImageNet / online0.94 ± 0.111.31 ± 0.180.98 ± 0.070.64 ± 0.062.03 ± 0.21
FM / ImageNet / offline1.29 ± 0.121.26 ± 0.120.86 ± 0.070.57 ± 0.051.49 ± 0.16

Table 2. Matched references: standardized pAUC at 1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online89.95 ± 2.3786.92 ± 1.6267.78 ± 1.5876.09 ± 1.5591.97 ± 1.55
DDPM / CIFAR10 / offline77.31 ± 2.4276.32 ± 2.3864.51 ± 1.0861.41 ± 0.4484.64 ± 2.73
FM / CIFAR10 / online83.95 ± 1.5181.81 ± 0.7966.48 ± 0.7072.02 ± 0.5986.89 ± 0.93
FM / CIFAR10 / offline72.63 ± 1.0171.57 ± 1.0362.63 ± 0.5960.29 ± 0.4179.69 ± 1.34
DDPM / CelebA / online63.91 ± 2.0062.86 ± 1.4854.23 ± 0.4254.56 ± 0.4664.18 ± 1.61
DDPM / CelebA / offline62.50 ± 1.4762.43 ± 1.4553.76 ± 0.2952.17 ± 0.0862.87 ± 1.58
FM / CelebA / online80.35 ± 2.1378.11 ± 1.0561.17 ± 0.8165.79 ± 1.1482.20 ± 1.71
FM / CelebA / offline73.01 ± 1.5372.60 ± 1.4958.94 ± 0.5155.63 ± 0.0877.01 ± 2.25
FM / ImageNet / online51.38 ± 0.1051.62 ± 0.0850.96 ± 0.0450.48 ± 0.0551.97 ± 0.09
FM / ImageNet / offline51.71 ± 0.1051.69 ± 0.1050.68 ± 0.0450.47 ± 0.0351.83 ± 0.09

Table 3. Mixed references: TPR at 0.1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online48.45 ± 7.2534.82 ± 5.2332.28 ± 3.6244.71 ± 3.6765.62 ± 3.74
DDPM / CIFAR10 / offline24.08 ± 3.7320.73 ± 2.9316.86 ± 2.6615.87 ± 2.0253.77 ± 5.15
DDPM / CelebA / online5.71 ± 1.095.16 ± 0.763.49 ± 0.503.41 ± 0.4813.88 ± 2.23
DDPM / CelebA / offline5.93 ± 0.944.91 ± 0.811.99 ± 0.261.79 ± 0.2513.77 ± 2.31
FM / CIFAR10 / online17.92 ± 3.5531.93 ± 3.8028.99 ± 1.9739.14 ± 3.0555.39 ± 2.75
FM / CelebA / online12.04 ± 2.3418.82 ± 1.5017.40 ± 1.1122.86 ± 1.4535.20 ± 2.34

Table 4. Mixed references: standardized pAUC at 1% FPR.

Model / dataset / modeBASELiRABaVarIA-tBaVarIA-nOurs
DDPM / CIFAR10 / online82.49 ± 2.9778.35 ± 2.7071.94 ± 2.0578.81 ± 1.6886.14 ± 1.65
DDPM / CIFAR10 / offline71.43 ± 2.7269.92 ± 2.6463.25 ± 1.3762.22 ± 1.0381.35 ± 2.23
DDPM / CelebA / online56.01 ± 0.8255.22 ± 0.6653.42 ± 0.4053.48 ± 0.4161.18 ± 1.34
DDPM / CelebA / offline55.66 ± 0.7354.94 ± 0.6551.96 ± 0.2151.77 ± 0.1961.83 ± 1.51
FM / CIFAR10 / online65.16 ± 2.2076.60 ± 1.0669.97 ± 0.7974.46 ± 0.7882.56 ± 1.33
FM / CelebA / online62.69 ± 1.8766.56 ± 1.1363.64 ± 0.8967.87 ± 1.1274.23 ± 1.57

Tables 1–4 demonstrate that the BaVarIA attacks are performing significantly worse than our hierarchical Bayesian attack. In the matched setting, BaVarIA is in worse than LiRA and BASE for the majority of benchmark. However, in the mixed setting, BaVarIA closes the gap to BASE and LiRA and sometimes overtakes them, further suggesting that the Bayesian modelling could increase robustness to this type of mismatches in the reference model population.

Note, the LiRA performance here differ somewhat from the submitted version; we have used the class-specific global-variance version instead of the shared local-variance since it has the highest average rank rather than the highest average TPR at 0.1% FPR. Average rank is arguably a better metric to determine which variance computation is overall best across the different datasets, and as a result the LiRA performance is higher on CelebA and ImageNet. The updated LiRA performance are in most cases better, yet still worse than our hierarchical attack.

Weakness 2.

In the paper we outline the group-level attack for a set of reference models that are "out" with respect to all groups. The reason is that this simplifies the notation and hopefully makes the ideas more clear. In the experiments, for practical reasons, we have reference models such that each group (individual) is represented in the training data for half of the models. Thus, for each group, some reference models will be in-models and have to be filtered out since we only want to use observations from out-models when estimating the null-distribution, and which models to filter depend on the group in question. The implementation uses a mask to accomplish this. Each reference model retains the same index and coordinate across groups, although its observations are used only for groups absent from its training data. BASE, LiRA, and our attack receive the same groups for auditing, queried records, and reference models. Our method additionally uses observations across groups to estimate the shared model adjustment.

Clarification of Appendix D.4. The implementation use alternating ridge updates followed by centering and rescaling the model coordinates. Let On\mathcal O_n denote the out-reference models for point nn's group, and use rnmr_{nm} and wn=sn−2w_n=s_n^{-2} from Eq. (112). Before normalization, the ridge updates minimize, one block at a time,

$ Q(\lambda,z)= \frac12\sum_n\sum_{m\in\mathcal O_n}w_n(r_{nm}-\lambda_nz_m)^2 +\frac12\sum_n w_n\lambda_n^2 +\frac12\sum_m z_m^2. $

This differs somewhat from the objective in Eq. (113): the loading penalty is weighted by the local predictive precision, and the coordinates are penalized before being normalized. The subsequent normalization is a separate step, so the complete procedure is not alternating minimization of Eq. (113). We will correct the description in Appendix D.4 accordingly.

As for the rank-one latent factor, our motivation to include it was precisely to counteract systematic shifts of outputs between different models, primarily arising in situations where models can be of different types. It should be seen as a modeling choice allowing us to use a broader range of reference models, which is particularly relevant when under adversarial uncertainty about the target model distribution. Our claim is not that Bayesian marginalization is responsible for the enhanced performance in the mixed reference benchmarks, but rather that this rank-one factor successfully handles this case as it was designed to do.

Effect of the rank-one factor. To isolate the factor's contribution, we compared the full group attack with a version retaining the same local Bayesian hierarchy but disabling factor fitting, reference residual refitting, and target-coordinate adaptation. We compare them over the benchmarks included in the paper, with the same settings.

Table 5. Group attack with and without the rank-one factor: TPR at 0.1% FPR. Values are percentages, mean ± population standard deviation over ten target models. Differences are the full attack minus the no-factor variant TPR in percentage points. Audit modes “0 train” and “1 train” allow zero and at most one target training record per audited group, respectively.

DatasetReferencesAudit modeTargetFull attackNo factorDifference
CelebAmatched0 trainDDPM14.00 ± 2.0114.86 ± 2.26−0.86
CelebAmatched0 trainFM9.32 ± 2.299.49 ± 2.74−0.16
CelebAmatched1 trainDDPM42.72 ± 4.9844.12 ± 6.94−1.41
CelebAmatched1 trainFM22.98 ± 4.9022.70 ± 5.06+0.28
CelebAmixed0 trainDDPM15.27 ± 1.866.62 ± 0.85+8.65
CelebAmixed0 trainFM9.55 ± 2.056.79 ± 1.17+2.76
CelebAmixed1 trainDDPM43.82 ± 5.8021.86 ± 3.54+21.96
CelebAmixed1 trainFM23.14 ± 3.8314.43 ± 2.14+8.71
VGGFace2matched0 trainFM8.60 ± 2.198.64 ± 2.07−0.04

The factor substantially improves performance with mixed references, while only slightly reducing mean performance with matched references. Note that our group-level attack is not doing a full Bayesian marginalization, but instead uses a modular empirical-Bayes procedure with partial uncertainty propagation. We estimate hyperparameters from out-reference observations, integrate local mean and variance uncertainty into predictive moments, and fit the reference factor separately. The group score uses Gaussian approximations with these moments and integrates over the cross-fitted posterior of the target coordinate. This is a semi-Bayesian construction: uncertainty in the fitted hyperparameters and reference factor is not jointly integrated out. Thus, we are not attributing the success under mixed reference models to marginalization, but rather to our explicit rank-one factor component. We will revise the paper and clarify this point.

We also acknowledge that the group-level robustness claims to partly mixed reference model types are supported by limited evidence, with only one dataset and two model types due to resource constraints. We will include more results for the VGGFace2 dataset in the final version of the paper, if accepted.

Question 1.

See response to Weakness 1.

Question 2.

See response to Weakness 2.

Question 3.

We confirm that there is a typo in equation 12 and in its current form it is valid only for r=1. The experiments used a r determined empirically as described in appendix B.3. The implemented predictive expression uses $\kappa=\tau/r$, with mean (κU+λξ)/(Mκ+λ)(\kappa U+\lambda\xi)/(M\kappa+\lambda) and variance $r/\tau+(M\kappa+\lambda)^{-1}$, as in Appendix B.5. We will revise the paper to make sure it is included everywhere.

In the experiments, we used 128 Gibbs draws after 64 warm-up iterations, with 65 quadrature nodes for matched-online and offline experiments and 50 for mixed-online experiments. (There was no particular reason to pick 65 or 50 quadrature nodes, and as long as it is reasonably large, >30>30, it has little impact on the results.) Increasing the Gibbs iterations further could potentially result in small improvements at the cost of more computation, (yet audits are cheep at the current setting and audits run in one to a few minutes so increasing them is possible). We used the shared-τ\tau version of our record-level attack for the online experiments. We will include these details in the Experimental Details appendix E.

Dear reviewer,

We have posted responses to all the points raised in the reviews. We addressed all of your comments and extend our experiments, with the BaVarIA attack added to the baselines and an ablation to isolate the effect of the rank-one factor in our group-level attack. Our attack substantially outperforms BaVarIA, demonstrating the need for better Bayesian modeling. The rank-one factor ablation shows its effectiveness in dealing with a heterogeneous reference population. As the discussion period ends on November 18, we would be grateful if you could let us know whether our responses address the points you raised. We would be happy to provide any further clarification or answer any additional questions.

Thank you for your time and consideration.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.