- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes PaFMIA, a membership inference attack against denoising generative models that jointly exploits prediction residuals across spatial-frequency bands and generative-path positions. The method uses DCT-based frequency decomposition and constructs either full frequency profiles or learned frequency aggregations for membership classification. The paper provides theoretical analyses of information compression, complementary path positions, and membership signals in a linear rectified-flow model. Experiments on four image datasets and RF, VP-FM, and DDPM models demonstrate improvements over several existing attacks.
Critical strengths
- The paper combines spatial-frequency information and generative-path positions for membership inference. This provides a meaningful extension of existing frequency-based and path-based attacks. The theoretical analysis also offers useful explanations for why membership signals may vary across frequency bands and path positions.
- The evaluation covers four datasets, multiple attack baselines, and different generative objectives. In particular, PaFMIA achieves 67.41% TPR at 1% FPR on TinyImageNet with 32 queries, compared with 19.78% for DIME-RF and 49.72% for the Global-profile MLP (Table 1). The ablations in Tables 2–3 and the appendix further support the contributions of frequency and path information.
Critical weaknesses
- The methodological novelty is somewhat limited. FreMIA (ICML 2026) has already explored frequency information for diffusion-model membership inference, while Sesmat et al. (ICML 2026) studied membership leakage along the flow-matching interpolation path and multi-position attacks. PaFMIA mainly combines these two established directions, and parts of its theoretical analysis follow standard information-compression and Gaussian-discrimination results. Although the experiments demonstrate the benefit of the joint representation, the distinct technical contribution could be better established.
- The generalizability and practical applicability remain unclear. The main experiments use unconditional U-Nets with the same architecture, trained for 500,000 steps on only 20,000 images per model (Appendix A.1), which may lead to strong memorization. Evaluations across different training stages, dataset sizes, and larger conditional generators are missing. Moreover, source and target models share the same candidate population, and some targets were involved in earlier exploratory analyses. Although final attack configurations are selected on source data and no direct test-label leakage is evident, evaluation on completely fresh targets and disjoint data populations would strengthen the conclusions. The attack also requires access to the model's prediction interface rather than a standard generation-only API.
Questions for authors
- Could the authors clarify the main distinct contributions of PaFMIA compared with FreMIA and Sesmat et al.? Beyond combining two previously studied feature dimensions, what are the key new technical or theoretical contributions?
- The models are trained for 500,000 steps on 20,000 images. How sensitive are the improvements over global-error baselines to training duration or training-set size? Are additional results available for less heavily trained models?
- Appendix A.1 mentions that some target models were involved in earlier exploratory analyses. Could the authors clarify whether these analyses influenced the final attack design, and discuss the expected transferability to entirely fresh target models or disjoint candidate populations?
5 comments
Response to Reviewer 91Rx: W2&Q2: Training Duration, Dataset Size, and Pretrained Models
Q2&W2. Sensitivity to training duration and training-set size
We evaluated RF on four datasets and DDPM on two at 20k, 40k and 100k updates, keeping the training set at 20,000 images per model. The 500k rows give the original RF results (Table 1) and the CIFAR-10 DDPM results on the Table 4 checkpoints. Attack configurations are selected on the source and frozen before target evaluation.
Entries are mean TPR at 1% FPR (%) across two targets, with 32 predictions per candidate for each method. Bold marks the highest value in each row, including ties. Error averages squared prediction errors. The Rao–Moyer statistic , called “RF error” in Table 1, averages velocity predictions before computing the error. RF uses DIME-RF and FreMIA-RF. DDPM uses Loss-MC, DIME and FreMIA-Naive-MC. A dash marks an unreported or inapplicable result.
| Model / dataset | Stage | Error (MC baseline) | Rao–Moyer (ECCV 2026) | DIME (arXiv 2026) | FreMIA (ICML 2026) | PaFMIA (aggregation, ours) |
|---|---|---|---|---|---|---|
| RF / CIFAR-10 | 20k | 1.03 | 1.03 | 1.03 | 1.09 | 0.99 |
| RF / CIFAR-10 | 40k | 1.08 | 0.99 | 1.00 | 1.11 | 1.19 |
| RF / CIFAR-10 | 100k | 1.36 | 1.62 | 5.54 | 2.06 | 13.59 |
| RF / CIFAR-10 | 500k | — | 59.90 | 88.35 | 65.12 | 97.13 |
| RF / SVHN | 20k | 1.15 | 1.08 | 1.22 | 1.13 | 1.12 |
| RF / SVHN | 40k | 1.16 | 1.04 | 2.33 | 0.98 | 1.91 |
| RF / SVHN | 100k | 1.75 | 1.76 | 13.09 | 3.29 | 14.54 |
| RF / SVHN | 500k | — | 67.80 | 94.69 | 89.76 | 95.75 |
| RF / TinyImageNet | 20k | 0.96 | 1.00 | 1.03 | 0.98 | 1.03 |
| RF / TinyImageNet | 40k | 0.96 | 1.02 | 1.04 | 0.99 | 1.07 |
| RF / TinyImageNet | 100k | 0.97 | 1.01 | 0.94 | 1.14 | 1.29 |
| RF / TinyImageNet | 500k | — | 5.51 | 19.78 | 15.30 | 65.69 |
| RF / CelebA | 20k | 1.08 | 1.18 | 1.02 | 0.95 | 1.09 |
| RF / CelebA | 40k | 1.06 | 1.33 | 0.94 | 1.05 | 1.09 |
| RF / CelebA | 100k | 1.64 | 2.28 | 2.19 | 1.78 | 2.80 |
| RF / CelebA | 500k | — | 38.63 | 73.12 | 38.86 | 89.60 |
| DDPM / CIFAR-10 | 20k | 1.03 | — | 0.94 | 1.10 | 0.95 |
| DDPM / CIFAR-10 | 40k | 1.05 | — | 1.00 | 1.19 | 1.65 |
| DDPM / CIFAR-10 | 100k | 1.59 | — | 5.50 | 2.02 | 26.67 |
| DDPM / CIFAR-10 | 500k | 41.71 | — | 96.53 | 58.68 | 99.55 |
| DDPM / TinyImageNet | 20k | 0.96 | — | 1.06 | 0.94 | 1.04 |
| DDPM / TinyImageNet | 40k | 0.91 | — | 0.986 | 0.993 | 0.98 |
| DDPM / TinyImageNet | 100k | 0.94 | — | 0.81 | 1.10 | 2.69 |
At 20k and 40k, most methods detect around 1% of members at 1% FPR. At 100k, PaFMIA aggregation reaches 26.67% on DDPM CIFAR-10 and 2.69% on DDPM TinyImageNet, compared with 24.81% and 1.47% for the independently tuned global-profile MLP. On DDPM TinyImageNet it also exceeds global LR (1.81%) on both targets, by 0.90 and 0.86 percentage points (paired 95% CIs [0.39, 1.46] and [0.31, 1.44]). At 500k, the largest RF margin over global-profile MLP is on TinyImageNet, where aggregation reaches 65.69% versus 49.72%.
For a smaller training set, Response 3 reports fresh TinyImageNet models trained on 5,000 images for 50k updates. Under the two class-diversity settings, full-profile PaFMIA reaches 72.88% and 73.18% versus 28.80% and 34.63% for the global profile, with both readouts selected on the source.
Practical scope and larger generators
PaFMIA requires time-conditioned predictions, which open-weight models expose. Generation-only APIs expose only the final images. We additionally tested released 256×256 unconditional ADM and conditional SD1.4 without fine-tuning either generator.
ADM. With auxiliary labeled examples disjoint from the test set, full-profile PaFMIA achieves 60.02% AUROC and 3.865% TPR@1%FPR, versus global LR's 56.23% and 3.230%, both at 32 predictions. The TPR gain is 0.635 points (95% CI [0.080, 1.225]). Separately, frozen CIFAR-10 DDPM-to-ADM transfer is near chance (47.68% AUROC).
SD1.4. We use the trained empty-prompt branch so that each candidate image can be evaluated without a caption. The model was trained with 10% conditioning dropout. On LAION-mi, with auxiliary labels and a disjoint 2,000-member/4,000-nonmember test set, aggregation LR achieves 54.09% AUROC versus 52.41% for global LR, 52.93% for global MLP and 52.65% for MC loss, all at 32 predictions. Compared with global LR, aggregation gains 1.67 AUROC points (95% CI [0.18, 3.18]) and 0.60 points in TPR@1% FPR (1.85% vs. 1.25%, 95% CI for the gain [−0.35, 1.60]).
Response to Reviewer 91Rx: W2&Q3: Exploratory Analyses and Transfer to Fresh Targets
Q3&W2. Exploratory analyses and validation on new targets
How the original targets shaped our study. In early tests on two CIFAR-10 and two SVHN rectified-flow targets, reducing the budget from 96 to 32 queries lowered detection more on SVHN for our band-based classifier. After seeing those test results, we investigated how to distribute a fixed query budget across path positions. For the comparisons reported in the paper, we selected the schedules and fitted the classifiers on source data alone, then applied them unchanged to the targets.
Transfer to new targets. We trained new TinyImageNet models under two settings, using either 20 or 150 background training classes and the same 50 evaluation classes. In each setting, we trained one source and two targets from scratch on 5,000 images per model for 50,000 steps. Within this experiment, the source and target image pools were disjoint, as verified by image IDs, file hashes and decoded-pixel hashes. The two targets in each setting shared the target pool. We froze the attack and its calibration on the source before collecting target predictions. PaFMIA’s advantage over the global-error baseline carried over to all four newly trained TinyImageNet targets.
Results with logistic regression and MLPs. All methods used the same eight path positions and four predictions per position (32 queries per image). We fixed logistic regression (LR) for the primary comparison so that only the input representation changed. In a second comparison, source validation chose between LR and an MLP with 16 hidden units. It selected the MLP for both representations in both settings. The table reports mean TPR at 1% FPR (%) across the two targets in each setting.
| Background training classes | Global (LR) | PaFMIA full (LR) | Global (MLP) | PaFMIA full (MLP) |
|---|---|---|---|---|
| 20 | 24.83 | 61.55 | 28.80 | 72.88 |
| 150 | 26.20 | 61.73 | 34.63 | 73.18 |
Gains on individual targets. Adding the band energies to the global profile improved detection on each of the four newly trained targets. The gains ranged from 33.55 to 37.60 percentage points with LR and from 38.30 to 45.50 points with the MLP. As a further check, we expanded the global baseline’s search to include raw, log and combined inputs and a larger MLP. Source validation selected the 64–32 MLP with log inputs in both settings. It reached 30.65% and 31.20%, respectively, on the same query schedule.
Response to Reviewer 91Rx: W1&Q1: What PaFMIA Adds to Frequency- and Path-Based Attacks
Thank you for your careful review of our paper. We address your questions in three replies. We first discuss what PaFMIA adds to prior work, then report results on training duration and dataset size, and finally describe our evaluation on fresh targets.
W1&Q1. Distinct technical and theoretical contributions
Two images can have the same global prediction error at every position along the generative path, yet differ in which spatial frequencies account for that error. PaFMIA measures these differences and learns how they distinguish training images from held-out images.
FreMIA attenuates high frequencies to reduce variation in a residual score, then applies a threshold to that score. Sesmat et al., whose evaluation emphasizes audio, train an MLP on the sequence of global reconstruction errors along the path. PaFMIA retains the error in each spatial-frequency band at each selected position, including high-frequency responses. This lets the attack learn where along the path each band carries membership evidence for images. FreMIA · Sesmat et al.
To see where the global profile loses frequency detail, consider how it summarizes the residual from a single prediction. At position t, let Et,b be the mean residual energy in band b and ωb its fraction of coefficients. Global error is the weighted average
Gt=∑bωbEt,b.
A shift in energy between bands can leave Gt unchanged. PaFMIA’s full profile keeps both Gt and the individual band energies. Its aggregation variant keeps Gt and a second average whose band weights are learned on the source model for each position. Table 2 shows how much information the global average can hide. On TinyImageNet, with the same two path positions, the same 32 predictions and the same logistic regression fitting procedure, adding the band energies raises mean TPR at 1% FPR from 7.68% to 36.78%.
The next question is whether a band should carry the same weight everywhere along the path. In a separate TinyImageNet comparison, we fixed eight positions and 32 queries. Using the same band weights at every position gave 42.18% TPR at 1% FPR. Keeping the relative weights of the bands fixed but learning one multiplier per position raised TPR to 54.75%. Learning the relative band weights separately at each position raised it to 57.91%. Most of the gain came from learning how strongly to weight each position. The remaining gain came from changing the balance among bands within a position.
Our theory explains why position can matter. In a linear rectified-flow model evaluated with fresh noise, we derive the expected error gap between members and nonmembers along the path (C1). For frequency directions with different variances, the limiting gap peaks at different positions (C2).
The theory also explains why we study both aggregation and the full profile. In the model of B1, one suitably weighted band average per position, together with global error, preserves all the information needed for an optimal membership decision. In B2, the affected path–frequency coordinate varies across examples. Under that model, any fixed linear compression to fewer coordinates loses membership information that the full profile retains.