acceptodds
Desk rejected at ICLR 2027

Beyond Global Error: Path–Frequency Membership Inference in Denoising Generative Models

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Denoising generative models expose training membership through their predictions on noisy inputs. Existing attacks often summarize prediction residuals into error statistics, compressing the available evidence before membership discrimination. Focusing on rectified flow, we uncover a richer structure: member–nonmember differences vary across spatial-frequency bands, and these frequency responses change along the interpolation path. Global error conceals how residual energy is distributed across bands, while isolated path positions miss complementary responses. To exploit this structure, we introduce PaFMIA (Path–Frequency Membership Inference Attack), which represents queried residuals jointly across path positions and spatial frequencies. PaFMIA either retains the complete band measurements or learns position-dependent weights that aggregate them, then fits a lightweight membership classifier using labeled source-model examples. The resulting attack learns from both the distribution of residual energy and its variation along the path. All frequency measurements reuse existing predictions, adding no generator queries. Our mathematical analysis characterizes information lost through aggregation, identifies conditions for complementary positions to improve detection at a fixed query budget, and derives frequency-dependent membership responses in a linear rectified-flow model. Experiments on four image datasets demonstrate the value of this representation: at 32 queries, PaFMIA reaches 67.41% TPR at 1% FPR on TinyImageNet, compared with 19.78% for DIME adapted to rectified flow. Controlled ablations identify the contributions of frequency resolution and joint positions, while variance-preserving flow matching and DDPM experiments extend the approach across flow and diffusion objectives.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Membership inference attacks, Generative model privacy, Rectified flow, Path–frequency representations, Flow matching, Diffusion models

Reviews

Among:
Shape:
1234this paper, Top 32%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.50 over 4 reviews, above 68% of the others.
Reviewer_91RxOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PaFMIA, a membership inference attack against denoising generative models that jointly exploits prediction residuals across spatial-frequency bands and generative-path positions. The method uses DCT-based frequency decomposition and constructs either full frequency profiles or learned frequency aggregations for membership classification. The paper provides theoretical analyses of information compression, complementary path positions, and membership signals in a linear rectified-flow model. Experiments on four image datasets and RF, VP-FM, and DDPM models demonstrate improvements over several existing attacks.

Critical strengths
  1. The paper combines spatial-frequency information and generative-path positions for membership inference. This provides a meaningful extension of existing frequency-based and path-based attacks. The theoretical analysis also offers useful explanations for why membership signals may vary across frequency bands and path positions.
  2. The evaluation covers four datasets, multiple attack baselines, and different generative objectives. In particular, PaFMIA achieves 67.41% TPR at 1% FPR on TinyImageNet with 32 queries, compared with 19.78% for DIME-RF and 49.72% for the Global-profile MLP (Table 1). The ablations in Tables 2–3 and the appendix further support the contributions of frequency and path information.
Critical weaknesses
  1. The methodological novelty is somewhat limited. FreMIA (ICML 2026) has already explored frequency information for diffusion-model membership inference, while Sesmat et al. (ICML 2026) studied membership leakage along the flow-matching interpolation path and multi-position attacks. PaFMIA mainly combines these two established directions, and parts of its theoretical analysis follow standard information-compression and Gaussian-discrimination results. Although the experiments demonstrate the benefit of the joint representation, the distinct technical contribution could be better established.
  2. The generalizability and practical applicability remain unclear. The main experiments use unconditional U-Nets with the same architecture, trained for 500,000 steps on only 20,000 images per model (Appendix A.1), which may lead to strong memorization. Evaluations across different training stages, dataset sizes, and larger conditional generators are missing. Moreover, source and target models share the same candidate population, and some targets were involved in earlier exploratory analyses. Although final attack configurations are selected on source data and no direct test-label leakage is evident, evaluation on completely fresh targets and disjoint data populations would strengthen the conclusions. The attack also requires access to the model's prediction interface rather than a standard generation-only API.
Questions for authors
  1. Could the authors clarify the main distinct contributions of PaFMIA compared with FreMIA and Sesmat et al.? Beyond combining two previously studied feature dimensions, what are the key new technical or theoretical contributions?
  2. The models are trained for 500,000 steps on 20,000 images. How sensitive are the improvements over global-error baselines to training duration or training-set size? Are additional results available for less heavily trained models?
  3. Appendix A.1 mentions that some target models were involved in earlier exploratory analyses. Could the authors clarify whether these analyses influenced the final attack design, and discuss the expected transferability to entirely fresh target models or disjoint candidate populations?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 91Rx: W1&Q1: What PaFMIA Adds to Frequency- and Path-Based Attacks

Thank you for your careful review of our paper. We address your questions in three replies. We first discuss what PaFMIA adds to prior work, then report results on training duration and dataset size, and finally describe our evaluation on fresh targets.

W1&Q1. Distinct technical and theoretical contributions

Two images can have the same global prediction error at every position along the generative path, yet differ in which spatial frequencies account for that error. PaFMIA measures these differences and learns how they distinguish training images from held-out images.

FreMIA attenuates high frequencies to reduce variation in a residual score, then applies a threshold to that score. Sesmat et al., whose evaluation emphasizes audio, train an MLP on the sequence of global reconstruction errors along the path. PaFMIA retains the error in each spatial-frequency band at each selected position, including high-frequency responses. This lets the attack learn where along the path each band carries membership evidence for images. FreMIA · Sesmat et al.

To see where the global profile loses frequency detail, consider how it summarizes the residual from a single prediction. At position tt, let Et,bE_{t,b} be the mean residual energy in band bb and ωb\omega_b its fraction of coefficients. Global error is the weighted average

Gt=∑bωbEt,b.G_t=\sum_b\omega_b E_{t,b}.

A shift in energy between bands can leave GtG_t unchanged. PaFMIA’s full profile keeps both GtG_t and the individual band energies. Its aggregation variant keeps GtG_t and a second average whose band weights are learned on the source model for each position. Table 2 shows how much information the global average can hide. On TinyImageNet, with the same two path positions, the same 32 predictions and the same logistic regression fitting procedure, adding the band energies raises mean TPR at 1% FPR from 7.68% to 36.78%.

The next question is whether a band should carry the same weight everywhere along the path. In a separate TinyImageNet comparison, we fixed eight positions and 32 queries. Using the same band weights at every position gave 42.18% TPR at 1% FPR. Keeping the relative weights of the bands fixed but learning one multiplier per position raised TPR to 54.75%. Learning the relative band weights separately at each position raised it to 57.91%. Most of the gain came from learning how strongly to weight each position. The remaining gain came from changing the balance among bands within a position.

Our theory explains why position can matter. In a linear rectified-flow model evaluated with fresh noise, we derive the expected error gap between members and nonmembers along the path (C1). For frequency directions with different variances, the limiting gap peaks at different positions (C2).

The theory also explains why we study both aggregation and the full profile. In the model of B1, one suitably weighted band average per position, together with global error, preserves all the information needed for an optimal membership decision. In B2, the affected path–frequency coordinate varies across examples. Under that model, any fixed linear compression to fewer coordinates loses membership information that the full profile retains.

AuthorsOpenReview ↗

Response to Reviewer 91Rx: W2&Q2: Training Duration, Dataset Size, and Pretrained Models

Q2&W2. Sensitivity to training duration and training-set size

We evaluated RF on four datasets and DDPM on two at 20k, 40k and 100k updates, keeping the training set at 20,000 images per model. The 500k rows give the original RF results (Table 1) and the CIFAR-10 DDPM results on the Table 4 checkpoints. Attack configurations are selected on the source and frozen before target evaluation.

Entries are mean TPR at 1% FPR (%) across two targets, with 32 predictions per candidate for each method. Bold marks the highest value in each row, including ties. Error averages squared prediction errors. The Rao–Moyer statistic TmcT_{\mathrm{mc}}, called “RF error” in Table 1, averages velocity predictions before computing the error. RF uses DIME-RF and FreMIA-RF. DDPM uses Loss-MC, DIME and FreMIA-Naive-MC. A dash marks an unreported or inapplicable result.

Model / datasetStageError (MC baseline)Rao–Moyer (ECCV 2026)DIME (arXiv 2026)FreMIA (ICML 2026)PaFMIA (aggregation, ours)
RF / CIFAR-1020k1.031.031.031.090.99
RF / CIFAR-1040k1.080.991.001.111.19
RF / CIFAR-10100k1.361.625.542.0613.59
RF / CIFAR-10500k—59.9088.3565.1297.13
RF / SVHN20k1.151.081.221.131.12
RF / SVHN40k1.161.042.330.981.91
RF / SVHN100k1.751.7613.093.2914.54
RF / SVHN500k—67.8094.6989.7695.75
RF / TinyImageNet20k0.961.001.030.981.03
RF / TinyImageNet40k0.961.021.040.991.07
RF / TinyImageNet100k0.971.010.941.141.29
RF / TinyImageNet500k—5.5119.7815.3065.69
RF / CelebA20k1.081.181.020.951.09
RF / CelebA40k1.061.330.941.051.09
RF / CelebA100k1.642.282.191.782.80
RF / CelebA500k—38.6373.1238.8689.60
DDPM / CIFAR-1020k1.03—0.941.100.95
DDPM / CIFAR-1040k1.05—1.001.191.65
DDPM / CIFAR-10100k1.59—5.502.0226.67
DDPM / CIFAR-10500k41.71—96.5358.6899.55
DDPM / TinyImageNet20k0.96—1.060.941.04
DDPM / TinyImageNet40k0.91—0.9860.9930.98
DDPM / TinyImageNet100k0.94—0.811.102.69

At 20k and 40k, most methods detect around 1% of members at 1% FPR. At 100k, PaFMIA aggregation reaches 26.67% on DDPM CIFAR-10 and 2.69% on DDPM TinyImageNet, compared with 24.81% and 1.47% for the independently tuned global-profile MLP. On DDPM TinyImageNet it also exceeds global LR (1.81%) on both targets, by 0.90 and 0.86 percentage points (paired 95% CIs [0.39, 1.46] and [0.31, 1.44]). At 500k, the largest RF margin over global-profile MLP is on TinyImageNet, where aggregation reaches 65.69% versus 49.72%.

For a smaller training set, Response 3 reports fresh TinyImageNet models trained on 5,000 images for 50k updates. Under the two class-diversity settings, full-profile PaFMIA reaches 72.88% and 73.18% versus 28.80% and 34.63% for the global profile, with both readouts selected on the source.

Practical scope and larger generators

PaFMIA requires time-conditioned predictions, which open-weight models expose. Generation-only APIs expose only the final images. We additionally tested released 256×256 unconditional ADM and conditional SD1.4 without fine-tuning either generator.

ADM. With auxiliary labeled examples disjoint from the test set, full-profile PaFMIA achieves 60.02% AUROC and 3.865% TPR@1%FPR, versus global LR's 56.23% and 3.230%, both at 32 predictions. The TPR gain is 0.635 points (95% CI [0.080, 1.225]). Separately, frozen CIFAR-10 DDPM-to-ADM transfer is near chance (47.68% AUROC).

SD1.4. We use the trained empty-prompt branch so that each candidate image can be evaluated without a caption. The model was trained with 10% conditioning dropout. On LAION-mi, with auxiliary labels and a disjoint 2,000-member/4,000-nonmember test set, aggregation LR achieves 54.09% AUROC versus 52.41% for global LR, 52.93% for global MLP and 52.65% for MC loss, all at 32 predictions. Compared with global LR, aggregation gains 1.67 AUROC points (95% CI [0.18, 3.18]) and 0.60 points in TPR@1% FPR (1.85% vs. 1.25%, 95% CI for the gain [−0.35, 1.60]).

AuthorsOpenReview ↗

Response to Reviewer 91Rx: W2&Q3: Exploratory Analyses and Transfer to Fresh Targets

Q3&W2. Exploratory analyses and validation on new targets

How the original targets shaped our study. In early tests on two CIFAR-10 and two SVHN rectified-flow targets, reducing the budget from 96 to 32 queries lowered detection more on SVHN for our band-based classifier. After seeing those test results, we investigated how to distribute a fixed query budget across path positions. For the comparisons reported in the paper, we selected the schedules and fitted the classifiers on source data alone, then applied them unchanged to the targets.

Transfer to new targets. We trained new TinyImageNet models under two settings, using either 20 or 150 background training classes and the same 50 evaluation classes. In each setting, we trained one source and two targets from scratch on 5,000 images per model for 50,000 steps. Within this experiment, the source and target image pools were disjoint, as verified by image IDs, file hashes and decoded-pixel hashes. The two targets in each setting shared the target pool. We froze the attack and its calibration on the source before collecting target predictions. PaFMIA’s advantage over the global-error baseline carried over to all four newly trained TinyImageNet targets.

Results with logistic regression and MLPs. All methods used the same eight path positions and four predictions per position (32 queries per image). We fixed logistic regression (LR) for the primary comparison so that only the input representation changed. In a second comparison, source validation chose between LR and an MLP with 16 hidden units. It selected the MLP for both representations in both settings. The table reports mean TPR at 1% FPR (%) across the two targets in each setting.

Background training classesGlobal (LR)PaFMIA full (LR)Global (MLP)PaFMIA full (MLP)
2024.8361.5528.8072.88
15026.2061.7334.6373.18

Gains on individual targets. Adding the band energies to the global profile improved detection on each of the four newly trained targets. The gains ranged from 33.55 to 37.60 percentage points with LR and from 38.30 to 45.50 points with the MLP. As a further check, we expanded the global baseline’s search to include raw, log and combined inputs and a larger MLP. Source validation selected the 64–32 MLP with log inputs in both settings. It reached 30.65% and 31.20%, respectively, on the same query schedule.

Reviewer_qEsfOpenReview ↗
Rating
1
Confidence
1
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies membership inference attacks on denoising generative models. Specifically, the authors demonstrate that membership information can be decoded from the residuals' (input-output) spatial frequency pattern. Their Path–Frequency Membership Inference Attack (PaFMIA) passes spatial frequency information from multiple queries (noised inputs with different levels of noise and multiple repetitions) to a lightweight classifier for the final prediction. They compare their method on 4 different datasets (CIFAR10, SVHN, TinyImageNet, CelebA) and compare to previous methods. PaFMIA achieves the greatest improvements on TinyImagenet.

(I want to note that this paper is out of my expertise, and while I understand the gist of it, it is plausible that I may have misunderstood certain aspects)

Critical strengths
  • The submission demonstrates that separating frequency bands from residuals contains more discriminative information than the entire image, therefore effectively obtaining more information from a single pass.
  • PaFMIA achieves a strong improvement on TinyImagenet compared to the benchmarked methods (+17.69% TPR at 1% FPR).
  • The paper contains multiple useful ablations to determine positions, profile aggregation, and number of queries.
Critical weaknesses
  • It is unclear how well this method scaled beyond the tested (self-trained) generators. All generators are pretty well-trained (500K steps with 20K images without augmentations), and it is unclear what would happen if augmentations or non-uniform training were used, as well as how different architectures and training recipes would have been affected. I also believe that some training on real-world models (e.g., Flux) would be necessary to prove effectiveness.
  • PaFMIA outperforms other methods on TinyImagenet but barely on all other tasks, it is unclear why this happens (complexity, resolution, datasize?). It would be useful to ablate the source of benefits more carefully.
  • The AI polishing of the manuscript made some sections unnecessarily hard to read. I would highly recommend rewriting large parts of the paper.
Questions for authors
  • How well does PaFMIA work on real world models?
3 comments
AuthorsOpenReview ↗

Response to Reviewer qEsf: Q1&W1: PaFMIA on Public Generators and at Different Training Stages

Thank you for recognizing the representation, query efficiency, and ablations. PaFMIA targets unconditional image generative models and retains membership evidence discarded by global errors.

W1&Q1. Public pretrained models

We evaluated two released generators with frozen weights. All tables report the full PaFMIA profile.

Unconditional ADM (256×256). We fitted attacks on labeled ADM development images and used validation data to select the full profile before testing on 20,000 ImageNet training images and 20,000 validation images held out from attack development. Both metrics are percentages.

MethodAUROCTPR@1%FPR
MC loss (baseline)53.9581.985
SimA (TMLR 2026)56.0911.690
PIA (ICLR 2024)51.6171.165
FreMIA-PIA (ICML 2026)52.1171.120
FreMIA-Naive-MC (ICML 2026)51.1531.275
PaFMIA (ours)60.0163.865

PaFMIA exceeds the listed attacks in both metrics. Its TPR improvement over MC loss is 1.880 percentage points, paired 95% CI [1.375, 2.530].

Stable Diffusion v1.4. To test PaFMIA on a publicly deployed text-to-image generator, we evaluated SD1.4 with an empty prompt. This keeps the attack input image-only. SD1.4 trained this branch by dropping text conditioning on 10% of examples.

We chose SD1.4 because LAION-mi provides published benchmark labels. We could not verify comparable per-image labels for FLUX. The recovered labels identify source-corpus membership. LAION-mi

After fitting on separate labeled examples, we tested on 2,000 benchmark members and 4,000 nonmembers.

MethodAUROC (%)TPR@1%FPR (%)
MC loss (baseline)52.651.35
PaFMIA (ours)54.061.60

Relative to MC loss, AUROC increases by 1.41 pp and TPR by 0.25 pp in point estimates. The low-FPR gain remains uncertain. Source-distribution differences are present in both benchmarks, limiting attribution to memorization alone.

W1. Training duration and membership signals

Rao and Moyer trained RF models for 500k updates on CIFAR-10 and SVHN. In a CIFAR-10 generation study, Nichol and Dhariwal evaluated DDPMs at 200k and 500k updates and reported FID 3.19 at 500k with their cosine schedule and hybrid objective. We used 500k across datasets to hold training duration fixed. RF study, §5.1 · DDPM study, Table 2 and Fig. 8

We also evaluated 20k, 40k and 100k updates. At each stage we selected attack configurations on source data and froze them before target evaluation. Each cell shows DIME / PaFMIA, averaged over two targets. We use the RF adaptation of DIME for RF models. DIME (arXiv preprint, 2026)

TPR at 1% FPR (%)

Model / dataset20k40k100k500k
RF / CIFAR-101.03 / 0.951.00 / 1.285.54 / 12.8088.35 / 97.00
RF / SVHN1.22 / 1.052.33 / 1.5213.09 / 11.9094.69 / 95.26
RF / TinyImageNet1.02 / 0.921.04 / 1.000.94 / 1.2419.78 / 67.41
RF / CelebA1.02 / 1.150.94 / 1.192.19 / 2.9573.12 / 89.26
DDPM / CIFAR-100.94 / 0.781.00 / 1.665.50 / 26.4696.53 / 99.57
DDPM / TinyImageNet1.06 / 1.050.99 / 1.100.81 / 2.40—

AUROC (%)

Model / dataset20k40k100k500k
RF / CIFAR-1049.83 / 50.1649.84 / 51.6263.89 / 73.4799.51 / 99.85
RF / SVHN51.31 / 51.7755.57 / 54.4574.63 / 73.3799.79 / 99.81
RF / TinyImageNet49.48 / 49.9649.44 / 49.8149.62 / 51.9770.57 / 96.31
RF / CelebA50.99 / 49.9952.37 / 51.3364.56 / 64.4899.01 / 99.39
DDPM / CIFAR-1049.82 / 50.0249.77 / 54.4668.25 / 86.8399.78 / 99.94
DDPM / TinyImageNet49.91 / 49.7149.89 / 50.1748.66 / 59.52—

At 20k, both attacks are close to chance. At 100k, DDPM CIFAR-10 reaches 86.83% AUROC and 26.46% TPR for PaFMIA, versus 68.25% and 5.50% for DIME. At 500k, PaFMIA has higher point estimates on all five evaluated settings for both metrics.

The 20k and 40k checkpoints share a trajectory. The later columns use archived evaluations, except for DDPM TinyImageNet, whose trajectory continues to 100k. The dash marks its unavailable 500k result.

Under uniform sampling,

=\frac{\text{steps}\times\text{effective batch size}}{\text{training-set size}}.$$ With 20,000 training images and a fixed batch size within each recipe, the early stages give **4%, 8% and 20%** of the 500k per-image exposure.
AuthorsOpenReview ↗

Response to Reviewer qEsf: W2-W3: Why Frequency Profiles Help Most on TinyImageNet and Readability Improvement

W2. Why are the gains largest on TinyImageNet?

TinyImageNet benefits most because its residuals retain substantial membership information beyond total error. PaFMIA preserves how this error is distributed across frequencies and how those patterns change along the generative path, then learns to combine them. This retains useful frequency contrasts that global-error or fixed-filter summaries can suppress. CIFAR-10/SVHN already have strong global-error attacks, leaving less room for an absolute gain. We derive when bands add linear separation beyond global error. We then compare gains across four datasets under one protocol, train on the same images at 32×32 and 64×64, and vary class diversity at a fixed training-set size.

Mathematical mechanism (Section 3.1 and Appendix C.2). Global error is a fixed weighted sum of orthonormal DCT-band energies:

Gt(x)=∑b=1LdbdEt,b(x).G_t(x)=\sum_{b=1}^{L}\frac{d_b}{d}E_{t,b}(x).

Different frequency allocations can share the same global error. Proposition D1 quantifies their additional linear discrimination. Let gg and ff collect log global and log band energies, with member–nonmember mean differences δg,δf\delta_g,\delta_f. For finite second moments and positive-definite pooled within-class covariance Σ\Sigma, the optimal squared Fisher separations obey

D_{g,f}^{2}-D_g^{2}&=\delta_r^\top S^{-1}\delta_r,\\ \delta_r&=\delta_f-\Sigma_{fg}\Sigma_{gg}^{-1}\delta_g,\\ S&=\Sigma_{ff}-\Sigma_{fg}\Sigma_{gg}^{-1}\Sigma_{gf}. \end{aligned}$$ Fisher separation maximizes squared class-mean contrast relative to within-class variance over linear scores. The increment measures membership contrast remaining after linear prediction from global features, standardized by residual covariance; it is positive exactly when $\delta_r\ne0$. The original TinyImageNet residualization experiment supports this mechanism, increasing TPR@1%FPR from **38.03% to 57.07%**. **A common protocol across all four datasets.** We fixed eight positions, four noise draws per position, and the same LR fitting/selection procedure. This removes schedule and classifier choices as explanations for the representation comparison. The table reports two-target mean TPR@1%FPR (%). | Dataset | Global | Global + frequency bands | Gain (pp) | |---|---:|---:|---:| | CIFAR-10 | 93.05 | 94.60 | +1.56 | | SVHN | 93.18 | 94.96 | +1.79 | | TinyImageNet | 37.46 | 57.67 | +20.21 | | CelebA | 83.08 | 88.21 | +5.14 | Both targets on every dataset have positive paired 95% confidence intervals for the frequency increment in TPR@1%FPR and pAUC. With TPR in $[0,1]$, the absolute-gain ceiling is $\mathrm{TPR}_F-\mathrm{TPR}_G\leq1-\mathrm{TPR}_G$: these global baselines leave about **7 pp** on CIFAR-10/SVHN and **63 pp** on TinyImageNet. Residual discriminative structure and available headroom provide a quantitative interpretation of the observed pattern. In the original Table 1, source-selected PaFMIA full reaches **67.41%** TPR@1%FPR on TinyImageNet versus **49.72%** for the global-profile MLP (**+17.69 pp**). Both use 32 queries; the controlled table above fixes positions and LR. **Same-image resolution intervention.** We trained 32×32 counterparts using the same image identities, membership split, 20,000 training images, 500k updates, parameter count, and training recipe as the 64×64 models. | Resolution | Global | Frequency profile | Gain (pp) | |---|---:|---:|---:| | 32×32 | 94.075 | 95.604 | +1.529 | | 64×64 | 37.461 | 57.682 | +20.221 | Define the paired interaction as $$\Delta_{\mathrm{res}} =(\mathrm{TPR}_{F}-\mathrm{TPR}_{G})_{64} -(\mathrm{TPR}_{F}-\mathrm{TPR}_{G})_{32}.$$ The two targets give **+17.864 pp [15.836, 20.950]** and **+19.521 pp [17.214, 21.736]**. This controlled downsampling intervention changes spatial detail and relative receptive-field coverage while fixing image identities and training-set size. The positive interactions establish a larger frequency increment at 64×64 in this comparison. **Training-category diversity.** A separate control fixes 5,000 training images, 50k updates, and the evaluation images. The 3,000 background training images comprise either 20 classes ×150 images or 150 classes ×20 images. Mean frequency gains are **36.725 pp** and **35.525 pp**; the high-minus-low interaction is **−1.20 pp [−5.025, 3.551]**. Increasing category count while reducing images per background class yields no statistically detectable positive increment in this setting. These findings build on the submitted paper's explanation in Section 6 and mechanism analyses in Sections 3.1–3.4 and Appendices A.6/C.2. The new controls clarify the roles of residual frequency information, resolution, and category diversity. Reported intervals condition on the evaluated generators and frozen attackers. **W3. Presentation.** Thank you for the feedback on readability. We will simplify dense passages, make the terminology and experimental comparisons clearer, and upload a revised manuscript.

Thank you for your rebuttal. W2 was addressed, and I appreciate the results; all other concerns still stand:

  • PaFMIA underperforms on public models: In both cases, ADM and SD 1.4 (Table 25), an Image-only diagnostic outperforms PaFMIA significantly. I wonder why these results were added to the paper but not shown in this rebuttal.

  • PaFMIA only works in heavily overfitted exotic settings: the new Table 22 shows that after 20k and 40k steps (i.e., 64-128 epochs) the method is at chance for RF and DDPM on all datasets! For TinyImageNet, this is even the case at 100k steps! The reported results therefore require about 500k steps (or 1600 epochs) over 20k images with no augmentation. That is an extremely unrealistic setting, and my concern about the lack of any form of data augmentation was not addressed. PaFMIA seems to only work in heavy memorization settings.

  • "On the 500k-step models, PaFMIA detected 83.73% of members at a 0.1% false positive rate, compared with 55.50% for DIME, the strongest baseline at that operating point". This result in the rebuttal is another example of hiding a row in the paper, showing that PaFMIA actually underperforms. Table 23, Appendix E.2 shows that PaFMIA significantly underperforms both Global LR (independent schedule) and Global MLP (independent schedule), at 83.73 vs 88.33 and 86.35%, respectively.

  • The paper presentation is largely incomprehensible and was not improved in the rebuttal.

In summary: I may not be an expert in this field, but judging from the author's own responses and tables, the proposed method only works on one specific dataset, in a heavily overfitted, self-trained regime. In all other settings, it clearly fails. The paper is unreadable, and all my concerns still stand. I am very firm in my recommendation for clear rejection. I am maintaining my rating.

Reviewer_jd7ZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a membership inference attack method PaFMIA (Path–Frequency Membership Inference Attack). The method constructs a path-frequency representation consisting of interpolation position and spatial frequency band, which will serve as an input to a classifier for membership classification. The proposed method exceeds the baselines in terms of membership inference accuracy on multiple datasets at low FPR.

Critical strengths

This method is innovative because it focuses on both decomposed frequency signals and multiple points along the interpolation path, which might be helpful in capturing the leaking information relevant to membership.

Critical weaknesses

All the evaluation datasets are limited to low resolution images. It would be a stronger demonstration of generalizability if the method can be transferred/tested on higher resolution images such as 224x224.

Questions for authors

How sensitive is this approach to different architecture for the targeted shadow model?

4 comments
AuthorsOpenReview ↗

Response to Reviewer jd7Z:W1. Evaluating PaFMIA on 256 × 256 ImageNet Images

Thank you for highlighting the value of combining frequency decomposition with multiple path positions. We have supplemented the evaluation with public pretrained models and a controlled resolution comparison.

W1. Higher-resolution images.

The original experiments cover 32×3232\times32 images on CIFAR-10 and SVHN, and 64×6464\times64 images on TinyImageNet and CelebA (Sec. 4.1 and App. A.1). We now extend the evaluation to the public, unconditional ImageNet ADM checkpoint at 256×256256\times256 (Dhariwal and Nichol, 2021), keeping the generator frozen.

We evaluate 20,000 ImageNet training images and 20,000 validation images, using the public training convention to assign membership and screening for duplicates. In the target-assisted setting, attacks are fitted or configured on separate development images with known membership, then frozen before testing. The table compares PaFMIA with SimA, PIA, and two FreMIA variants. Both metrics are reported as percentages.

MethodAUROCTPR@1% FPR
SimA (TMLR 2026)56.091.690
PIA (ICLR 2024)51.621.165
FreMIA-PIA (ICML 2026)52.121.120
FreMIA-Naive-MC (ICML 2026)51.151.275
PaFMIA aggregation (Ours)60.234.345
PaFMIA full (Ours)60.023.865

Both PaFMIA variants exceed the listed baselines on both metrics. The matched and independently selected global comparators coincide in this setting.

Full was the PaFMIA variant selected on development data. Its TPR gain over global is 0.635 percentage points, with a paired 95% confidence interval of [0.080, 1.225].

We also tested direct transfer of a frozen CIFAR-10 DDPM attack to this ADM: the source-selected aggregation variant obtained 47.68% AUROC and 0.885% TPR. This broader transfer changes the data distribution and model as well as resolution.

A controlled resolution comparison.

To examine resolution with the data held fixed, we trained and evaluated on the same TinyImageNet images at 32×3232\times32 and 64×6464\times64, preserving membership splits, architecture, parameter count, training-set size (20,000), and training budget (500,000 steps). Both readouts use logistic regression and the same eight path positions with four noise samples per position. The two-target mean TPR@1% FPR is:

ResolutionGlobalGlobal + bandsGain (pp)
32×3232\times3294.07595.604+1.529
64×6464\times6437.46157.682+20.221

The larger frequency gain at 64×6464\times64 is supported on both targets by paired 95% confidence intervals. Thus, in this matched comparison, retaining frequency information is substantially more useful at the higher resolution.

AuthorsOpenReview ↗

Response to Reviewer jd7Z:Q1. How Sensitive Is PaFMIA to the Shadow Model Architecture?

Q1. Sensitivity to the shadow-model architecture.

A shadow model is a separately trained generator whose training membership is known. We query the shadow model on member and nonmember images to obtain labelled residual profiles, which are used to fit and select the membership attack. The resulting attack is frozen and applied to the target generator; target membership labels are not used for fitting or selecting the attack. We train the shadow independently using a known membership split of the same candidate population, without distillation from the target.

To test architecture mismatch, we keep the original two target generators fixed on each of CIFAR-10 and TinyImageNet, and replace the original shadow U-Net with an ADM-style U-Net. ADM denotes the Ablated Diffusion Model of Dhariwal and Nichol (2021), whose U-Net uses adaptive normalization, residual up/downsampling blocks, and attention. We train this backbone from scratch with the same rectified-flow objective as the targets. The source membership split, 20,000 training images, optimizer/EMA settings, and 500,000 training steps are retained; the ADM shadow has 30.3M parameters versus 28.5M for the original U-Net.

For each shadow architecture, attack positions, weights, normalization, and readout are selected using only source fitting/validation data, then frozen. Each entry gives the mean TPR@1% FPR (%) across the same two targets, with the original shadow on the left and the ADM shadow on the right. The first two rows use Rao and Moyer’s RF error statistic and its calibration by the image’s JPEG file size. For the RF adaptations, parentheses give the source publication.

AttackCIFAR-10TinyImageNet
RF error, TmcT_{\mathrm{mc}} (ECCV 2026)59.90 → 59.905.51 → 5.70
JPEG-calibrated RF error (ECCV 2026)80.39 → 80.4117.68 → 17.94
FreMIA-RF (ICML 2026)65.12 → 65.1215.30 → 14.46
DIME-RF (arXiv 2026)88.35 → 83.6819.78 → 6.20
SimA-MC-RF (TMLR 2026)16.25 → 16.394.03 → 4.03
PaFMIA aggregation (Ours)97.13 → 95.1265.69 → 63.36
PaFMIA full (Ours)97.00 → 94.0667.41 → 61.71

PaFMIA remains effective with the mismatched shadow, although the sensitivity differs by dataset. On TinyImageNet, aggregation and full retain gains of 14.47 and 12.81 percentage points over the independently selected global-profile MLP; both gains have positive paired 95% confidence intervals on both targets. On CIFAR-10, their TPRs remain 95.12% and 94.06%, while the global comparator reaches 96.30%. The original small advantage over global therefore does not persist in this setting.

Aggregation loses less than full on both datasets: 2.01 versus 2.95 points on CIFAR-10, and 2.32 versus 5.70 points on TinyImageNet. Its constrained frequency compression may help transfer, but this comparison also includes changes in source-selected attack configurations. For example, CIFAR full selects a different position allocation and switches from MLP to LR under the ADM shadow. We therefore interpret the results as the sensitivity of the complete source-trained attack procedure. With one ADM shadow per dataset, they do not separate architecture effects from variation across training runs.

We will include the comparison and its experimental details in the final version of the paper.

I appreciate the authors clarification and additional experiments. My concerns are resolved and I maintain my current rating.

AuthorsOpenReview ↗

Thank you for the follow-up

Thank you for taking the time to review the additional experiments. We’re glad they addressed your concerns.

Reviewer_tzuAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies membership inference attacks against denoising generative models (primarily rectified flow) and argues that global error statistics discard useful information. It proposes PaFMIA, which uses residual information jointly across path positions and frequency bands, and evaluates it on four datasets.

Critical strengths
  1. The motivation is clear, and the proposed representation is well justified.

  2. The paper provides theoretical analysis to support its design choices.

  3. The experiments include several useful ablations, and the paper is clearly written.

Critical weaknesses
  1. Table 1 highlights the better variant on each target dataset, but this choice is unavailable in practice. The authors should select the variant on source validation and clarify why the aggregation–MLP model is not trained end to end.

  2. The paper does not show whether DCT bands are preferable to other features. Comparisons with alternative representations and ablations on band design would strengthen the results.

  3. The learned frequency weights are not shown, making it unclear which frequencies matter at each path position.

  4. Several established attacks are missing, particularly in the DDPM experiments, where they are directly applicable. The authors should include these baselines.

Questions for authors

See weaknesses.

2 comments
AuthorsOpenReview ↗

Response to reviewer tzuA: W1–W2: Variant selection, end-to-end training, and alternatives to DCT

Thank you for recognizing the motivation, theoretical analysis and ablations. We clarify the attack configuration and provide additional experimental evidence addressing your questions below.

W1. Variant selection and aggregation–MLP training

Default variant. Our main contribution is the membership information retained across path positions and frequency bands, which simple classifiers can exploit. We will therefore specify full profile as the default across datasets, retaining every measured band at each selected position. Aggregation remains a compact alternative and ablation; both use the same generator queries.

Table 1 originally reported both variants to compare retaining individual bands with compressing them. For each representation, the classifier—logistic regression (LR) or a multilayer perceptron (MLP)—and its settings are selected using known members and nonmembers of the source model reserved for validation, then fixed for target evaluation (Appendix A.3).

At KK path positions, global uses KK error features; aggregation adds one weighted band summary per position (2K2K features); full retains global error and all 12 band errors (13K13K features). PaFMIA log-transforms these energies and standardizes them using source-fitting statistics (Eqs. 8–9). The Table 1 results below are two-target mean TPR@1% FPR (%) at B=32B=32, or 32 predictions per image.

Profile / classifierCIFAR-10SVHNTinyImageNetCelebA
Global-profile MLP96.4395.9849.7288.05
Aggregation + MLP97.1395.7565.6989.60
Full profile + MLP (default)97.0095.2667.4189.26

The global-profile MLP retains Table 1's independently source-selected input transform and query schedule.

At this budget, using full on every dataset is within 0.50 percentage points of the higher TPR of the two PaFMIA variants in each column. Full is higher than aggregation on TinyImageNet; aggregation is slightly higher on the other three. Compared with the independently selected global-profile MLP, full achieves higher mean TPR on three of four datasets, including a 17.69-point gain on TinyImageNet; on SVHN it is approximately 0.73 points lower.

Why not train aggregation–MLP end to end? The band weights and the MLP can be trained end to end. In Table 1, we used two stages because we wanted to know whether a linear classifier and a small MLP could both use the membership information in the same path–frequency features. We learned the band weights with LR and kept them fixed when fitting the MLP. This way, switching classifiers changed only how the features were used.

We also compared two ways to fine-tune that two-stage classifier: keep the band weights fixed or update them jointly with the MLP. Both runs started from the same fitted weights; we did not test joint training from scratch. Entries are mean TPR@1% FPR / [email protected]% FPR (%) across two targets.

DatasetUpdate MLP onlyUpdate MLP + band weights
CIFAR-1097.19 / 81.7396.94 / 80.86
SVHN95.75 / 81.0295.75 / 81.02
TinyImageNet65.75 / 45.7465.69 / 46.01
CelebA89.60 / 28.6989.60 / 28.69

Joint fine-tuning gave no consistent gain over updating the MLP alone. On SVHN and CelebA, source validation selected the starting weights in both runs, hence the identical results.

W2. Alternative representations and frequency groupings

All three frequency representations improve on global error alone. We compared 2D discrete cosine transform (DCT), Fourier (FFT) and Haar wavelet decompositions. DCT gives the highest mean TPR at 1% FPR on three of four datasets; Haar leads on TinyImageNet. The gap between the best and worst transform is about one percentage point or less on CIFAR-10, SVHN and CelebA, and 2.75 points on TinyImageNet.

DCT and FFT use 12 equal-width radial bands; three-level Haar uses 10 subbands grouped by scale and orientation. All three use the same eight path positions, four noise queries per position (B=32B=32), residual samples, source splits and LR classifier. All bands reuse the same predictions.

Global uses eight global-error features; the other columns retain global error and all band energies (104 features for DCT/FFT; 88 for Haar). This fixed-schedule LR comparison isolates the representation, whereas Table 1 uses source-selected schedules and classifiers. The two-target mean TPR@1% FPR (%) is:

DatasetGlobalDCTFFTHaar
CIFAR-1093.0594.6094.1894.34
SVHN93.1894.9694.5094.12
TinyImageNet37.4657.6855.8058.55
CelebA83.0888.2187.5187.17
AuthorsOpenReview ↗

Response to reviewer tzuA: W3–W4: Learned frequency weights and additional DDPM baselines

W3. Learned frequency weights

The learned frequency weights vary with path position and dataset. CIFAR-10 puts most weight on low-to-mid bands and CelebA favors higher bands. On SVHN and TinyImageNet, the most heavily weighted bands change markedly across positions.

These are the normalized band weights at,ba_{t,b} of the compact aggregation variant at B=32B=32 (Eq. 9). They are learned on the source and held fixed on the targets. The default full profile instead retains each band as a separate classifier input.

The table shows all eight source-selected positions on TinyImageNet. Here tt runs from noise (00) toward data (11). Bands are numbered 0–11 from low to high frequency, as in Appendix A.2. For display only, we sum the 12 weights over three ranges. Entries are percentages of total band weight, summing to 100% per row up to rounding; bold marks the largest sum.

Path position ttLow bands 0–1Bands 2–8High bands 9–11
0.0470.6994.305.00
0.1190.2045.1354.66
0.1820.040.7199.25
0.4507.7891.840.38
0.47560.1237.742.14
0.5000.094.4395.48
0.6793.5467.2029.27
0.81896.553.040.40

For example, on TinyImageNet the high bands receive 99.25% of the weight at t=0.182t=0.182, while the low bands receive 96.55% at t=0.818t=0.818.

In the revised paper, we will include heatmaps showing all 12 bands separately at every selected position for each of the four datasets. They will also show each learned weight relative to the weight that would reconstruct global mean squared error (proportional to the band's DCT coefficient count; Eq. 5). A positive log-ratio means the aggregation gives that band more weight than this reference; a negative value means less.

W4. Additional DDPM baselines

Rectified flow (RF) is our primary setting. The DDPM extension applies the same path–frequency representation to noise-prediction residuals (Section 4.4). We have expanded the comparison with SecMI's statistical variant (SecMI-stat), PIA, FreMIA, DIME and quantile regression (QR). For FreMIA, we average the filtered residual error over noise draws (Naive-MC).

We use the CIFAR-10 DDPM checkpoints and test images from Table 4: 500,000 training steps and exponential moving average (EMA) weights. Configurations are selected on source validation and frozen before target scoring, except for QR's target-nonmember fitting described below. TPR values are percentages averaged over two targets.

MethodTPR@1% FPR[email protected]% FPR
SecMI-stat (ICML 2023)42.5010.84
PIA (ICLR 2024)81.1021.99
FreMIA (ICML 2026), Naive-MC58.683.44
DIME (arXiv 2026)96.5355.50
QR (ICML 2024), α=0.00176.9424.88
QR (ICML 2024), α=0.0175.5621.17
PaFMIA (full, default)99.5783.73

All methods use the same ROC rule: the largest TPR attainable at a measured FPR no greater than the stated bound, without interpolation.

QR predicts a nonmember error threshold for each image. For the ROC comparison, we define QR-margin as the median threshold from seven predictors minus the image's measured error, with the quantile level α fixed within each row. QR uses 4,000 auxiliary target nonmembers: 3,000 for fitting and 1,000 for diagnostics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.