- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper proposes a training-free private generation mechanism via diffusion. It relies on the standard diffusion SDE involving a private empirical denoiser instead of the standard (privately or non-privately trained) empirical denoisers. In this setting, privacy arises from the Gaussian mechanism naturally induced from the diffusion term in the SDE. To scale this method without catastrophic privacy loss accumulation, the paper introduces two tricks: a diffusion model pretrained on a public dataset used instead of the private denoiser for lower noise levels, and a batch subsampling per generated sample to compute the empirical denoiser. The authors then compute the privacy guarantees of such procedure and test it w.r.t. standard DP diffusion models.
Critical strengths
- The paper provides an in-depth study of the privacy incurred by the stochastic sampling procedure of diffusion models. The general approach is original and the strong privacy guarantees yield competitive generation results w.r.t. the state of the art. While the impact on future work on DP diffusion is hard to assess, it may inspire other works to improve existing DP-trained diffusion models.
- The paper is, for the most part, clear and easy to read. All ideas and hypotheses are well exposed. Up to the extent of my assessment -- I could not review the appendices in-depth, as my technical expertise in differential privacy remains limited --, the theoretical results presented in the paper appear to be sound.
Critical weaknesses
The paper is limited by two important weaknesses.
Motivation and consequences of privatizing the sampling procedure only
- The paper provides little and insufficient motivation to privatize the sampling procedure of diffusion only. The statement that "regulators see only generated samples" is incorrect; for example, the EU supervising regulatory body for personal data protection specifically concluded that a model may not be considered anonymous if personal data could be retrieved from model weights (https://www.edpb.europa.eu/news/edpb-opinion-on-ai-models-gdpr-principles-support-responsible-ai_en). Further, even users may have more advanced access to the model, e.g. deployers of a model or even individuals working with open-weight models. The privatization of the sampling procedure may have other motivations, but they are not developed in this paper.
- The presented method has a crucial weakness that is discussed and partially alleviated in the paper but that is not recognized as a blocker for scaling and not analyzed thoroughly: the privacy guarantees heavily depend on the number of generated samples. In many application cases of generative models, the number of samples they are tasked to produce is large and may be much larger than the size of their training dataset. The proposed method is therefore impractical.
- In particular, the paper should include an analysis of the evolution of the privacy guarantees w.r.t. the number of generated samples.
Experimental relevance
- The trick leveraging a denoiser pretrained on public data is clever, but confuses the interpretation of the experimental results. It could be used for all other baselines, which may improve their results. Moreover, the paper should verify the FID of the generation process using this public denoiser only, to ensure that it is not even better than the proposed method itself.
- The proposed method will not scale well with the dataset size as it relies on the empirical denoiser. In my opinion, this is not a weakness of the paper since related works also have scaling issues preventing their application at scale. Nonetheless, this should be discussed in the paper along with an analysis of the computational complexity incurred by the empirical denoiser.
[Minor] Clarity
The clarity of some parts of the paper should be improved, should it be accepted after the reviewing process. In particular:
- more technical background for Gaussian DP should be included in the paper;
- the purpose of Section 2.3 is unclear;
- Proposition 4.1 and its surroundings are overly complex to parse and interpret, especially as it uses some notations that have not been previously defined.
Questions for authors
Given the above assessment, I think that this paper presents an interesting method but is limited by its lack of motivation and some of its empirical validation. For these reasons, I recommend a weak rejection. I remain open to discussing with the authors and other reviewers before providing my final recommendation.
In particular, I may change my assessment primarily if:
- the above concern about the motivation for a private sampling procedure and the scaling analysis are adequately addressed in the author's response;
- the experimental issues are fixed, notably the clarification of the role of the public denoiser.
Post-rebuttal update
I would like to thank the authors for their thorough response, and to apologize for not having had the time to acknowledge it sooner. My main concerns have been adequately addressed. While I'm not quite sure about the impact of the method itself, I believe the developed ideas are worth sharing and could be leveraged in other works. I am therefore happy to now recommend a weak accept.
2 comments
Sampling with multiple releases
Released samples do not need to be regenerated to serve more users. Reusing those samples costs no privacy, and any model trained only on the release and public data inherits its guarantee. For CIFAR-10, we fine-tuned a class-conditional latent diffusion model (LDM) (Rombach et al., 2022), pretrained on ImageNet 32×32 with 16×16 latents, on the DP released samples from our headline configuration (), with labels from a public ImageNet classifier applied to the released images. For CelebA-64, we fine-tuned an unconditional LDM pretrained on FFHQ 256x256 on our release (). No step reads the private data itself.
| FID (50k samples, one seed) | CIFAR-10 | CelebA-64 |
|---|---|---|
| Our release | 25.00 | 23.00 |
| Pretrained LDM, no fine-tuning | 25.99 | 43.99 |
| Attention modules 9–16, high noise only | 23.01 () | 22.52 () |
| Full UNet, high noise only | () | () |
| Attention modules 9–16, more noise levels | 30.43 (all ) | 28.08 () |
Fine-tuning improves on the utility of the samples themselves. Training only at high noise works best, which suggests the released samples mainly carry coarse structure while the public prior supplies the fine details. This split is a utility choice, since training on the released samples is free at every noise level. Our method can therefore give a model with unlimited output coverage at the release's budget, without DP training.
Despite better baseline FID, the pretrained ImageNet LDM cannot replace the public denoiser (ImageNet EDM) inside our sampler. Our sampler runs the empirical and public denoisers on the same pixel-space trajectory, while the LDM denoises in latent space at every step. It is still useful given the released samples, as post-processing, to improve the utility at no additional privacy cost.
Contribution of the public denoiser
Public-only sampling gives FID 30.70 0.17 (CIFAR-10) and 49.26 0.09 (CelebA-64), against 24.88 0.10 and 22.03 0.10 for our hybrid at the headline setting (three seeds). The private block thus improves FID by 5.82 and 27.23 compared to the public-only baseline. On CIFAR-10, the utility gains appear from the 62% handoff onward, and on CelebA-64 it holds across the whole sweep (see our response to gMB8).
Comparison fairness
Five of the six baselines also use public ImageNet data. DP-Diffusion (Ghalebikesabi et al., 2023) and DP-Promise (Wang et al., 2024) pretrain a diffusion model on ImageNet and fine-tune it on the private data. DP-LDM (Liu et al., 2024) fine-tunes only the attention modules of an ImageNet LDM with DP-SGD. DP-API (Lin et al., 2024) queries an ImageNet diffusion model through its API and, like ours, trains no network on the private data. DP-MEPF (Harder et al., 2023) uses an ImageNet feature extractor. Only DP-DM (Dockhorn et al., 2023) uses no public data. Table 2 uses certified bounds below and marks the comparison as contextual.
Computational scaling
For outputs, private steps and -dimensional images, the empirical denoiser costs with full batches and with output-wise subsampling, on top of public inference. Space complexity is per trajectory, plus for the dataset. Per-output cost is independent of at fixed , but grows linearly with at fixed . A DP-SGD model instead pays a one-time training cost (epochs , noise multiplicity , forward cost ) plus per-example clipping. Our sampler trains nothing. Its private block (subsampling, kernel weighting, clipping and the Gaussian step) costs per output. For a finite release for our experiments at , our total cost is comparable to ordinary public sampling. Empirically, a 50,000-image CIFAR-10 release takes about 2.5 L40S GPU-hours.
Clarity
Section 2.2 now gives GDP's hypothesis testing interpretation, Section 2.3 opens with PLD's role in obtaining certified bounds, and Section 4 defines all quantities before Proposition 4.1 and explains the proof in three steps: compose within the block, amplify once, compose over outputs.
We would appreciate your assessment of whether these changes resolve your two main concerns.
Thank you for recognizing the originality of the sampling-time analysis, and for identifying the two issues that could change your assessment: the motivation and scaling of a private sampler, and the contribution of the public denoiser. We agree that both needed stronger treatment.
Motivation for privatizing sampling
Thank you for the EDPB reference. You are right that "users and regulators see only generated samples" is incorrect. Model weights can reveal personal data, and deployers or open-weight users can access them directly. We removed the statement and rewrote the motivation.
No network is trained on the private data. The private dataset, D, enters only through the clipped empirical denoiser during sampling. The neural denoiser is trained on public data disjoint from D (Algorithm 1), so there are no private weights to leak. A trusted data holder needs access to D to run the hybrid sampler. Deployers and open-weight users receive only the released samples, the public checkpoint, or a model trained on the released samples, and none of these reveal more about D than the (ε,δ) guarantee allows. The public checkpoint must be fixed independently of D.
Why privatize sampling? Private sampling gives a formal guarantee without training any network on the private data. This avoids private training methods and its cost in sample quality, and one public model can serve many private datasets with no training per dataset. Because each step's sensitivity is explicit, the analysis also shows where privacy loss accumulates along the trajectory, which motivates our hybrid sampler. A model trained on the released samples inherits its guarantee by post-processing.
When DP training is better. Each new sample from D adds privacy cost. If an application needs repeated, fresh access to the private data, a DP-trained model may give a better trade-off. Our method is the right fit when the release is bounded and avoiding private training methods altogether is valuable.
Scaling with the number of outputs
We agree this is a genuine limitation. We compute how the certified bound grows with the number of released images R, with the same PLD accountant and everything else fixed (CIFAR-10, n=50,000 B=25, q=5×10−4, δ=10−5, add/remove adjacency). Outputs are generated independently, so per-image quality does not depend on R, and larger releases need only new accountant runs.
The cost grows sublinearly in R. In the regime where the hybrid sampler improves FID, a privacy budget of ε≤5 allows about 9n sampled images at the 62% handoff and about 0.5n images at 65%. For ε≤10, we can generate about 3n samples at 65% handoff. A data holder can therefore fix the handoff and the release size before generating to fit within a certain privacy budget.
Unlimited coverage does not mean unlimited utility. Our outputs, like samples from a DP-trained model, are independent draws from one fixed distribution, so downstream utility eventually saturates. On CIFAR-10, Ghalebikesabi et al. (2023) show that a classifier trained from scratch on their synthetic images keeps improving with more samples (their Figure 3). The same classifier pretrained on ImageNet-32 reaches the same accuracy at 50K synthetic images and gains little accuracy from further samples (their Figure 10). Our setting already assumes public data, so downstream models can be pretrained in the same way. Released samples can also be expanded by post-processing at no privacy cost, as Lin et al. (2024) do for Private Evolution.