- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces PF-LDM, a framework for training latent diffusion models with local differential privacy, targeting the loss of image quality that privacy noise causes in high-dimensional pixel space. In PF-LDM, each client encodes its images with a shared, fixed public autoencoder, clips the latents, adds Gaussian noise once at a chosen diffusion timestep, and sends the privatized latents to a server. The server trains a global denoiser on the pooled noisy latents, while each client keeps a local denoiser trained on its own clean latents. At generation time, the global model produces a sample at the noise level used for privatization, and the local model refines it to recover client-specific details. The paper provides local differential privacy guarantee and utility analysis for learning Gaussian mixture models. Experiments on face and chest X-ray datasets show that PF-LDM has better generation quality for underrepresented groups, better downstream classification, and near-chance membership inference against the shared model.
Critical strengths
- Private training of generative models for sensitive domains such as medical imaging is an important problem. Releasing privatized latents once, with a guarantee that does not depend on the training length and without a trusted server, is a practical setting.
- The paper is generally readable, and the pipeline of private release, global training and local refinement is easy to follow.
Critical weaknesses
- The idea of privatizing data once with the forward diffusion noise and using the noisy samples for server-side training, together with the collaborative sampling that hands off to a local model, comes from PFDM (Patel et al., 2026). The main change here is applying it in the latent space of a pretrained autoencoder, which is a fairly incremental step.
- The theory and the empirical explanation of "what favors collaboration" are loosely connected to the method. The utility result assumes that all clients draw from the same Gaussian mixture, which does not match the heterogeneous setting that motivates personalization, and the benefit of collaboration is stated under an assumption that the global model is more accurate at the handoff level.
- The experimental evidence for the utility claims is not sufficient. There are only two clients with a hand-crafted imbalance, and the only baselines are the non-private and non-collaborative extremes. The most relevant private PFDM and DP-SGD baselines are missing. The FID gains are also mixed, since the non-collaborative baseline is better on the majority group or on average in several settings.
- The practical benefit is not yet convincing. A single, fairly large privacy level is used, so the privacy–utility trade-off is not actually shown, so the guarantee is meaningful only at a fairly large privacy budget. The gains over a non-collaborative model are modest in many cases. Without a comparison to PFDM and DP-SGD baselines, it is hard to judge how much is gained by moving to latent space.
Questions for authors
- How does PF-LDM compare to the PFDM method it extends and to a DP-SGD-based approach at the same privacy level? Without these baselines, it is hard to attribute the gains to latent-space privatization.
- How does the performance change with the privacy budget (and the corresponding clipping radius and noise timestep)? The results use a single, fairly large budget, so it is unclear how the method behaves at stronger privacy.
- The utility theorem assumes that all clients share the same latent distribution, while the experiments and motivation are about heterogeneous clients. Can the analysis be extended to heterogeneous distributions?