acceptodds
Under review as a conference paper at ICLR 2027

Invisible Theft: Memorization-Based Client-to-Client Data Leaks in Federated Diffusion Models

Abstract

Federated diffusion has emerged as a promising framework for collaboratively training visual generation models without sharing private training data, making it particularly valuable for data-scarce yet privacy-sensitive federated settings. However, we reveal a previously overlooked client-to-client privacy vulnerability in this framework: because diffusion models can memorize and reproduce their training samples, a malicious client may exploit the shared global model to generate images highly similar to other clients' private training data, without accessing their gradients or interfering with the federated training process. To demonstrate the severity and practicality of this vulnerability, we propose a memorization-residual guided data stealing framework. Our key intuition is to leverage two models naturally accessible to the attacker—an attacker-private model trained only on its own data and the public pretrained model—as negative references, and use their prediction residuals with the global model as sampling-time guidance signals. This residual-guided sampling substantially increases the likelihood that the global model generates images highly similar to private training samples from other clients. Experiments show that our attack consistently increases both the frequency of high-similarity reproductions and the coverage of distinct threat images. Moreover, our attack is performed entirely after federated training and requires no modification to the training protocol, making it naturally stealthy from the perspective of federated training.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
privacy leakage, federated diffusion model

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_iivBOpenReview ↗
Rating
4
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

In this paper, the authors identify a client-to-client privacy vulnerability in federated diffusion. Since the global model is technically fine-tuned on every client's data and diffusion models are known to memorize samples, a malicious client can regenerate other clients' private training images from the distributed global model. The malicious clients are able to do so without observing their gradients, accessing the server, or interfering with training. The challenge faced by this malicious attacker is entangled memorization, where the global model's memorization gets mixed with the clients' data, the attacker's own data, and the pretraining prior. The last two are unwanted. The attacker has access to its own data and the public pretrained checkpoint, so it is able to construct two negative guidance residuals at sampling time to get rid of unwanted information. Overall, I really like the idea, presentation, and implementation of the paper. I have some minor concerns and questions, which I have listed below.

Critical strengths
  • The threat model is novel and well motivated. The consequence is alarming in a useful way; in a medical federated setting, every participating hospital could become a serious attacker.

  • The decomposition of the global model's memorization is done very elegantly into three sources. Two of these sources contribute to the negative-guidance terms, thereby isolating the contribution of other clients' data from the attacker's own data and the pretraining knowledge of the global model.

  • Overall, the paper is a good read. It is well structured and easy to understand. The experiments and ablations are solid, and I don't really have any major concerns.

Critical weaknesses
  • It is not clear which parameters are fine-tuned. The paper mentions that the model is initialized from Stable Diffusion and fine-tuned. Unless I have completely missed it, it is not mentioned anywhere in the paper whether it is full fine-tuning, U-Net only, U-Net plus text encoder, or some other LoRA scheme.

  • The client count and the round budget should be in the main text. The number of clients is never given as a proper number, and the reader has to infer K=4 for Fed-Fundus and K=6 for Fed-Prostrate. The round budgets appear in the Appendix too and should be in the main text since that is something a reader wants to know when reading about an FL experiment.

  • Figure 3 was hard to understand for me. a) and b) regions are not visually separated, so it is unclear where one ends and the other begins. The color scheme is a bit confusing as well. The b) label is red, and various elements in the figure are red as well. Is there a correlation between b) and the red part of the figure?

Questions for authors
  • Is the federated fine-tuning for the full model, or is it some LoRA configuration? If it is full fine-tuning, does the attack retain its effectiveness under various LoRA ranks?

  • Why are there 5 rounds for Fed-Fundus vs. 25 rounds for Fed-Prostate?

  • Does the attack still work if the clients adopt a memorization-mitigation strategy or DP noise?

  • How does the attack success scale with increasing the number of clients?

Reviewer_gfV2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies offline data extraction by an honest-but-curious client in federated diffusion. It guides sampling with differences between the global model and two accessible references: the public pretrained model and a model trained on the attacker's own data. Experiments on two medical datasets show higher similarity to other clients' training images and greater unique-target coverage than direct sampling.

Critical strengths

The threat model is compelling: extraction requires neither victim updates nor changes to federated training. The gains over direct sampling are consistent across the two datasets, and the component ablations, CFG sweep, and five-seed results help establish that the sampling modification has a real effect. Reporting unique-target coverage alongside output-level similarity is also useful, since repeated recovery of one image would otherwise inflate the apparent leakage.

Critical weaknesses
  1. The evidence does not yet cleanly distinguish training-image extraction from similarity to the underlying medical-image distribution. SSCD is supplemented with DINOv2 and LPIPS in Appendix A.5, but there is no matched, patient-disjoint nonmember control or calibration of the false-match rate at the chosen thresholds.

  2. The interpretation as separating other clients' memorization is less established than the extraction gains. Pretraining drifting provides most of the improvement in Table 1, while the attacker-specific term adds a smaller gain. A reference-model control matched for training budget but trained on unrelated or distribution-matched data would help show whether the attacker's own data are specifically important.

  3. FineXtract (Wu et al., ICML 2025) appears to be a missing close baseline: it already uses pretrained/fine-tuned model guidance to extract fine-tuning data. Please clarify how pre-training drifting differs and compare under matched prompts and computational budgets. Table 1 suggests that this component accounts for much of the improvement. Without this comparison, the additional contribution of the private-client reference—and hence the methodological novelty—remains unclear.

Questions for authors
  1. At the same retrieval-set sizes and thresholds, how often do generated images match patient-disjoint held-out images from the same centers? How do training-versus-held-out matches compare?

  2. Does the additional benefit of attacker drifting persist when its reference is replaced by a model trained for the same budget on unrelated or distribution-matched data?

Reviewer_PhvhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

the paper studies client-to-client privacy leakage in federated diffusion models. The key observation is that a global diffusion model may memorize private samples from participating clients, allowing an honest-but-curious client to recover samples belonging to other clients without accessing their data, gradients, or local models. The paper proposes a residual-guided sampling method using two attacker-accessible reference models: (1) a private model trained only on the attacker's own data and (2) the original pretrained model. Differences between these models and the global model are used during sampling to steer generation toward information contributed by other clients. Experiments on Fed-Fundus and Fed-Prostate show consistent improvements over direct sampling and other baselines.

Critical strengths
  • The client-to-client threat model is interesting. In particular, the attack does not require poisoning, modification of federated training, or access to other clients' gradients.

  • The proposed residual-guidance idea is simple and intuitive, and the two components are reasonably motivated.

  • The method consistently improves memorization/extraction metrics over direct sampling across the two evaluated datasets. The ablations, sensitivity analysis, cross-embedding evaluation, and seed analysis provide useful supporting evidence.

Critical weaknesses

My main concern is that the current experiments do not yet establish the generalization or practical severity of the claimed vulnerability.

The baseline global model already exhibits extremely strong memorization. For example, on Fed-Fundus, direct sampling achieves 86.7% [email protected] and 65.7% [email protected], compared with 94.8% and 79.1% for the proposed attack. Thus, while the proposed method clearly amplifies leakage, it is difficult to disentangle the contribution of the attack from severe memorization already present in the underlying model/training regime.

The headline Mem metrics give a substantially stronger impression of extraction than the unique-target coverage results. At the stricter 0.7 threshold, the full method recovers only 4.05% and 1.97% of unique threat samples on Fed-Fundus and Fed-Prostate, respectively. Even at 0.6, unique coverage remains below 9%. This suggests that many successful generations may repeatedly correspond to a relatively small subset of highly memorized samples. I think this distinction is important when interpreting the practical severity of "data stealing."

The experimental regime is narrow and potentially unusually prone to memorization. Both datasets are small medical datasets, the experiments use Stable Diffusion 2.1 with FedAvg, and clients perform substantial local fine-tuning. It remains unclear whether similar leakage would occur for larger datasets, different data domains, architectures, FL algorithms, or less overfit training regimes.

I would particularly like to see a centralized fine-tuning control using the same union of training data. The core attack relies on differences between the pretrained and downstream fine-tuned models. Without this comparison, it is unclear how much of the demonstrated vulnerability is specifically associated with federated learning, versus a more general consequence of diffusion-model memorization after fine-tuning.

The interpretation of the residual terms is intuitive but not fully established. In particular, model predictions do not necessarily decompose linearly into pretraining, attacker-data, and other-client contributions. The empirical results show that these residual terms provide effective sampling guidance, but provide less evidence that they isolate the corresponding sources of memorization in the manner suggested by the method's motivation. Relatedly, pre-training drifting accounts for a substantial fraction of the gains, raising the question of whether the main mechanism is more generally amplifying the difference between pretrained and fine-tuned models.

Questions for authors

I find the threat model interesting and believe the paper identifies a privacy issue that is worth studying. The proposed approach is also simple and shows consistent empirical improvements over direct sampling. However, I am not yet convinced that the current evidence supports the broader claims regarding the severity and generality of client-to-client data stealing in federated diffusion models.

In particular, the combination of extremely high baseline memorization, relatively low unique-target coverage, and evaluation on only two small medical datasets makes it difficult to determine whether the observed vulnerability generalizes beyond this highly memorization-prone setting.

Here are a few questions, answering these could strengthen the paper:

  1. Direct sampling already shows very high leakage, particularly on Fed-Fundus (e.g., 86.7% [email protected]). Can the authors clarify how much of the observed privacy risk should be attributed to the proposed attack versus unusually strong memorization in the underlying federated model/training regime?

  2. The generation-level Mem scores are high, but unique-target coverage is substantially lower (e.g., [email protected] of 4.05% on Fed-Fundus and 1.97% on Fed-Prostate). How should these results be interpreted in terms of practical data extraction? Does increasing the sampling budget substantially increase unique-target recovery, or mostly reproduce the same highly memorized samples?

  3. Have the authors evaluated an equivalent centralized fine-tuning setting using the same union of client data? If so, does the proposed residual guidance yield comparable extraction rates? This would help clarify whether the vulnerability is specific to federated training or is primarily a general fine-tuning/memorization phenomenon.

  4. Can the authors provide evidence that the attack remains effective in a less memorization-prone regime, e.g., with larger per-client datasets, fewer local training steps, or otherwise reduced overfitting? This would materially affect my assessment of the generality of the threat.

Suggestions for Future Work:

  • Evaluation across additional data domains, diffusion architectures, and federated-learning configurations would help establish the broader generality of the attack.
  • Further analysis of why the residual directions isolate or amplify memorized information would strengthen the mechanistic interpretation of the method.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.