Partial Images Raise Further Privacy Concerns in Personalized Diffusion Models
Abstract
Personalized diffusion models can adapt to user-specific subjects from a small set of training images, enabling generation in novel scenes, styles, and compositions. Their checkpoints can be shared for reuse without disclosing the private fine-tuning images, which may appear privacy-preserving. However, information retained in the checkpoints can still enable recovery of those images. While prior extraction methods primarily study this risk through free-form generation, partially obscured or redacted versions of private images may also unintentionally remain accessible through social media or other public sources. The privacy implications of such auxiliary information remain largely unexplored. We therefore investigate how this information can raise further privacy concerns in released personalized checkpoints. To demonstrate it, we introduce Partial-Image-Assisted Reconstruction (PIAR), which combines image inpainting with personalization guidance to reconstruct complete images from partial observations. Our experiments show that PIAR reconstructs private images substantially more accurately than free-form extraction. The risk extends beyond the fine-tuning images, as PIAR also improves reconstruction of unseen photographs of the same personalized subject. Together, these findings highlight how partial visual information can expose sensitive content through released checkpoints and motivate stronger privacy protections for personalized diffusion models.
est. 32% chance this paper gets accepted at ICLR 2027.
What do you think this paper will get?
All positions stay anonymous.