FaceDR-Guard for Robust Face-Swap Identity-Reuse Defense in Semantic Code Space

Public face images can be exploited as reusable identity sources by face-swapping systems. This identity-reuse threat differs from conventional face-recognition evasion: the central question is whether a protected image can still provide identity information to a downstream generative model. We introduce FaceDR-Guard, a proactive defense that operates on the semantic code of a frozen pretrained diffusion autoencoder. For each source image, FaceDR-Guard optimizes a bounded semantic-code offset through an end-to-end differentiable surrogate pipeline comprising image reconstruction, face swapping, and identity evaluation. The optimization suppresses the source identity retained in the swap output, while skin-aware multi-scale, chromatic-consistency, pixel-fidelity, and semantic-parsing objectives regularize visual distortion. We further employ short-chain optimization followed by long-chain rendering to reduce optimization cost and improve final reconstruction quality without updating any pretrained model weights. Experiments reveal a consistent advantage under deployment shifts: FaceDR-Guard maintains strong identity-reuse suppression as conventional pixel-space attacks deteriorate, while achieving robustness competitive with feature-space cloaking and better preserving benign face-recognition utility. These findings highlight swap-output-aligned semantic reconstruction as a promising approach to robust identity-reuse defense.