- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors propose VEIL, which adds small perturbations to user images to suppress identity reproduction after diffusion-model personalization. The method stabilizes optimization using fixed low-noise timesteps and VAE posterior-mean encoding, combined with attribution-guided attention suppression and spatial saliency constraints. Experiments on two face datasets and multiple personalization settings evaluate protection effectiveness and image fidelity.
Critical strengths
1.The method addresses optimization stability, attention-position selection, and visual fidelity with clear design motivations. The experiments are fairly comprehensive, and ablation studies support the role of each component.
- Under the main experimental settings, the method achieves a good balance between preserving the original image quality and suppressing personalized generation, with comparisons against multiple existing protection methods.
Critical weaknesses
-
The quantitative results show some improvement, but the visual advantages over existing methods are not clear in the qualitative comparisons provided in the supplementary material. I find it difficult to judge whether these metric gains translate into meaningful improvements in identity protection, so their practical benefit remains a concern.
-
The proposed components have clear precedents in related work, and the overall design appears largely combinatorial. The paper does not sufficiently establish a contribution beyond combining existing techniques and ideas, so I find its technical novelty and broader methodological insight limited.
Questions for authors
-
Could the authors explain, using qualitative examples, where the quantitative improvements are reflected? Do these gains indicate stronger identity protection, or mainly further degradation of generated image quality?
-
FDFR measures face detection failure. How does this establish reduced identity leakage? Among generations where faces remain detectable, is identity similarity significantly reduced? How much does assigning an ISM of zero when no face is detected affect the comparisons?
-
Gaussian blur and diffusion-based purification substantially weaken protection, and changing the identifier also reduces effectiveness. How much practical protection remains when an attacker can adjust these settings?
-
What is the key new design beyond existing timestep selection, attention suppression, and saliency constraints? What evidence demonstrates a new mechanism or advantage beyond combining these components?
5 comments
Reply to Reviewer SV6N [Part 2/2]
Q3. How much protection remains when the attacker changes the setup?
Under all four preprocessing operations on CelebA-HQ, VEIL ranks first or second on ISM and CLIP-F (Table 2). Under JPEG compression, VEIL keeps the highest FDFR (0.91), and only 0.06 of its generations are recognized as the reference identity, compared with 0.81 without protection. Gaussian blur brings the FDFR of every method to at most 0.17. Diffusion-based purification regenerates the image content and removes most of the FDFR gain for every evaluated method. Even under DiffPure, the ISM of VEIL remains below that of ORG (0.45 vs. 0.53), whereas the other methods stay between 0.50 and 0.55. With the t@t identifier, 0.33 of the generations are recognized as the reference identity, compared with 0.93 without protection under the same identifier. The revision states the scope of the evaluation in the threat model (Section 3.2).
W2 / Q4. Novelty of the method
The novelty of VEIL lies in attributing the limit on protection to the gradient signal of the surrogate and in designing the whole optimization pipeline around stabilizing and then concentrating this signal. Under a fixed perturbation budget, the gradient signal of the surrogate has two problems. First, it is unstable. The timesteps drawn at random in each iteration and the stochastic VAE sampling give the same image gradients with inconsistent directions across neighboring iterations, so part of the updates cancel out. Second, it is diffuse. Sign-gradient updates spread the perturbation uniformly over all pixels, and attention-based methods suppress all conditioning positions or a fixed subset of them, without distinguishing how much each position contributes to the protection objective. VEIL treats the two as one problem. It first stabilizes the gradient (fixed low-noise anchors and posterior-mean encoding) and then concentrates it (attribution-based selection of conditioning positions on the empty prompt, and saliency-weighted allocation of the perturbation).
Two sets of results directly support this view.
- Fixed low-noise anchors bring a gain under the same iteration budget (Table 9). With the same 6 timesteps, 112 iterations, surrogate, and budget, replacing random timesteps with fixed anchors in the same early interval raises the FDFR by 0.12 (P1) and 0.14 (P2) at the same LPIPS. Moving the anchors from the low-noise end to the middle and late intervals lowers the P2 FDFR from 0.88 to 0.48 and 0.08. This agrees with the observation of UIFace (Lin et al., ICLR 2025) that identity-related details are recovered at low noise levels.
- Under the same budget, VEIL reaches the highest or tied-highest FDFR with a smaller equivalent perturbation. The equivalent RMSE derived from the PSNR in Table 1 is 3.23 / 255 for VEIL (CelebA-HQ), against 4.51–4.89 / 255 for Anti-DB, CAAT, DisDiff, and SimAC with the same budget, about 30% smaller. Removing the spatially selective update and reverting to sign gradients raises the FDFR from 0.83 to 0.86 but degrades LPIPS from 0.134 to 0.163 (Table 3). Concentrated allocation thus trades a small amount of suppression for clearly higher fidelity.
Because the conditioning positions are selected on the empty prompt, constructing the protected images does not depend on the identifier or prompt used by the attacker. The evaluation prompt P1 is also a held-out prompt that does not appear in personalization training.
We hope these additional results address your questions. We will respond promptly to any further questions during the discussion period.
References
[R1] Ang Li et al. PID: Prompt-Independent Data Protection Against Latent Diffusion Models. In ICML, 2024.
Follow-up on our response
Dear Reviewer SV6N,
We appreciate your questions on what the metrics capture. To make the discussion easier, we summarize the main additions. Figures 3 and 4 in the main text show the protected images and generations of all methods, and Table 13 shows that VEIL lowers the fraction of generations recognized as the person to 0.05 to 0.19 (W1 / Q1). Appendix B.2.2 reports how many identities receive the zero ISM (Q2), and the revised Section 2 explains how VEIL stabilizes and then concentrates the gradient signal (W2 / Q4).
As the discussion deadline approaches, we would be happy to clarify any remaining questions. If your concerns have been resolved, we would appreciate your reconsideration of the rating and support for our paper.
Regards, Authors
Reply to Reviewer SV6N [Part 1/2]
We thank the reviewer for the questions. All changes in the revised PDF are highlighted in orange-red, and the new experiments are collected in Appendix F.
W1 / Q1. Do the metric gains reflect identity protection, or only degraded generations?
The gain shows mainly as a large drop in the identity-recovery rate. We directly measured whether the identity is reproduced: the fraction of all generations recognized as the reference identity (a face is detected and its cosine similarity to the reference exceeds the verification threshold of 0.32). Because the denominator is all generations, the metric reflects how often the identity is reproduced:
Without protection, more than 80% of the generations are recognized as the person. VEIL lowers this fraction to 5%–19% (revised Section 5.2, Appendix F Table 13).
The random-noise control shows that a larger perturbation on the released image does not by itself protect the identity. Uniform random noise with twice the L∞ budget of VEIL lowers the PSNR to 30.9 dB (37.9 dB for VEIL), yet its recovery rate stays close to that of ORG. The protection of VEIL therefore cannot be explained by the perturbation magnitude alone.
The new Figure 3 in the revised main text shows the protected images and residuals of all methods. Under the same L∞ budget, the residual of VEIL is visibly weaker than those of DisDiff and SimAC. The new Figure 4 shows the generations of all 10 methods on both datasets under P1 and P2. SDS-, VCPro, and MYOPIA clearly reproduce the person, Anti-DB, Anti-Diff, and CAAT add visible artifacts while the face remains recognizable, and DisDiff, SimAC, and VEIL suppress the identity most strongly. VEIL thus matches DisDiff and SimAC in suppressing the identity in the generations, while leaving a clearly weaker residual on the released image. This is where its fidelity gain in the quantitative metrics comes from.
Q2. Does FDFR indicate reduced identity leakage? How much does the zero assignment affect ISM?
A single generation without a detected face is not assigned zero. It counts only toward FDFR, and ISM averages the ArcFace similarity over the generations in which a face is detected. Only when none of the 32 generations of an identity under a prompt contains a detected face is the ISM of that identity set to 0 (Appendix B.2.2). All methods in the tables are evaluated under the same rule.
Among generations with a detected face, the identity similarity of VEIL is clearly lower. In Table 1, the ISM of VEIL is 0.20 / 0.34 / 0.22 / 0.30 across the four settings, against 0.47 / 0.71 / 0.49 / 0.67 for unprotected ORG. After excluding the identities with no detected face in any generation, the ISM of VEIL rises by at most 0.07 and remains clearly below that of ORG in all four settings. FDFR by itself does not measure identity similarity, which is why the paper also reports ISM, and the unconditional recovery rate above directly answers whether the identity leaks.
The zero rule is consistent with the evaluation code released by the authors of DisDiff (Liu et al., MM 2024b) and the public evaluation code of GAP-Diff (Zhu et al., NDSS 2025). Other public evaluation code also keeps failed generations in the average. PID (Li et al., ICML 2024) [R1] assigns a similarity of 0 to every generation without a detected face, and VCPro (Mi et al., AAAI 2026) assigns the lowest score of −1 to generations that cannot be matched to the reference.
The table below gives the number of identities whose 32 generations all lack a detected face and are therefore assigned 0 (50 identities per dataset, counted separately per prompt):
Among the other methods, CAAT has one such identity in each of two settings, and all others have none. Excluding these identities raises the ISM of the three methods by 0.02–0.07, and the three remain within 0.04 of each other. The exclusion removes the identities for which none of the 32 generations contains a detected face. The remaining identities are those with relatively weaker protection, so the ISM after exclusion is biased toward weaker protection. The recovery rate computed over all generations (W1 / Q1) keeps every generation in the denominator and needs no ISM value for detection failures. Under this metric, VEIL is lowest or tied-lowest in three of the four settings. The revised Appendix B.2.2 adds the range of these counts and the resulting change in ISM.