acceptodds
Under review as a conference paper at ICLR 2027

VEIL: Visual-State Early-Anchor Stabilization for Identity Leakage Suppression

12 views

Abstract

Text-to-image personalization can bind a subject's visual identity to an identifier using only a few reference images, enabling unauthorized synthesis from publicly shared face photos. User-side defenses perturb images before release to impair downstream identity recovery while preserving visual quality. However, existing methods typically optimize denoising or cross-attention signals without jointly controlling the diffusion states used for supervision and the conditioning positions and image regions targeted during optimization. In this paper, we propose VEIL (Visual-state Early-anchor Identity Leakage suppression), a user-side defense that stabilizes perturbation optimization using fixed ultra-early denoising anchors and posterior-mean latent encoding. It further targets conditioning positions and image regions that contribute most to the protection objective through attribution-guided selection, saliency-weighted updates, and soft shrinkage. Under DreamBooth on CelebA-HQ and VGGFace2, VEIL achieves the highest SSIM and PSNR among the compared defenses on both datasets, with an SSIM of 0.94 and PSNRs of 37.94 and 38.21 dB. Across the four primary dataset–prompt settings, VEIL achieves or ties for the highest FDFR, with values ranging from 0.73 to 0.85. Protection also transfers across the evaluated Stable Diffusion variants and personalization procedures without re-optimizing the released images. Code is available at https://anonymous.4open.science/r/VEAL_demo.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
text-to-image diffusion models, personalization, privacy protection, adversarial perturbation, face identity protection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_SV6NOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose VEIL, which adds small perturbations to user images to suppress identity reproduction after diffusion-model personalization. The method stabilizes optimization using fixed low-noise timesteps and VAE posterior-mean encoding, combined with attribution-guided attention suppression and spatial saliency constraints. Experiments on two face datasets and multiple personalization settings evaluate protection effectiveness and image fidelity.

Critical strengths

1.The method addresses optimization stability, attention-position selection, and visual fidelity with clear design motivations. The experiments are fairly comprehensive, and ablation studies support the role of each component.

  1. Under the main experimental settings, the method achieves a good balance between preserving the original image quality and suppressing personalized generation, with comparisons against multiple existing protection methods.
Critical weaknesses
  1. The quantitative results show some improvement, but the visual advantages over existing methods are not clear in the qualitative comparisons provided in the supplementary material. I find it difficult to judge whether these metric gains translate into meaningful improvements in identity protection, so their practical benefit remains a concern.

  2. The proposed components have clear precedents in related work, and the overall design appears largely combinatorial. The paper does not sufficiently establish a contribution beyond combining existing techniques and ideas, so I find its technical novelty and broader methodological insight limited.

Questions for authors
  1. Could the authors explain, using qualitative examples, where the quantitative improvements are reflected? Do these gains indicate stronger identity protection, or mainly further degradation of generated image quality?

  2. FDFR measures face detection failure. How does this establish reduced identity leakage? Among generations where faces remain detectable, is identity similarity significantly reduced? How much does assigning an ISM of zero when no face is detected affect the comparisons?

  3. Gaussian blur and diffusion-based purification substantially weaken protection, and changing the identifier also reduces effectiveness. How much practical protection remains when an attacker can adjust these settings?

  4. What is the key new design beyond existing timestep selection, attention suppression, and saliency constraints? What evidence demonstrates a new mechanism or advantage beyond combining these components?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer SV6N [Part 1/2]

We thank the reviewer for the questions. All changes in the revised PDF are highlighted in orange-red, and the new experiments are collected in Appendix F.

W1 / Q1. Do the metric gains reflect identity protection, or only degraded generations?

The gain shows mainly as a large drop in the identity-recovery rate. We directly measured whether the identity is reproduced: the fraction of all generations recognized as the reference identity (a face is detected and its cosine similarity to the reference exceeds the verification threshold of 0.32). Because the denominator is all generations, the metric reflects how often the identity is reproduced:

SettingORGRandom noiseVEIL
CelebA-HQ P1 / P20.80 / 0.990.77 / 0.990.05 / 0.12
VGGFace2 P1 / P20.87 / 0.980.87 / 0.980.10 / 0.19

Without protection, more than 80% of the generations are recognized as the person. VEIL lowers this fraction to 5%–19% (revised Section 5.2, Appendix F Table 13).

The random-noise control shows that a larger perturbation on the released image does not by itself protect the identity. Uniform random noise with twice the L∞ budget of VEIL lowers the PSNR to 30.9 dB (37.9 dB for VEIL), yet its recovery rate stays close to that of ORG. The protection of VEIL therefore cannot be explained by the perturbation magnitude alone.

The new Figure 3 in the revised main text shows the protected images and residuals of all methods. Under the same L∞ budget, the residual of VEIL is visibly weaker than those of DisDiff and SimAC. The new Figure 4 shows the generations of all 10 methods on both datasets under P1 and P2. SDS-, VCPro, and MYOPIA clearly reproduce the person, Anti-DB, Anti-Diff, and CAAT add visible artifacts while the face remains recognizable, and DisDiff, SimAC, and VEIL suppress the identity most strongly. VEIL thus matches DisDiff and SimAC in suppressing the identity in the generations, while leaving a clearly weaker residual on the released image. This is where its fidelity gain in the quantitative metrics comes from.

Q2. Does FDFR indicate reduced identity leakage? How much does the zero assignment affect ISM?

A single generation without a detected face is not assigned zero. It counts only toward FDFR, and ISM averages the ArcFace similarity over the generations in which a face is detected. Only when none of the 32 generations of an identity under a prompt contains a detected face is the ISM of that identity set to 0 (Appendix B.2.2). All methods in the tables are evaluated under the same rule.

Among generations with a detected face, the identity similarity of VEIL is clearly lower. In Table 1, the ISM of VEIL is 0.20 / 0.34 / 0.22 / 0.30 across the four settings, against 0.47 / 0.71 / 0.49 / 0.67 for unprotected ORG. After excluding the identities with no detected face in any generation, the ISM of VEIL rises by at most 0.07 and remains clearly below that of ORG in all four settings. FDFR by itself does not measure identity similarity, which is why the paper also reports ISM, and the unconditional recovery rate above directly answers whether the identity leaks.

The zero rule is consistent with the evaluation code released by the authors of DisDiff (Liu et al., MM 2024b) and the public evaluation code of GAP-Diff (Zhu et al., NDSS 2025). Other public evaluation code also keeps failed generations in the average. PID (Li et al., ICML 2024) [R1] assigns a similarity of 0 to every generation without a detected face, and VCPro (Mi et al., AAAI 2026) assigns the lowest score of −1 to generations that cannot be matched to the reference.

The table below gives the number of identities whose 32 generations all lack a detected face and are therefore assigned 0 (50 identities per dataset, counted separately per prompt):

MethodCelebA-HQ P1CelebA-HQ P2VGGFace2 P1VGGFace2 P2
DisDiff76103
SimAC94132
VEIL124115

Among the other methods, CAAT has one such identity in each of two settings, and all others have none. Excluding these identities raises the ISM of the three methods by 0.02–0.07, and the three remain within 0.04 of each other. The exclusion removes the identities for which none of the 32 generations contains a detected face. The remaining identities are those with relatively weaker protection, so the ISM after exclusion is biased toward weaker protection. The recovery rate computed over all generations (W1 / Q1) keeps every generation in the denominator and needs no ISM value for detection failures. Under this metric, VEIL is lowest or tied-lowest in three of the four settings. The revised Appendix B.2.2 adds the range of these counts and the resulting change in ISM.

AuthorsOpenReview ↗

Reply to Reviewer SV6N [Part 2/2]

Q3. How much protection remains when the attacker changes the setup?

Under all four preprocessing operations on CelebA-HQ, VEIL ranks first or second on ISM and CLIP-F (Table 2). Under JPEG compression, VEIL keeps the highest FDFR (0.91), and only 0.06 of its generations are recognized as the reference identity, compared with 0.81 without protection. Gaussian blur brings the FDFR of every method to at most 0.17. Diffusion-based purification regenerates the image content and removes most of the FDFR gain for every evaluated method. Even under DiffPure, the ISM of VEIL remains below that of ORG (0.45 vs. 0.53), whereas the other methods stay between 0.50 and 0.55. With the t@t identifier, 0.33 of the generations are recognized as the reference identity, compared with 0.93 without protection under the same identifier. The revision states the scope of the evaluation in the threat model (Section 3.2).

W2 / Q4. Novelty of the method

The novelty of VEIL lies in attributing the limit on protection to the gradient signal of the surrogate and in designing the whole optimization pipeline around stabilizing and then concentrating this signal. Under a fixed perturbation budget, the gradient signal of the surrogate has two problems. First, it is unstable. The timesteps drawn at random in each iteration and the stochastic VAE sampling give the same image gradients with inconsistent directions across neighboring iterations, so part of the updates cancel out. Second, it is diffuse. Sign-gradient updates spread the perturbation uniformly over all pixels, and attention-based methods suppress all conditioning positions or a fixed subset of them, without distinguishing how much each position contributes to the protection objective. VEIL treats the two as one problem. It first stabilizes the gradient (fixed low-noise anchors and posterior-mean encoding) and then concentrates it (attribution-based selection of conditioning positions on the empty prompt, and saliency-weighted allocation of the perturbation).

Two sets of results directly support this view.

  1. Fixed low-noise anchors bring a gain under the same iteration budget (Table 9). With the same 6 timesteps, 112 iterations, surrogate, and budget, replacing random timesteps with fixed anchors in the same early interval raises the FDFR by 0.12 (P1) and 0.14 (P2) at the same LPIPS. Moving the anchors from the low-noise end to the middle and late intervals lowers the P2 FDFR from 0.88 to 0.48 and 0.08. This agrees with the observation of UIFace (Lin et al., ICLR 2025) that identity-related details are recovered at low noise levels.
  2. Under the same budget, VEIL reaches the highest or tied-highest FDFR with a smaller equivalent perturbation. The equivalent RMSE derived from the PSNR in Table 1 is 3.23 / 255 for VEIL (CelebA-HQ), against 4.51–4.89 / 255 for Anti-DB, CAAT, DisDiff, and SimAC with the same budget, about 30% smaller. Removing the spatially selective update and reverting to sign gradients raises the FDFR from 0.83 to 0.86 but degrades LPIPS from 0.134 to 0.163 (Table 3). Concentrated allocation thus trades a small amount of suppression for clearly higher fidelity.

Because the conditioning positions are selected on the empty prompt, constructing the protected images does not depend on the identifier or prompt used by the attacker. The evaluation prompt P1 is also a held-out prompt that does not appear in personalization training.

We hope these additional results address your questions. We will respond promptly to any further questions during the discussion period.

References

[R1] Ang Li et al. PID: Prompt-Independent Data Protection Against Latent Diffusion Models. In ICML, 2024.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer SV6N,

We appreciate your questions on what the metrics capture. To make the discussion easier, we summarize the main additions. Figures 3 and 4 in the main text show the protected images and generations of all methods, and Table 13 shows that VEIL lowers the fraction of generations recognized as the person to 0.05 to 0.19 (W1 / Q1). Appendix B.2.2 reports how many identities receive the zero ISM (Q2), and the revised Section 2 explains how VEIL stabilizes and then concentrates the gradient signal (W2 / Q4).

As the discussion deadline approaches, we would be happy to clarify any remaining questions. If your concerns have been resolved, we would appreciate your reconsideration of the rating and support for our paper.

Regards, Authors

Reviewer_6njCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies pre-release protection of face images against unauthorized text-to-image personalization. VEIL combines a fixed set of ultra-early diffusion timesteps and posterior-mean VAE encoding to stabilize the surrogate optimization signal, attribution-guided selection of conditioning positions for attention suppression, and saliency-weighted perturbation updates with soft shrinkage. An alternating surrogate-update procedure tracks the evolving protected samples. Evaluations on CelebA-HQ and VGGFace2 against DreamBooth and several published defenses indicate a favorable balance between protected-image fidelity and disruption of downstream facial identity recovery, with additional transfer and preprocessing tests.

Critical strengths
  1. The paper articulates a concrete optimization-design hypothesis: temporal supervision, text-conditioning selectivity, and spatial allocation of a fixed perturbation budget should be considered jointly rather than independently. This framing is coherent and practically motivated.
  2. The main comparison is relatively broad for the targeted task. Table 1 includes denoising-based, attention-based, and fidelity-oriented alternatives; VEIL reports SSIM 0.94 and PSNR 37.94/38.21 dB while attaining FDFR 0.73–0.85 across the four primary dataset–prompt cases.
  3. The paper evaluates more than a single training prompt: P1 is held out, P2 matches training, and the appendix reports P3–P6. Transfer across model variants/personalization routines and robustness to JPEG, blur, and purification are valuable checks (Sec. 5; Apps. C–D).
  4. Ablations on the main modules, anchor placement, and surrogate adaptation improve the interpretability of the engineering choices (Table 3 and Appendix tables 9–10).
Critical weaknesses
  1. The improvement cannot yet be attributed cleanly to the claimed temporal, conditioning, and spatial principles. VEIL introduces multiple coupled changes plus periodically adapted surrogate training. Module-removal ablations help, but do not establish whether gains come from these principles or simply a larger optimization budget or a stronger adaptive surrogate. A compute-matched baseline using the same surrogate updates and number of gradient evaluations would be essential (Sec. 4, Sec. 5.5, App. D).
  2. The attacker is explicitly oblivious (Sec. 3.2). In the practical threat model, an informed attacker could recognize or preprocess protected images, modify fine-tuning duration/objectives, or adapt to the defense. Results against fixed purification operators do not constitute a convincing evaluation of adaptive attacker-side optimization; the security claim should be narrowed or stress-tested.
  3. FDFR counts generations where RetinaFace fails to detect a face (Sec. 5.1). This is a useful failure indicator but does not by itself measure identity suppression. Because ISM and SER-FIQ are computed only for detected faces, the high-FDFR regime causes selection bias in those secondary metrics. It would help to report unconditional identity-related statistics and face-generation utility, ideally using multiple detectors/recognizers.
  4. The fairness of perceptual-quality comparisons is difficult to interpret when different defenses use different allowed perturbation budgets: additive pixel-space methods use epsilon=0.05, whereas MYOPIA uses its much larger default epsilon=0.5 (Sec. 5.1). Equal-budget and equal-runtime Pareto plots would make the fidelity-versus-defense claim more defensible.
  5. The full 50-identity sets are used for the principal DreamBooth setting, while the remaining analyses use only the first 10 identities (Sec. 5.1). Uncertainty intervals, per-identity distributions, and repeated seeds are important, particularly for transfer and ablation conclusions.
  6. The defense is based on SD 2.1 and evaluated across selected Stable Diffusion variants, but it is not established how well the proposed ultra-early anchoring and conditioning-position selection transfer to more modern flow-matching/DiT personalization backbones. The scope of generalization needs explicit limits and, preferably, a stronger contemporary experiment.
Questions for authors
  1. Can the authors provide compute-matched comparisons with adaptive-surrogate and multi-anchor variants of the strongest baselines (same surrogate-training steps, optimization iterations, and wall-clock/GPU cost)?
  2. How many conditioning tokens are selected per iteration, how stable is the selection across iterations/seeds, and how often are non-identifier tokens chosen? Can token-selection plots or frequency statistics be shown?
  3. What are the per-identity confidence intervals and results across multiple protection/random-generation seeds? Do the rankings persist when evaluation identities are randomly sampled rather than taking the first ten?
  4. How does VEIL perform against an attacker who knows the defense family and adapts the personalization learning rate, number of examples/steps, or purification pipeline?
  5. Can identity suppression be evaluated with at least one additional face detector/recognizer and with an unconditional metric that does not discard generations lacking a detected face?
  6. What are the total defense runtime and GPU memory per protected identity compared with DisDiff, SimAC, and VCPro at comparable SSIM/PSNR?
6 comments
AuthorsOpenReview ↗

Reply to Reviewer 6njC [Part 1/3]

We thank the reviewer for the detailed questions. All changes in the revised PDF are highlighted in orange-red. In response, the revision adds three groups of results:

  • The main comparison and per-identity paired tests on the 40 identities not used for tuning, together with repetitions over three random seeds (Appendix F, Tables 11 and 12).
  • An identity-recovery rate computed over all generations, and an evaluation with a second detector and recognizer, MTCNN and VGG-Face (Table 13).
  • Measured perturbation norms, MYOPIA under the unified budget, and runtime and memory under both bf16 and fp32 (Tables 14–16).

W1 / Q1. Do the gains come from the stated principles, or from more compute and an adaptive surrogate?

The adaptive surrogate is not specific to VEIL. The strongest baselines, Anti-DB, SimAC, and DisDiff, already use the ASPL procedure, which alternates surrogate fine-tuning with PGD (50 rounds in the official configuration, each with 3 surrogate steps and 6 PGD steps).

At equal compute, the timestep rule alone yields a clear gain. The paper already contains this control (Table 9).

Temporal principle at equal compute (Table 9, CelebA-HQ). Every row uses 6 timesteps per iteration, 112 iterations, the same surrogate adaptation, and the same budget. Only the rule for choosing timesteps changes:

Timestep ruleLPIPSP1 FDFRP2 FDFR
Fixed {0, 25, …, 125} (VEIL)0.130.950.88
Random in [0, 999], resampled each iter.0.170.830.81
Random in [0, 125], resampled each iter.0.130.830.74
Fixed {400, …, 525}0.200.540.48
Fixed {800, …, 925}0.150.040.08

Table 9 isolates the effect of the timestep rule within the VEIL framework. The row "random in [0, 999]" uses the same uniform timestep sampling as Anti-DB, with an adaptive surrogate, 6 timesteps per iteration, and the same number of iterations, and all other settings follow VEIL. Replacing its random timesteps with fixed low-noise anchors raises the FDFR from 0.83 / 0.81 to 0.95 / 0.88 and lowers LPIPS from 0.17 to 0.13. The revision states the controlled conditions of this comparison in Section 5.5.

Surrogate adaptation (Table 10). With the same 112 iterations, the adaptive surrogate gives an FDFR 0.15 to 0.21 higher than the static surrogate.

W2 / Q4. Oblivious versus adaptive attackers

This paper evaluates an oblivious attacker, who uses existing personalization pipelines with standard preprocessing or purification and does not tailor the attack to the mechanism of VEIL. The revision states this scope in the threat model (Section 3.2). Within this scope, the paper covers variation in both the personalization setup and the purification pipeline. In Appendix D.2, the same protected images are used without re-optimization under three personalization methods with markedly different training configurations: DreamBooth-LoRA (2,000 steps, learning rate 1×10⁻⁴), Textual Inversion (3,000 steps, base learning rate 5×10⁻⁴), and SVDiff (500 steps, learning rates of 1×10⁻³ and 1×10⁻⁶ for matrix-valued and one-dimensional shifts). The P2 FDFR of VEIL is 0.94, 0.62, and 0.86, respectively (Figure 6a). For purification, the paper tests JPEG, Gaussian blur, DiffPure, and GrIDPure (Table 2). Under JPEG compression, VEIL keeps the highest FDFR (0.91), and only 0.06 of its generations are recognized as the reference identity (0.81 without protection). Diffusion-based purification regenerates the image content and removes most of the FDFR gain for every evaluated method.

This threat model is consistent with prior work in this area, including Anti-DreamBooth (Le et al., ICCV 2023), SimAC (Wang et al., CVPR 2024), DisDiff (Liu et al., MM 2024b), and the more recent Anti-Diff (Li et al., AAAI 2025), MYOPIA (Wu et al., AAAI 2025), AntiPure (Yang et al., ICCV 2025), and VCPro (Mi et al., AAAI 2026).

AuthorsOpenReview ↗

Reply to Reviewer 6njC [Part 2/3]

W3 / Q5. FDFR, selection bias, and a single detector

The conclusions hold under a metric computed over all generations and under a second detector. Our face metrics follow the evaluation protocol introduced by Anti-DreamBooth (Le et al., ICCV 2023) and adopted by SimAC (Wang et al., CVPR 2024), DisDiff (Liu et al., MM 2024b), and GAP-Diff (Zhu et al., NDSS 2025). RetinaFace detection gives FDFR and ArcFace gives the identity similarity ISM, both computed with the Anti-DreamBooth evaluation script. All methods in the tables are evaluated with the same script. To address the reviewer's question, we add two evaluations:

  1. Unconditional identity-recovery rate: the fraction of all generations in which a face is detected and its cosine similarity to the reference exceeds the verification threshold of 0.32. The denominator is all generations, so no averaging is restricted to detected faces.
  2. A second detector and recognizer: MTCNN for detection and VGG-Face for similarity, with the same threshold of 0.32. The public evaluation code of VCPro (Mi et al., AAAI 2026) also uses VGG-Face for identity similarity.

Identity-recovery rate (lower is better):

SettingORGRandom noiseDisDiffSimACVEIL
CelebA-HQ P1, ArcFace / VGG-Face0.80 / 0.790.77 / 0.800.07 / 0.070.07 / 0.070.05 / 0.06
CelebA-HQ P2, ArcFace / VGG-Face0.99 / 0.990.99 / 0.990.13 / 0.260.20 / 0.330.12 / 0.23
VGGFace2 P1, ArcFace / VGG-Face0.87 / 0.870.87 / 0.880.10 / 0.120.08 / 0.120.10 / 0.13
VGGFace2 P2, ArcFace / VGG-Face0.98 / 0.980.98 / 0.980.28 / 0.460.26 / 0.430.19 / 0.32

Under both detector–recognizer pairs, VEIL has the lowest recovery rate in three of the four settings, and the two pairs give the same ranking. Random noise with twice the L∞ budget of VEIL leaves the recovery rate close to that of ORG, which shows that a larger perturbation on the released image is not by itself enough to lower it. The revision defines the recovery rate in Section 5.1, reports the result in Section 5.2, and gives the full results in Appendix F (Table 13). The utility of the generated faces is measured by SER-FIQ and CLIP-F in the paper (Table 1).

W4. Perceptual-quality comparison under different budgets

The ε of MYOPIA has a different meaning from that of the other methods. Its perturbation is first scaled pixel-wise by a JND (just-noticeable-difference) map and then clipped to ε in the [-1, 1] space. The actual magnitude is set mainly by the JND map, ε = 0.5 is only an outer bound, and the measured L∞ is 0.15 in [0, 1] units.

Under an equal budget, VEIL still has a PSNR about 3 dB higher and the highest or tied-highest FDFR. For a fair comparison, we used a unified budget of ε = 0.05 for all additive pixel-space methods as far as possible. The official implementations define ε on different pixel ranges, some on [0, 1] and some on [-1, 1]. To keep each compared implementation intact, we did not modify the official code, so the same ε = 0.05 does not correspond to exactly the same perturbation magnitude. We measured the L∞ of every released protected image, in [0, 1] units:

MethodMeasured L∞
VEIL0.024
Anti-DB, SimAC, DisDiff, CAAT0.024
SDS-, VCPro0.051
Anti-Diff0.059
MYOPIA (JND-weighted, ε = 0.5)0.15

SDS-, VCPro, and Anti-Diff use a larger actual perturbation than VEIL. The four methods with the same magnitude as VEIL form a strict equal-budget comparison. VEIL reaches a PSNR of 37.94 / 38.21 dB on CelebA-HQ / VGGFace2, against 34.34–35.04 / 34.52–35.23 dB for these four methods, and its FDFR is the highest or tied-highest in all four settings (Table 1). SDS- and VCPro have a measured L∞ about twice that of VEIL, and their FDFR is still at most 0.02. The revision describes the pixel range of each method in Section 5.1 and Appendix B.5, states the equal-budget fidelity comparison in Section 5.2, and reports the measured norms in Appendix F (Table 14).

For MYOPIA, we use its official default ε = 0.5. To check whether a unified budget changes the conclusion, we also ran MYOPIA at ε = 0.05 on the first 10 identities of each dataset. Its SSIM is then 0.96, but it gives almost no identity protection: the FDFR is at most 0.01, and 0.86–0.95 of the generations are recognized as the reference identity, comparable to unprotected ORG (0.77–0.98). At ε = 0.5 this fraction is likewise 0.76–0.94. On the same identities, VEIL lowers it to 0.01–0.19. Under a unified budget, the advantage of VEIL in identity suppression therefore remains (Appendix B.5 and Appendix F, Table 15).

AuthorsOpenReview ↗

Reply to Reviewer 6njC [Part 3/3]

W5 / Q3. Uncertainty, and the use of only the first 10 identities

The results are stable on held-out identities, under random subsampling, and across seeds. We re-evaluated the main comparison on the 40 identities not used for hyperparameter selection and report per-identity paired bootstrap intervals and Wilcoxon tests:

  • SSIM and PSNR are significantly higher than every baseline on both datasets. Against DisDiff and SimAC, which share the budget of VEIL, the PSNR gain is about 3 dB with 95% CI lower bounds above 2.8 dB.
  • FDFR is significantly higher than Anti-DB, SDS-, VCPro, MYOPIA, Anti-Diff, and CAAT, and differs from DisDiff and SimAC by −0.06 to +0.08.

To test rank stability, we also drew 10 identities at random from the 50 identities of the main experiment, 1,000 times. In all four settings, the mean FDFR rank of VEIL lies within the top 3 of the 9 methods (1.31–2.35), and VEIL ranks first in 70% and 76% of the draws on CelebA-HQ P1 and VGGFace2 P2.

On seeds, we retrained DreamBooth and regenerated the images with two additional random seeds (0 and 2026, in addition to the original 42), keeping the protected images (seed 42) fixed, and compared VEIL, SimAC, and DisDiff on the first 10 identities of each dataset. VEIL has the highest three-seed mean FDFR in all four settings and ranks first in 11 of the 12 seed–setting combinations (revised Section 5.2, Appendix F Table 12).

W6. Flow-matching and DiT backbones

The evaluation scope of this paper is the Stable Diffusion U-Net backbone, consistent with prior work including Anti-DreamBooth, SimAC, and DisDiff, and the more recent Anti-Diff, MYOPIA, AntiPure, and VCPro.

Flow-matching and DiT backbones differ from U-Net diffusion models in three respects: the noise schedule, the timestep distribution sampled during personalization training, and the way text conditioning is injected (MMDiT uses joint image–text attention instead of separate cross-attention layers). Extending VEIL to these backbones requires re-determining the anchor positions and the conditioning positions with respect to these three points, which we leave to future work. The revision states the evaluation scope in the threat model (Section 3.2).

Q2. Which conditioning positions are selected

The conditioning positions are selected by attribution on the empty-prompt encoding of the surrogate (Section 4.3). This sequence contains no identifier token, so all selected positions are non-identifier positions, and constructing the protected images does not require knowing the identifier or prompt used by the attacker. For SD v2.1, the encoding has N = 77 positions: the start token followed by end and padding tokens. With ρ = 0.6, k = ⌊ρN⌋ = 46 positions are reselected by attribution score at every anchor and every iteration. Table 3 shows that adding attribution-guided selection on top of the attention loss raises the FDFR from 0.80 to 0.83 and lowers CLIP-F from 0.138 to 0.129.

Q6. Runtime and memory

Aligned by SSIM / PSNR, the only method in Table 1 close to VEIL is VCPro (37.40 vs. 37.94 dB), but its FDFR is at most 0.02 and it provides almost no protection. DisDiff and SimAC reach an FDFR comparable to VEIL, but their PSNR is about 3 dB lower, so their perturbation is more visible under the same budget.

The runtime and memory of these methods depend mainly on the number of iterations, the surrogate model, and the numerical precision. With these settings fixed, the perturbation budget only bounds each update, and runtime and memory remain essentially unchanged. The table below reports the protection time and peak memory per identity (4 images, including model loading), measured under the same protocol on one A100 (40 GB), for both bf16 and fp32 model precision. The official implementations of DisDiff, SimAC, and VCPro run in bf16.

Methodbf16: time / memoryfp32: time / memory
DisDiff313 s / 17.6 GB687 s / 36.2 GB
SimAC372 s / 29.8 GB— / 40 GB+
VCPro114 s / 10.7 GB154 s / 21.0 GB
VEIL661 s / 14.2 GB1,448 s / 26.9 GB

Under both precisions, the runtime of VEIL is of the same order as that of Anti-DB and MYOPIA (661 s, 481 s, and 608 s in bf16), and its peak memory is lower than that of DisDiff and SimAC. SimAC exceeds 40 GB and cannot run in fp32. Across all 8 baselines in the paper, VEIL uses less peak memory than Anti-DB, DisDiff, MYOPIA, and SimAC under both precisions. All protected images of VEIL in the paper are generated in fp32, and the bf16 column changes only the model precision. The full comparison is in the revised Appendix B.3 and Appendix F (Table 16).

We hope these additional results address your questions. We will respond promptly to any further questions during the discussion period.

Reviewer_BsSmOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VEIL, a defense method that prevents facial photos from being used without authorization to train diffusion models. The method adds bounded perturbations before the photos are published, reducing the ability of personalized models to recover the target person's identity while preserving the visual quality of the photos as much as possible. VEIL uses fixed low-noise diffusion timesteps and VAE posterior mean encoding to stabilize perturbation optimization, selects conditioning positions in the empty-prompt encoding sequence based on first-order attribution scores and suppresses their cross-attention, and combines saliency weighting with soft shrinkage to achieve spatially selective perturbation updates. In addition, the method alternates between surrogate model adaptation and perturbation optimization. The authors conduct experiments on CelebA-HQ and VGGFace2, demonstrating a good balance between protection effectiveness and image quality under the main DreamBooth setting, and further evaluate performance across models, personalization methods, changes in identity identifier tokens, and image preprocessing conditions.

Critical strengths

(1) A favorable trade-off between protection and image fidelity. In Table 1, VEIL achieves the highest SSIM and PSNR on both datasets, along with the highest or tied-highest FDFR in all four primary dataset–prompt settings. Although its identity similarity scores are not consistently the lowest, the combination of image fidelity and generation suppression is useful for a defense applied before photos are shared.

(2) Controlled ablations support the main design choices. Table 9 compares fixed anchors with random sampling using six timesteps within the same low-noise range. Fixed anchors achieve higher FDFR under both prompts at comparable LPIPS, supporting the choice to keep the timesteps fixed during optimization. Table 3 examines posterior-mean encoding, conditioning-position selection, and spatial updates. Removing the spatially selective update increases FDFR but worsens LPIPS, showing that this component helps preserve image quality at some cost to generation suppression.

Critical weaknesses

(1) The reliability of the smaller performance gains remains unclear. Appendix B.3 states that ρ and λattn are selected using the first 10 identities in each dataset. These identities are also used for the robustness, transfer, and ablation experiments. The main evaluation includes all 50 identities, but results for the 40 identities not used to select these parameters are not reported separately. The reported configuration uses seed 42, with 32 generated images per identity–prompt pair and matched generation seeds across methods. However, the paper does not report variation across independent repetitions of protection and personalization training. It is therefore unclear whether small gains, such as VEIL’s FDFR of 0.83 versus DisDiff’s 0.82 on CelebA-HQ under P2, are consistent across runs.

(2) Protection is sensitive to preprocessing and the choice of identifier token. In the robustness experiments on the first 10 CelebA-HQ identities, averaged over P1 and P2, VEIL’s FDFR falls to 0.17 after Gaussian blur, 0.04 after DiffPure, and 0.03 after GrIDPure. On the same set of identities, changing the identifier token from sks to t@t reduces FDFR under P2 from 0.88 to 0.32. These results limit confidence in the protection offered when released images are processed or the personalization configuration changes.

Questions for authors

(1) Appendix B.3 states that ρ and λattn are selected using the first 10 identities in each dataset. Could you separately report the main comparisons on the remaining 40 identities and provide identity-level paired confidence intervals relative to the main baselines to assess whether some of the smaller advantages are stable?

(2) Table 8 does not report ORG results trained on clean photos using the t@t identifier token. Could you add this reference to help distinguish the effect of the identifier token itself on personalization capability from changes in defense effectiveness?

5 comments
AuthorsOpenReview ↗

Reply to Reviewer BsSm [Part 1/2]

We thank the reviewer for the careful reading and the concrete suggestions. All changes in the revised PDF are highlighted in orange-red, and the new experiments are collected in Appendix F.

W1 / Q1. Hyperparameter selection on the first 10 identities, and stability of the results

On the 40 identities not used for selection, VEIL still attains significantly higher SSIM and PSNR than every baseline, and its suppression is on par with the strongest baselines. ρ and λ_attn were selected only on the first 10 identities of each dataset (Appendix B.3). We re-evaluated the main comparison on the remaining 40 identities:

DatasetMethodSSIMPSNRP1 FDFRP2 FDFR
CelebA-HQAnti-DB0.9034.320.430.38
CAAT0.9134.450.440.60
VCPro0.9237.380.030.01
DisDiff0.9235.020.780.81
SimAC0.9235.010.780.73
VEIL0.9437.910.830.80
VGGFace2Anti-DB0.9034.530.450.34
CAAT0.9034.660.520.48
VCPro0.9036.460.020.01
DisDiff0.9135.240.840.65
SimAC0.9135.210.850.68
VEIL0.9438.240.790.73

Paired tests across identities (10,000 bootstrap resamples and the Wilcoxon signed-rank test) lead to two conclusions.

  1. The fidelity advantage is stable. On these 40 identities, VEIL attains significantly higher SSIM and PSNR than every baseline. Against the strongest baselines with the same budget, the PSNR gain is 2.9 dB over DisDiff (95% CI [2.80, 2.97]) and 3.0 dB over SimAC (VGGFace2, [2.95, 3.12]). Against VCPro, the baseline closest in fidelity, the gain is still 0.53 dB (CelebA-HQ, [0.12, 0.95]) and 1.78 dB (VGGFace2, [1.35, 2.20]).
  2. The suppression of VEIL is on par with the strongest baselines. On these 40 identities, VEIL attains a significantly higher FDFR than Anti-DB, SDS-, VCPro, MYOPIA, Anti-Diff, and CAAT (p < 0.001). Against DisDiff and SimAC, which share its budget, the FDFR differences range from −0.06 to +0.08 across the four settings:
FDFR difference of VEIL (95% CI)vs. DisDiffvs. SimAC
CelebA-HQ P1+0.05 [−0.03, 0.13]+0.05 [−0.03, 0.13]
CelebA-HQ P2−0.01 [−0.09, 0.06]+0.07 [−0.02, 0.16]
VGGFace2 P1−0.05 [−0.12, 0.01]−0.06 [−0.12, −0.005]
VGGFace2 P2+0.08 [−0.01, 0.16]+0.05 [−0.04, 0.14]

Under the same budget, VEIL therefore matches the suppression of the strongest baselines with a PSNR about 3 dB higher.

Independent repetitions (first 10 identities, protected images fixed): VEIL has the highest three-seed mean FDFR in all four settings. For the example the reviewer raised, CelebA-HQ P2, VEIL ranks first under all three seeds, with a mean of 0.84 ± 0.03 (DisDiff 0.82 ± 0.04, SimAC 0.72 ± 0.06). We retrained DreamBooth and regenerated the images with two additional random seeds (0 and 2026, in addition to the original 42) and compared VEIL, SimAC, and DisDiff. The other settings follow the same pattern, for example 0.92 ± 0.02 on CelebA-HQ P1 (SimAC 0.80 ± 0.06, DisDiff 0.86 ± 0.00) and 0.74 ± 0.01 on VGGFace2 P2 (SimAC 0.56 ± 0.07, DisDiff 0.62 ± 0.03). Among the three methods, VEIL ranks first in 11 of the 12 seed–setting combinations. The revision summarizes both results in Section 5.2 and gives the full tables in Appendix F (Tables 11 and 12).

AuthorsOpenReview ↗

Reply to Reviewer BsSm [Part 2/2]

Q2. ORG under the t@t identifier

With t@t, personalization on clean images remains effective, and both the identity similarity and the recovery rate of VEIL stay well below those of ORG under the same identifier. The clean reference comes from the same evaluation run (CelebA-HQ, first 10 identities):

P1 FDFRP1 ISMP2 FDFRP2 ISM
ORG, t@t0.000.480.000.71
VEIL, t@t0.440.180.320.43

With t@t, ORG reaches an ISM of 0.48 under P1 and 0.71 under P2 with an FDFR of 0. The identity similarity of VEIL under t@t stays well below the clean reference. Averaged over the two prompts, 0.33 of the generations are recognized as the reference identity, compared with 0.93 for ORG. The revision adds the ORG row to Table 8.

W2. Sensitivity to preprocessing and to the identifier

Under all four preprocessing operations on CelebA-HQ, VEIL ranks first or second on ISM and CLIP-F (Table 2). Under JPEG compression, VEIL keeps the highest FDFR (0.91) and the lowest CLIP-F, and only 0.06 of its generations are recognized as the reference identity, compared with 0.81 without protection. Gaussian blur brings the FDFR of every method to at most 0.17. Diffusion-based purification regenerates the image content and removes most of the FDFR gain for every evaluated method. Even under DiffPure, the ISM of VEIL remains below that of ORG (0.45 vs. 0.53), whereas the other methods stay between 0.50 and 0.55. With the t@t identifier, 0.33 of the generations are recognized as the reference identity, compared with 0.93 for ORG under the same identifier. The revision states the scope of the evaluation in the threat model (Section 3.2).

We hope these additional results address your questions. We will respond promptly to any further questions during the discussion period.

AuthorsOpenReview ↗

Follow-up on our response

Dear Reviewer BsSm,

Thank you for your supportive assessment and for recognizing the balance between protection and image fidelity in VEIL. To make the discussion easier, we summarize the main additions. We reported the main comparison on the 40 held-out identities with paired confidence intervals (Table 11, W1 / Q1), repeated DreamBooth training with three seeds (Table 12, W1), added ORG under t@t (Table 8, Q2), and added the identity-recovery rate under JPEG compression (Section 5.3, W2).

As the discussion deadline approaches, we would be happy to clarify any remaining questions. If our response has addressed your concerns, we would appreciate your continued support and your consideration of a higher rating.

Regards, Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of revisions

Dear Area Chair and Reviewers,

We thank you for your time and constructive feedback. For convenience during the discussion, we summarize how the revision addresses the main points of each review. All changes in the revised PDF are highlighted in orange-red, and the new experiments are in Appendix F.

Reviewer BsSm (stability of the gains)

  • On the 40 identities not used for hyperparameter selection, VEIL attains significantly higher SSIM and PSNR than every baseline, and its FDFR is on par with DisDiff and SimAC under the same budget (Table 11).
  • With DreamBooth retrained under three random seeds on the first 10 identities, VEIL has the highest mean FDFR in all four settings, including the CelebA-HQ P2 example raised in the review (Table 12).
  • ORG under the t@t identifier is now reported (Table 8).

Reviewer 6njC (source of the gain and evaluation)

  • In the compute-matched control (Table 9), replacing random timesteps with fixed low-noise anchors alone raises the FDFR from 0.83 and 0.81 to 0.95 and 0.88 under P1 and P2, with the iterations, timesteps per iteration, surrogate adaptation, and budget held identical.
  • An identity-recovery rate over all generations, with a second detector and recognizer, gives the same conclusion (Table 13).
  • The measured perturbation norms show that VEIL shares the budget of Anti-DB, SimAC, DisDiff, and CAAT, with a PSNR about 3 dB higher (Table 14). Under the unified budget, MYOPIA provides almost no identity protection (Table 15).
  • Runtime and memory are reported under both bf16 and fp32 (Table 16).

Reviewer SV6N (identity protection and method design)

  • VEIL lowers the fraction of generations recognized as the person from 0.80 to 0.99 without protection to 0.05 to 0.19, whereas random noise with twice the budget of VEIL stays at the unprotected level (Table 13).
  • Figures 3 and 4 in the main text show the protected images and generations of all methods.
  • The revised Section 2 presents VEIL as one pipeline that first stabilizes and then concentrates the gradient signal of the surrogate.

We hope this summary is helpful for the discussion. Thank you again.

Regards, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.