- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an image immunization framework targeting black-box transferability and purification robustness. The core design targets early VAE features and combines feature differences with interference from cross-channel relationships to optimize defensive perturbations. Experiments across diverse generative models, manipulation tasks, and purification methods demonstrate effective protection with low computational overhead.
Critical strengths
- The research question has clear practical value, where black-box transferability and purification robustness are more closely aligned with real-world applications.
- The experiments are relatively well-designed.
Critical weaknesses
- Early feature perturbation (DiffusionGuard, ICLR'25) and VAE representation perturbation (PCA, TIFS'24) have already been explored in existing research. The advantages of combining these two methods have not yet been fully analyzed, so their integration primarily depends on engineering practice and empirical exploration, lacking substantial technical innovation.
- The paper argues that early VAE representations provide a unified attack surface for both black-box transferability and purification robustness. However, different generative models employ VAEs with distinct feature distributions, latent spaces, and encoding mappings. So, why perturbations optimized on one VAE can consistently disrupt other heterogeneous VAEs while remaining effective after purification?
Questions for authors
- The relationship perturbation loss is constructed using cross-channel cosine similarity, but the paper does not fully explain why disrupting the internal channel relationships in the surrogate model can enhance transferability to unknown models.
- The paper uses PWCCA to explain the cross-model transfer advantages of early VAE features, but it remains unclear whether higher representational correlation implies stronger adversarial transfer capabilities.
- Lack of comparison with the most relevant work: 10.1109/TIFS.2025.3639957
3 comments
[W2, Q2] Cross-VAE Transferability and Purification Robustness
Thank you for these constructive comments. We agree that higher representational correspondence alone does not establish adversarial transferability (L237-241). Our intention was not to establish a direct causal relationship between PWCCA and attack transferability, but to provide complementary evidence for the observed black-box protection behavior. The clean-PWCCA, shift-PWCCA, and latent shift survival analyses in Fig. 3 complement this evaluation by examining cross-encoder correspondence and the persistence of perturbation-induced effects after purification.
Importantly, our analysis goes beyond clean representation correspondence. Despite differences in VAE architectures and latent spaces, perturbation-induced changes also exhibit stronger cross-encoder correspondence at earlier stages (shift-PWCCA, Fig. 3b). This is consistent with the stronger black-box transferability observed in Fig. 2a, suggesting that shared responses to perturbations may help explain the transferability of early-stage attacks.
To further investigate shared responses across encoders, we additionally conduct an input-gradient analysis in response to your concern. At each stage, we compute the gradient of the norm of the channel vector at the central spatial location with respect to the RGB input. We then compare the full input gradients across encoders for the same clean image using cosine similarity. These diagnostic probes characterize input sensitivity without assuming one-to-one channel correspondence; they are not attack-loss gradients.
Table B. Cross-encoder input-gradient cosine similarity
| Depth | SD1.4 ↔ SD3 | SD1.4 ↔ FLUX.1 | SD3 ↔ FLUX.1 |
|---|---|---|---|
| B0 | 0.527 ± 0.207 | 0.515 ± 0.290 | 0.525 ± 0.384 |
| B1 | 0.417 ± 0.225 | 0.315 ± 0.343 | 0.389 ± 0.217 |
| B2 | 0.419 ± 0.125 | 0.134 ± 0.192 | 0.197 ± 0.184 |
| B3 | −0.097 ± 0.197 | −0.020 ± 0.111 | 0.130 ± 0.075 |
| B4 | −0.070 ± 0.171 | −0.003 ± 0.037 | 0.083 ± 0.083 |
As shown in Table B, B0 exhibits the highest mean gradient alignment for all three encoder pairs. This indicates that activation magnitudes at B0 respond to more similar input directions across encoders. Together with shift-PWCCA and the controlled depth-wise evaluation, these observations support the interpretation that shared early-stage input sensitivities and corresponding perturbation-induced responses may contribute to the observed transferability.
We also quantify the association between shift-PWCCA and actual black-box protection performance. For each surrogate–target pair, we compute Spearman and Pearson correlations across B0–B4 using results obtained under our method.
Table C. Correlation between shift-PWCCA and black-box protection performance (LPIPS)
| Surrogate → Target | Spearman ρ | Pearson r |
|---|---|---|
| SD1.4 → SD3 | 0.900 | 0.915 |
| SD1.4 → FLUX.1 | 0.900 | 0.982 |
| SD3 → SD1.4 | 0.900 | 0.977 |
| SD3 → FLUX.1 | 0.700 | 0.720 |
| FLUX.1 → SD1.4 | 1.000 | 0.973 |
| FLUX.1 → SD3 | 0.900 | 0.948 |
As shown in Table C, both correlation measures are positive across all six pairs, with Spearman ranging from 0.700 to 1.000 and Pearson r from 0.720 to 0.982. Thus, attack depths with stronger perturbation-induced cross-encoder correspondence also tend to yield stronger transferred protection. This quantitatively supports the relationship between the shift-PWCCA analysis and the depth-wise protection results.
For purification robustness, latent shift survival (Fig. 3c) measures how much of the perturbation-induced change in the final VAE latent remains after purification, accounting for both directional alignment and retained magnitude. Perturbations targeting early representations show greater latent shift survival, consistent with the protection-retention results in Fig. 2b.
Together, these analyses support early VAE representations as a favorable common attack surface: they exhibit stronger cross-encoder correspondence and input-gradient alignment, while perturbations optimized at these stages better preserve their effects on the final latent after purification. We have incorporated the additional gradient analysis and its detailed protocol into the revised manuscript. We sincerely thank you for helping us strengthen the analysis and clarify this rationale.
[Q1] Clarification on Relation Disruption Loss
Thank you for raising this concern. We distinguish the complementary roles of attack-surface selection and relation disruption. Our rationale for transferability primarily comes from targeting early VAE representations, particularly B0, supported by the controlled depth-wise evaluation and cross-encoder correspondence analyses in Figs. 2a and 3. We introduce to complement at this favorable attack surface.
While directly maximizes feature discrepancy, it does not explicitly require changes in inter-channel relationships. Since subsequent layers combine spatially weighted information across channels, disrupting these relationships can alter how early-stage perturbations are transformed downstream, potentially amplifying distortion in the final VAE latent supplied to the diffusion denoiser. This motivates applying to the normalized relationships between channel activation maps within B0’s residual branches.
To examine this downstream effect, we compare representation changes under the two objectives while keeping the attack location fixed at B0. Because feature scales vary across VAEs and encoder depths, we normalize the representation MSE for each image by the mean squared activation of the corresponding clean representation before averaging.
Table D. Downstream representation distortion measured by scaled MSE of B0-targeted attacks
| Representation | WB | WB | BB | BB |
|---|---|---|---|---|
| B1 | 0.9560 | 0.9126 | 0.8594 | 0.8644 |
| B2 | 0.7339 | 0.7730 | 0.7106 | 0.7450 |
| B3 | 0.3015 | 0.3460 | 0.3016 | 0.3201 |
| B4 | 0.2623 | 0.2893 | 0.2601 | 0.2704 |
| Latent | 0.3783 | 0.3974 | 0.3890 | 0.4037 |
Table E. Reconstruction distortion relative to clean reconstructions under B0-targeted attacks
| Setting | Surrogate → Target | Metric | + | |
|---|---|---|---|---|
| White-box | SD1.4 → SD1.4 | PSNR ↓ | 24.249 | 23.744 |
| LPIPS ↑ | 0.4860 | 0.5255 | ||
| SD3 → SD3 | PSNR ↓ | 25.660 | 25.287 | |
| LPIPS ↑ | 0.4538 | 0.4798 | ||
| FLUX.1 → FLUX.1 | PSNR ↓ | 25.583 | 25.280 | |
| LPIPS ↑ | 0.4886 | 0.5172 | ||
| Black-box | SD1.4 → SD3 | PSNR ↓ | 25.596 | 24.432 |
| LPIPS ↑ | 0.4827 | 0.5220 | ||
| SD1.4 → FLUX.1 | PSNR ↓ | 25.660 | 25.188 | |
| LPIPS ↑ | 0.4837 | 0.5245 | ||
| SD3 → SD1.4 | PSNR ↓ | 24.368 | 24.105 | |
| LPIPS ↑ | 0.4547 | 0.4795 | ||
| SD3 → FLUX.1 | PSNR ↓ | 26.011 | 25.596 | |
| LPIPS ↑ | 0.4512 | 0.4775 | ||
| FLUX.1 → SD1.4 | PSNR ↓ | 24.227 | 24.005 | |
| LPIPS ↑ | 0.4922 | 0.5144 | ||
| FLUX.1 → SD3 | PSNR ↓ | 25.171 | 24.991 | |
| LPIPS ↑ | 0.4891 | 0.5106 |
As shown in Table D, adding generally increases distortion in deeper encoder representations and, most importantly, in the final VAE latent under both white-box and black-box settings. Table E further confirms this effect at the reconstruction level: consistently produces lower PSNR and higher LPIPS than alone across all white-box and black-box surrogate-target pairs. Together with the improved black-box editing protection reported in Table 6, these results support our motivation for designing to complement direct feature displacement by propagating stronger distortion to the latent representation supplied to the diffusion denoiser.
As described in L314–316, this rationale does not assume shared relation matrices or corresponding channel indices across encoders. B0 provides favorable conditions for transfer, while strengthens disruption at that surface and its downstream effects. We sincerely thank you for helping us clarify this rationale. We have added this motivation and the accompanying analysis to the revised manuscript, with the additions highlighted in blue.
[A1] A Gray-Box Attack Against Latent Diffusion Model-Based Image Editing by Posterior Collapse. IEEE TIFS, 2025.
[W1, Q3] Distinction from DiffusionGuard and PCA
Thank you for the valuable comment. We first clarify our distinction from DiffusionGuard and PCA.
DiffusionGuard targets the early stage of the diffusion denoising process, specifically the first denoising step, by maximizing the norm of the noise predicted by the diffusion denoiser. In contrast, early in our method refers to network depth within the VAE encoder, specifically B0. Our optimization targets this earliest intermediate representation without involving the denoising network or selecting diffusion timesteps.
PCA [A1] targets the VAE encoder’s final output by manipulating the posterior mean and variance through a KL-divergence-based objective to induce collapse. In contrast, our method targets early intermediate encoder features, without optimizing the final posterior parameters.
To further examine the transferability benefit of targeting early VAE encoder representations, we additionally compare our method with PCA and DiffusionGuard.
Table A. Comparison with PCA and DiffusionGuard
As shown in Table A, our method achieves strong black-box protection. DiffusionGuard’s weaker transferability is consistent with our analysis that relying on denoiser representations can limit cross-model transfer. Compared with PCA’s final encoder-output objectives, our method targets only early encoder representations and achieves superior overall black-box performance, further supporting our choice of attack surface.
We also acknowledge that encoder-based protection is well established. PhotoGuard’s encoder attack, PCA, DCT-Shield, Editshield, BlurGuard’s encoder-based configuration, and PID have already demonstrated the value of encoder-based objectives for reducing computational cost or dependence on the denoising network. We therefore do not claim encoder-only optimization itself as our contribution.
Our contribution is to identify which encoder representations provide a favorable attack surface for both black-box transferability and purification robustness, and to investigate why. To this end, we conduct controlled depth-wise comparisons under a common objective, analyze cross-encoder correspondence using clean-PWCCA and shift-PWCCA, and measure the survival of perturbation-induced latent shifts after purification. These analyses motivate targeting early encoder representations. We have included the corresponding result in the revision, and will revise the discussion to clarify these distinctions and the scope of our contribution.