acceptodds
Under review as a conference paper at ICLR 2027

VTEGuard: Proactive Defense against Diffusion-Based Visual Text Editing

Abstract

Latent diffusion models enable realistic visual text editing but also expose publicly shared images to unauthorized textual manipulation. A central challenge in proactive protection is to prevent the intended text edits while preserving the visual quality of the released image. However, overall visual degradation of the edited output does not necessarily prevent the target text from being correctly rendered, motivating a focused investigation of protection against diffusion-based visual text editing. We propose VTEGuard, a proactive defense that couples full-image frequency-domain perturbation optimization with variational autoencoder (VAE) posterior-variance disruption. Specifically, VTEGuard parameterizes protective perturbations in the full-image YCbCr discrete cosine transform (DCT) domain, enabling globally structured modifications under a bounded perturbation constraint. We further introduce an Entropic-Smoothed Linear (ESL) objective that combines global posterior-variance expansion with adaptive emphasis on low-variance latent dimensions. This objective maintains dense, non-saturating gradients in variance space and guides the frequency coefficients through the VAE encoder and inverse frequency transform. Experiments on VTEGuardBench and SDEdit, DiffEdit, DreamBooth-based personalization, and diffusion-based inpainting show that VTEGuard achieves the best performance among the evaluated methods across multiple metrics. These results demonstrate a favorable trade-off between protection effectiveness and protected-image fidelity, while highlighting its broad applicability beyond visual text editing.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
proactive image protection, latent diffusion models, visual text editing, adversarial perturbations

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_R7LEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes VTEGuard, which protects images containing text from being edited by diffusion-based image text editors. Instead of optimizing perturbations in RGB pixel space, the method optimizes them in the YCC-DCT coefficient space and uses a loss based on the posterior variance of the VAE encoder, showing strong performance on text editing tasks.

Critical strengths
  • I think protection for the specific application of text forgery is very reasonable, and the benchmark protocol for this setting is also very good.

  • On this benchmark, the proposed method shows stronger protection with less perturbation, and since the method explicitly considers JPEG during protection, it also shows robustness against JPEG compression as purification. However, as far as I understand, the comparison with DCT-Shield is not clearly shown.

  • The comparison study is broad and clear, including evaluations against various recent methods and across multiple tasks.

Critical weaknesses
  • The paper overall seems to argue that visual text is different from the general image domain, but the proposed method itself looks more like a general image protection method rather than something specifically designed for visual text. I am not sure which part of the method is specifically tailored to visual text. The baselines are also mostly general image protection methods, although I understand that this is because there are few existing methods for this setting. Then, I am curious how the method performs on other image domains.

  • As discussed in the paper, the method seems to combine ideas from two existing research directions, such as LA or VOID, with JPEG-oriented frequency-focused methods such as DCT-Shield, and then integrate them through the ESL loss. I also wonder whether the motivation behind ESL is largely based on the analysis already presented in LA. Therefore, I am not fully convinced by the methodological novelty.

  • For the approach of increasing the posterior variance, I wonder whether it can be easily bypassed if the attacker changes the encoding process, e.g., through purification. In particular, since the method targets open-source models, if an attacker understands how the protection works, it may be relatively easy to bypass. Therefore, although the reported performance is very strong, I wonder whether the method may be impractical under an adaptive attacker.

  • For purification, I think comparison with DCT-Shield is necessary. Also, although the method works well against JPEG, more evaluation is needed to verify whether it is robust against other purification methods. In particular, methods such as [1, 2] should be included. I also think TESR and related metrics should be reported under purification settings.

  • Regarding the black-box or transferability claim, if different models use the same encoder, I wonder whether this should really be considered black-box. Since the proposed method directly targets the encoder posterior, I think results should also be reported for models with different VAEs, such as SDXL and SD3.0.

[1] Adversarial Perturbations Cannot Reliably Protect Artists From Generative AI, ICLR25

[2] Off-The-Shelf Image-to-Image Models Are All You Need To Defeat Image Protection Schemes, SaTML26

Questions for authors

See the weaknesses. In addition, I am curious what specific advantage comes from specializing the method for visual text. To me, the text-specific contribution seems to be mainly in the evaluation metrics and benchmark, rather than in the method itself.

Reviewer_VdkVOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies proactive protection of text-containing images against unauthorized diffusion-based visual text editing. Existing protective perturbations mainly target general image degradation, so the edited target text often remains readable, and pixel-space or block-wise DCT perturbations can be visible or fragile to purification. To address this, the paper proposes VTEGuard, which optimizes perturbations in a global YCbCr-DCT space to disrupt the VAE latent encoding of the editor. Specifically, it combines a full-image DCT parameterization with an Entropic-Smoothed Linear (ESL) loss that expands the VAE posterior variance while emphasizing low-variance dimensions, and claims improved imperceptibility and protection. Experiments on the new VTEGuardBench, including an OCR-based text editing success rate, show that VTEGuard outperforms various baselines, and ablation, transfer, and purification tests effectively support its contribution.

Critical strengths
  • S1: The paper addresses a timely and practical problem that prior image protection work has rarely studied: unauthorized text replacement with diffusion-based visual text editors. It points out that image-level degradation is not enough for this task, because a degraded output can still deliver the forged message. It also measures text editing success directly with a clearly defined OCR-based metric (TESR), which suits this task better than image-level metrics alone.
  • S2: VTEGuard achieves strong protection and high imperceptibility at the same time, which is the main trade-off in image protection. On VTEGuardBench, it reduces the text editing success rate to almost zero while keeping a clearly higher PSNR than all baselines. The magnified views in the qualitative results also show fewer visible artifacts than the baselines.
Critical weaknesses
  • W1: Gap between the text-specific claim and a text-agnostic, incremental method. The paper is motivated as a text-specific defense, but the method is a VAE-variance attack that destroys the whole output, and it is structurally close to the LA and LE losses of prior work. As a result, both the novelty of the problem setting and the technical novelty are weak, and the low TESR does not show text-aware protection.
  • W2: Practical validity is not shown beyond SD 1.5-family editors and simple purification. All editors share the attacked SD 1.5 VAE family, so the transfer results are close to white-box. Modern editors with different encoders and adaptive or stronger purification are not evaluated, even though the noise outputs make the protection obvious to an attacker.
Questions for authors
  • Q1: Since the optimization uses only the VAE posterior variance, which part of VTEGuard is specific to visual text editing? Could you compare ESL with the LA and LE losses under the same YCbCr-DCT parameterization? Could you also include in the main benchmark recent methods that attack other components, such as DIA (denoising trajectory) and Lo et al. (cross-attention), and report results with the perturbation restricted to the text region (mask M)?
  • Q2: How does VTEGuard perform on editors whose encoders differ from the SD 1.5 VAE, such as SDXL, SD3/3.5, FLUX-based editors, or recent DiT-based text editors? If the protection does not transfer, can the objective be extended to multiple encoders, and how would this affect imperceptibility?
  • Q3: Why does the perturbation survive JPEG compression but largely disappear under mild Gaussian smoothing? Could you show the frequency spectrum and effective magnitude (e.g., ε_freq and ℓ2 norm) of the learned perturbation, compared with baselines at matched PSNR? Could you also report TESR under several purification strengths and under adaptive purification such as VAE re-encoding or DiffPure?
Reviewer_U9rpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VTEGuard, a proactive protection framework against unauthorized image editing, specifically focused on visual text editing. It optimizes imperceptible protective perturbations directly in the frequency domain to increase VAE posterior variances while additionally emphasizing smaller variance components. Extensive experimental results show that the proposed framework outperforms diverse baseline protection methods both in perturbation imperceptibility and protection effectiveness. Moreover, the authors provide with extensive qualitative examples that supports the quantitative results.

Critical strengths

1. Strong quantitative results. VTEGuard outperforms all proposed baseline protection methods both in perturbation imperceptibility and protection effectiveness. Extensive qualitative examples on how the edited images collapse further support the effectiveness of its protection.

2. Extensive evaluation on diverse image editing tasks. The authors also provide results on other image editing tasks with diverse datasets, demonstrating that the protection effectiveness of VTEGuard is not bounded only to visual text editing task.

Critical weaknesses

W1. Lack of analysis on perturbations in the frequency domain. The paper reports promising results but provides limited analysis of how the proposed method achieves improved imperceptibility and protection effectiveness. Since optimizing perturbations in the frequency domain is presented as a key factor behind these improvements, further analysis would strengthen this explanation. For example, why does protection effectiveness decrease more under Gaussian smoothing than under JPEG compression at quality 75, although both can suppress high-frequency perturbations? Examining perturbation spectra before and after purification, or varying JPEG quality levels and Gaussian smoothing strengths or kernel sizes, could help explain which frequency components support VTEGuard’s effectiveness. These are illustrative suggestions; I do not expect all of these experiments to be included.

W2. Weak connection between the proposed method and visual text editing task. The connection between the proposed framework and visual text editing could be clearer. The objective promotes VAE posterior-variance expansion without explicitly incorporating text-related information. Although the results demonstrate effectiveness on text-editing tasks, further analysis would help explain why the proposed method is effective at preventing visual text editing.

Questions for authors

Q1. Clarification on frequency-domain perturbation budget. Could the authors clarify the value used for ϵfreq\epsilon_{\mathrm{freq}} in Algorithm 1? I could not locate its numerical value in the paper.

Q2. Robustness to inference-time random seeds. Could the authors evaluate VTEGuard across different inference-time random seeds for each fixed protected image? This is important since a malicious user could repeatedly attempt the same unauthorized edit with different sampling seeds until one succeeds. Reporting the mean protection effectiveness and standard deviation across seeds would help assess VTEGuard’s practical effectiveness.

Q3. Applicability to recent image-editing models. Could VTEGuard also optimize perturbations directly against recent image editing models, such as FLUX-based instruction-driven editors? To my understanding, TextDiffuser2 and AnyText2 use SD 1.5-based VAE encoders, and the additional evaluations also focus on SD 1.x/2.x-based models. I am particularly interested in whether VTEGuard can generate effective protective perturbations for models with different underlying architectures and stronger editing capabilities.

Q4. perturbation imperceptibility on other image editing tasks. Could the authors provide the perturbation imperceptibility on other image editing tasks, which correspond with the protection effectiveness reported in Table 3?

I am willing to increase my rating if these weaknessses and questions are adequately addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.