acceptodds
Withdrawn from ICLR 2027

DeContext as Defense: Safe Image Editing in Diffusion Transformers

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In-context diffusion models allow users to modify images with remarkable ease and realism. However, the same power raises serious privacy concerns: personal images can be easily manipulated for identity impersonation, misinformation, or other malicious uses, all without the owner’s consent. While prior work has explored safeguarding against misuse in personalized text-to-image generation, the robustness of modern, large-scale in-context DiT-based models remains largely unexamined. In this paper, we propose DeContext, a new method to safeguard input images from unauthorized in-context editing. Our key insight is that contextual information from the source image propagates to the output primarily through multimodal attention layers. By injecting small, targeted perturbations that weaken these cross-attention pathways, DeContext breaks this flow, effectively decoupling the link between input and output. This simple defense is both efficient and robust. We further show that early denoising steps and specific transformer blocks dominate context propagation, which allows us to concentrate perturbations where they matter most. Experiments on Flux Kontext and Step1X-Edit show that DeContext consistently blocks unwanted image edits while preserving visual quality. These results highlight the effectiveness of attention-based perturbations as a powerful defense against image manipulation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Generative models, adversarial attack

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_oeZCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DeContext, an adversarial perturbation-based defense against unauthorized identity-preserving image editing in Diffusion Transformer (DiT) models.

The authors first conduct attention intervention experiments and identify the attention connections from target queries to context keys as an important pathway for preserving identity information from reference images. Motivated by this observation, they formulate an adversarial optimization objective that minimizes the attention mass assigned by target tokens to context tokens. The input image is then optimized using Projected Gradient Ascent (PGA) under a bounded perturbation constraint, while keeping the model parameters frozen. Furthermore, the authors analyze the importance of different diffusion timesteps and Transformer blocks, concentrating the optimization on early denoising stages and selected single-stream blocks.

Experiments on VGGFace2 and CelebA-HQ, using FLUX Kontext and Step1X-Edit, demonstrate that DeContext can substantially reduce identity similarity between the reference and generated images while maintaining relatively natural-looking outputs.

Critical strengths

S1. Insightful mechanism analysis and a simple yet effective method. The authors investigate the influence of different attention directions between target and context tokens through controlled attention intervention experiments. Their findings highlight the importance of target-query-to-context-key attention connections in preserving identity information from reference images. This observation provides useful insights into contextual information propagation in DiT-based image editing models and offers a clear motivation for the proposed method. Building on this finding, the authors formulate context attention suppression as a constrained optimization problem. By applying Projected Gradient Ascent to the input image while keeping model parameters frozen, DeContext reduces the attention allocated to context information. The method is conceptually simple, and its effectiveness is further improved by focusing on influential diffusion timesteps and Transformer blocks. The clear connection between the initial mechanism analysis and the subsequent method design is a key strength supporting the contribution of this work.

S2. Promising experimental results supporting the effectiveness of the proposed method. The authors evaluate DeContext on VGGFace2 and CelebA-HQ and demonstrate its effectiveness on two DiT-based image editing models, FLUX Kontext and Step1X-Edit. For example, on CelebA-HQ with the prompt "A photo of this person," DeContext reduces the identity similarity score (ISM) from 0.79 to 0.12, substantially weakening identity preservation while producing relatively natural-looking images. In addition, the ablation studies on perturbation budgets, diffusion timesteps, and Transformer blocks provide empirical support for the main design choices. The authors also conduct preliminary robustness evaluations under common image preprocessing operations. These results provide convincing evidence of the method's effectiveness under the evaluated settings and constitute an important strength of the paper.

Critical weaknesses

W1. Insufficient fine-grained quantitative evidence for the causal relationship between attention connections and identity preservation. The authors demonstrate the importance of target-query-to-context-key attention through the attention masking experiments in Figure 2. They further analyze attention distributions and attack effectiveness across diffusion timesteps and Transformer blocks in Figure 4 and Table 5. However, these experiments do not directly quantify the causal contribution of specific attention connections to identity preservation. In particular, a higher attention mass does not necessarily imply a greater contribution to information transfer or to the final generated output. Therefore, while the experiments support the effectiveness of the proposed intervention strategy, the underlying explanation of contextual information propagation remains somewhat incomplete. This limitation weakens the paper's contribution in terms of understanding the internal mechanisms of DiT-based image editing models.

W2. Insufficient evaluation of adversarial perturbation generalization across models and prompts. As a defense intended for practical image protection, the effectiveness of DeContext depends not only on its performance under specific optimization settings but also on whether the resulting perturbations remain effective under unseen generation conditions. First, although the authors evaluate DeContext on Step1X-Edit, the current description does not clearly establish whether the same adversarial image optimized for one model can be directly transferred to another model without re-optimization. Since real-world adversaries may freely choose different image editing models, this issue is important for assessing the practical effectiveness of the defense. Second, the main quantitative evaluations of facial identity protection cover a relatively limited number of prompts. Moreover, some evaluation prompts in Table 2 overlap with the prompt pool used during perturbation optimization. Consequently, the current experiments provide limited evidence for generalization to diverse and previously unseen editing instructions. These limitations reduce my confidence in the robustness and reliability of DeContext under open-world attack scenarios.

Questions for authors

Q1. On the causal relationship between attention connections and identity preservation. The authors provide relevant evidence through attention masking, gradient analysis, and block-wise ablation studies. However, to what extent do these experiments establish the direct causal contribution of specific attention connections to identity preservation? Could the authors conduct more fine-grained interventions on individual Transformer layers or attention heads, and quantitatively analyze the relationship between intervention strength, changes in attention mass, and the resulting identity similarity? Such evidence would help clarify whether attention mass serves as a reliable indicator of contextual information contribution and would strengthen the proposed mechanism explanation. The results could materially affect my assessment of the paper's interpretability contribution.

Q2. On cross-model transferability of adversarial perturbations. Could the authors clarify the experimental setup in Section 4.4? Specifically, does the Step1X-Edit evaluation directly use adversarial perturbations optimized for FLUX Kontext, or are the perturbations re-optimized specifically for Step1X-Edit? If re-optimization is required, can the same protected image remain effective when directly transferred to another image editing model without any additional optimization? I would particularly appreciate cross-model transfer results between FLUX Kontext and Step1X-Edit. Such experiments would distinguish the applicability of the proposed optimization method across architectures from the transferability of the resulting adversarial perturbations themselves. This distinction is important for assessing the practical value of the proposed defense.

Q3. On prompt generalization. The authors evaluate two prompts excluded from the optimization prompt pool in Table 1 and provide additional results under different editing instructions in the main paper and appendix. However, the main quantitative evaluations still cover a relatively limited set of prompts. Furthermore, some evaluation prompts in Table 2 overlap with the optimization prompt pool in Appendix F.3. For example, "Add glasses to this person." appears in both. Could the authors clarify the distinction between seen and unseen prompts in the current evaluation and provide results on a broader, independent set of test prompts? In particular, it would be valuable to separately evaluate unseen prompts that are semantically similar to the optimization instructions and prompts belonging to editing categories not covered by the optimization pool. Such results would help establish the extent to which DeContext generalizes to previously unseen editing scenarios.

Reviewer_hV7FOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DeContext, an image immunization method to protect images from in-context editing in DiTs. Small perturbations are added to the image vai PGD optimization. The objective is to distrupt the attention between the reference-input-image and the noisy tokens. Consequently, results show that immunized images produce edits that do not contain any context (like subject identity) of the original image itself, thereby providing protection. Results are shown with Flux1Kontext and Step1xEdit.

Critical strengths
  1. The idea is quite intuitive. The simple motivation analysis is convincing. The mechanistic analysis of Context image as query, target image as key vs Target image as query, context image as key is also insightful.
  2. Logical and structured approach of finding timestep to use for optimization
Critical weaknesses
  1. The method demonstrates results only for 2 models. Many new models like Flux2Klein and QwenImageEdit have been available since a long time now and should have been included
  2. There is some confusion regarding the blocks to target for optimization. Fig 4b and 4c show context proportsion for single blocks only, which appear after the double blocks. Double blocks also perform attention across the full sequence and should have been considered in this anlysis
  3. The method samples from a set of prompts during optimization. There should be a study of the edit-protection comparing in-distruibtion and out-of-distribution prompts during inference on immunized images.
  4. While not a major weakness, I find the objective formulation unnecesarily convoluted. Instead of defining L = 1-r_ctx and then doing gradient ascent, why not simply do gradient descent on L=r_ctx.
Questions for authors
  1. What is the effect of using of double-blocks during optimization (Weakness 2)?
  2. Can the authors please perform an analyis of edit protection for in-distribution and out-of-distribution prompts during inference on immmunized images? (Weakness 3)
  3. While I greatly appreciate sharing the code, I found that the protection on immunized images were not consistent. Protection appears to be prompt and seed sensitive. Please share a study over prompts and seeds and also report the percentage of times the protection was succesful. This will greatly help in improving transparency.
  4. There is another paper VETO[1], that is quite similar and proposes the use of both interaction reference->noise and noise->reference. This is slightly contradictory as the claim in DeContext is that only one of these interactions work effectively, and therefore using both interactions as the objective should cocpetually degrade performance.

References: [1]Grebe, Jonas, et al. "VETO: Towards Protecting Images From Frontier AI Editing." arXiv preprint arXiv:2607.27292 (2026).

Reviewer_yrhUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DeContext, a proactive defense against unauthorized image editing in diffusion transformers. By minimizing target-to-context attention at selected transformer blocks and denoising timesteps, the method aims to prevent identity information from propagating into generated images while preserving output quality.

Critical strengths
  1. The paper investigates how reference image information propagates through multimodal attention and uses controlled interventions to motivate its defense.

  2. Unlike prior image protection methods that mainly focus on traditional U-Net-based diffusion models, this paper targets modern DiT-based image editors.

  3. The paper includes ablations on timesteps and transformer blocks, preprocessing robustness evaluations, and a user study.

Critical weaknesses
  1. Although the paper demonstrates the limitations of Diff-PGD, it does not sufficiently justify why attention suppression is preferable to alternative objectives, such as feature-targeted optimization. Discussion and comparisons with methods like PhotoGuard under the same editing model would strengthen the motivation.

  2. DeContext requires white-box access to the target editing model and performs model-specific optimization, limiting its practical applicability. It remains unclear whether the protection is effective when malicious users employ different editing models.

  3. The paper overlooks relevant model-side protection approaches, such as GuardDoor (Zeng et al., 2025) and PersGuard (Liu et al., 2026). If cooperation with model providers is considered a practical deployment scenario, these approaches should be discussed.

  4. The paper aims to protect images from unauthorized editing but explicitly encourages realistic, prompt-consistent generation with altered identities. It is unclear why preserving adherence to potentially malicious editing instructions is desirable from the protected image owner's perspective, or whether identity detachment alone is sufficient to prevent harmful image manipulation.

  5. Although DeContext is evaluated on multiple editors and several preprocessing operations, cross-model perturbation transferability remains unclear. Moreover, the degradation under JPEG compression and TV minimization raises concerns about its robustness against stronger preprocessing and adaptive attacks.

Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.