acceptodds
Under review as a conference paper at ICLR 2027

Concept Erasure for Text-to-Image Diffusion Models Against Image Probes

Abstract

Concept erasure aims to prevent text-to-image (T2I) diffusion models from generating unwanted concepts. Although T2I models can naturally accept images as input, existing erasure methods have limited the probing of erased models to only the text modality, e.g., “Generate a picture of a naked woman”. This makes existing erasure methods insufficient against image probes. In this paper, we address this research gap by proposing Image Probe Erase (IPErase), the first concept erasure method against image probes. IPErase adopts a frozen teacher (U-Net or DiT) model to construct denoising trajectories that extrapolate beyond the neutral prediction, away from the concept. It then trains a student model to match these trajectories using concept images as conditioning inputs through a frozen image prompt adapter (IP-Adapter), selectively updating the shared denoising backbone. On two widely used target concepts (nudity and Van Gogh style), our IPErase outperforms existing (text-based) erasure methods by (relative) 95.7%, 50.0%, and 91.8% against three increasingly challenging forms of image probes: black-box input condition, gray-box latent initialization, and white-box adversarial optimization. Further analyses show that despite relying solely on image supervision, our IPErase achieves perfect performance comparable to text-based methods against diverse text probes. This strong contrast in erasure generalizability between text-based methods to image probes and image-based methods to text probes suggests that images, which contain richer semantics, position target concepts much better than texts.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Text-to-Image Diffusion Model, Concept Erasure, Image Probes

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_2JA8OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper points out that prior concept erasure methods are still vulnerable to image probes and proposes Image Probe Erase (IPErase), a two-stage concept erasure method relying solely on image conditions. In the first stage, IPErase constructs concept-contained and concept-removed image pairs using external pre-trained models, such as image inpainting and segmentation models. Then, for each pair, it samples along the DDIM trajectory using a pre-trained diffusion model and applies negative CFG to construct an erasure target that moves away from the concept-contained image toward the concept-removed image. In the second stage, the student is conditioned on the concept-contained image features and trained to predict the erasure target. Experiments on nudity and Van Gogh-style erasure demonstrate that IPErase is effective against image probes and performs well against adversarial text probes.

Critical strengths
  • The paper shows that image-conditioned difference exhibits clearer target concept structures than the text-conditioned difference, motivating the proposed IPErase method.
  • IPErase is effective against image probes and achieves performance comparable to adversarially trained SOTA methods against adversarial text probes.
Critical weaknesses
  • Experiments are conducted on Van Gogh style and nudity only, which makes it difficult to evaluate IPErase’s generalizability. Prior works evaluate more diverse concepts. For example, AdvUnlearn covers artistic styles, nudity, and 4 objects. UCE evaluates object erasure on 10 objects.
  • The baselines compared in the experiments are outdated. Co-Erasing [ICML’25] is the only relatively recent baseline, while newer methods, such as SPEED [A1] and CASteer [A2], are not compared.
  • Since the model still accepts text prompts after erasure, quantitative evaluations on normal and paraphrased text prompts are necessary. For example, can the model still generate nudity from prompts such as "a man not wearing any clothes," "a naked woman," or "a lady that has no clothes on" after erasure?
  • The evaluation in Table 3 is unclear.
    • The data used to evaluate non-target preservation is unspecified. For example, UCE evaluates preservation on the remaining 9 objects when erasing 1 of 10 objects. MACE [A3] and Receler [A4] use MS-COCO 30K prompts to evaluate the non-target preservation for nudity erasure.
    • Preservation results should be reported for all 5 configurations rather than simply indicating that they pass a threshold.
    • In addition, comparisons with other erasure methods are missing, making it difficult to assess IPErase's non-target preservation performance.
  • The sensitivity analyses of the number of image pairs and DDIM steps per pair on erasure performance are not shown.

[A1] SPEED: Scalable, Precise, and Efficient Concept Erasure for Diffusion Models. Ouxiang Li et al. ICLR 2026.
[A2] CASteer: Cross-Attention Steering for Controllable Concept Erasure. Tatiana Gaintseva et al. ICLR 2026.
[A3] MACE: Mass Concept Erasure in Diffusion Models. Shilin Lu et al. CVPR 2024.
[A4] Receler: Reliable Concept Erasing of Text-to-Image Diffusion Models via Lightweight Erasers. Chi-Pin Huang et al. ECCV 2024.

Questions for authors
  • How long does it take to generate 50 image pairs and perform 50 DDIM steps per pair for each target concept? Does image pair construction require human judgment, like filtering out unnatural images generated by the inpainting model?
  • In Section 3.1, the paper shows that image-conditioned differences are more apparent than text-conditioned differences for nudity. Does this observation also hold for Van Gogh style, which is a more abstract concept?
  • In line 215, the paper says that ϵcfg,n\epsilon_\text{cfg,n} is used in the teacher model's updates. However, since ϵcfg,n\epsilon_\text{cfg,n} moves the generation toward the target concept, why should the teacher generate target concept images?
  • How does erasure performance change when the number of image pairs is reduced to 25 or 10, or when the number of DDIM steps per pair is reduced to 20?
Reviewer_GVr1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes IPErase, a concept erasure method for text-to-image diffusion models against image probes. It uses concept images and neutral images to construct erasure targets through a frozen teacher model. Then, the student model learns these targets by updating the shared denoising backbone. Experiments on nudity and Van Gogh style show that the method outperforms multiple baseline methods under IP-Adapter, img2img, and adversarial image probes.

Critical strengths
  1. The paper studies concept recovery through image probes, which is an important problem for concept erasure. The results show that existing methods may still generate erased concepts when images are used as inputs.
  2. The learned erasure can work across different image input interfaces, showing its generalization ability.
Critical weaknesses
  1. The paper claims that image supervision provides better erasure than text supervision. However, the compared methods use different training objectives and trainable parameters. Therefore, it is unclear whether the improvement comes from image supervision or the parameter update strategy.
  2. The main experiments only cover nudity and Van Gogh style. Meanwhile, Table 9 shows that the erasure performance decreases when img2img strength is reduced to 0.50. Therefore, the performance of the method is still limited.
  3. Table 4 shows that using shuffled neutral images achieves similar or even better results than matched image pairs. However, the proposed method uses the semantic correspondence between concept and neutral images to construct erasure targets. Therefore, the actual benefit of matched image pairs is unclear.
Questions for authors

Please see the weaknesses above. I am willing to reconsider my rating if these concerns are properly addressed.

Reviewer_nCrkOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that concept erasure for T2I diffusion models is tested almost only against text probes, and the investigation instead focus on image probes. Further evaluations demonstrate the robustness of the proposed erasure method against both image and text probes.

Critical strengths
  1. The experiments demonstrate significant performance improvements on the baselines tested
  2. The paper proposes a new, image-probe-based concept erasure method instead of the typical text probes more commonly used by the community
Critical weaknesses
  1. The novelty is limited. Existing works have already attempted to remove the concepts without explicit text probes, e.g. SafeGen[1], which is not cited, and ACE [2]. “first method designed to resist image probes” may thus be an overclaim. The proposed method also employed commonly used objective in the literature, and the IP-adaptor is not new either [3]
  2. The experiment scope is limited and narrower than the typical literature: The paper only includes numerical results for Nudity and Van Gogh style. Object related results are only included as example images instead of rigorous comparisons. The experiments also miss baselines such as SafeGen [1] and ACE [2]

[1] Li, Xinfeng, et al. "Safegen: Mitigating sexually explicit content generation in text-to-image models." Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security. 2024.

[2] Wang, Zihao, et al. "Ace: Anti-editing concept erasure in text-to-image models." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

[3] Ye, Hu, et al. "Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models." arXiv preprint arXiv:2308.06721, 2023

Questions for authors

Where do the baseline checkpoints used in the tables come from? Were they taken from the official release, or trained by the authors themselves? If it is the latter, what methods or experiments have the authors done to ensure the training is successful?

Comments on the paper

1 comment

Dear Reviewers,

The authors have posted their rebuttal. Please review their responses and the comments from your fellow reviewers, and provide any further feedback. Thank you.

The AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.