acceptodds
Withdrawn from ICLR 2027

VisualErase: Dual-Branch Visual Trajectory Redirection for Robust Concept Erasure in Text-to-Image Diffusion Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Concept erasure is essential for the safe deployment of text-to-image diffusion models, as they may reproduce harmful, copyrighted, or privacy-sensitive content learned from unconstrained large-scale data. Existing methods typically erase unwanted concepts while preserving general generation capability by redirecting target-related text-to-image mappings. However, recent studies show that erased models may still retain visual generative trajectories of target concepts, leaving them vulnerable to adversarial recovery attacks and revealing a fundamental gap between redirecting text-to-image mappings and truly removing visual knowledge. To bridge this gap, we propose VisualErase, a new paradigm that redirects concept-bearing visual generative trajectories toward explicitly defined concept-removed outcomes. To enable this redirection, we use structure-preserving image editing to construct content-aligned, concept-removed counterparts for source images, providing explicit visual endpoints that retain non-target content. We then derive a denoising target from each source-to-counterpart pair and use a dual-branch redirection loss to align both text-conditioned and unconditional predictions with this target, since conditional supervision alone does not explicitly constrain generation without textual guidance. To mitigate the adverse effects of concept erasure on non-target generation, we jointly optimize the redirection loss with a counterpart retention loss that matches denoising predictions from the frozen pretrained model. Across style, celebrity, and nudity erasure, VisualErase limits the maximum attack success rate over seven attacks to 0%, 8%, and 0.1%, respectively, while retaining general generation quality. These results highlight the importance of visual trajectory redirection for robust concept erasure beyond text-to-image mappings alone.

Primary area:
generative models
Keywords:
Text-to-Image Diffusion Models, Concept Erasure, Machine Unlearning, Adversarial Robustness, AI Safety

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_KcY4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VisualErase, a concept erasure method that redirects the generation trajectory of a target concept toward content-aligned but concept-removed images. First, for each target concept image, it uses image editing models to construct a concept-removed but structure-preserved image, forming an image pair. Then, during concept erasure training, VisualErase derives the redirection target from the image pair. The model learns to predict the redirection target under both conditional and unconditional generation. In addition, VisualErase uses the concept-removed images to preserve the non-target generation ability. Experiments are conducted on style, celebrity, and nudity domains.

Critical strengths
  • Constructing structure-preserved image pairs for concept erasure is novel, although the construction process seems complicated and may be time-consuming.
  • The paper is clear and easy to follow. The process of constructing image pairs for each erasure domain is explained clearly and in detail.
Critical weaknesses
  • The most important part of the method is the paired image construction. However, the paper does not provide any time analysis for this step and only reports the training time.
  • The paper uses image editing models to construct structure-preserved, concept-removed images for concept erasure. This is essentially leveraging additional information from other pre-trained models and constructing hard positive examples for the model to learn from. Therefore, it is not surprising that VisualErase achieves better concept erasure performance when it uses more information than prior methods.
  • The compared concept erasure baselines and cited related works are outdated. More recent and relevant papers, such as SPEED [A1], Co-Erasing [A2], and TICoE [A3], should be cited and included in the comparisons, especially Co-Erasing and TICoE, as both use text and image conditions for concept erasure.
  • All concept erasure experiments are conducted on Stable Diffusion 1.4. It remains unclear whether the method is effective for larger text-to-image diffusion models, such as Stable Diffusion XL, or even flow-matching models, such as FLUX.1.

[A1] SPEED: Scalable, Precise, and Efficient Concept Erasure for Diffusion Models. Ouxiang Li et al. ICLR 2026.
[A2] One Image is Worth a Thousand Words: A Usability Preservable Text-Image Collaborative Erasing Framework. Feiran Li et al. ICML 2025.
[A3] Beyond Text Prompts: Precise Concept Erasure through Text-Image Collaboration. Jun Li et al. CVPR 2026.

Questions for authors
  • How much time does it take to construct the image pairs?
  • Are humans involved in the paired image construction process, such as checking the quality of the final image pairs and filtering out invalid ones?
  • Is VisualErase effective on Stable Diffusion XL?
Reviewer_REtgOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VisualErase, a concept-erasure method that redirects visual denoising trajectories toward content-aligned, concept-removed counterparts. It derives denoising targets from edited image pairs, supervises both conditional and unconditional predictions, and uses a retention loss to preserve non-target generation. Experiments on SD1.4 evaluate Van Gogh style, Taylor Swift identity, and nudity erasure against seven attacks, including text-free TINA+, with the authors reporting strong resistance to concept recovery.

Critical strengths
  1. VisualErase demonstrates strong resistance across seven attacks without adversarial prompt mining and substantial improvements against TINA+ over existing baselines.

  2. The paired-image formulation provides a concrete post-erasure target, while the analytically derived denoising objective offers a clear mechanism for redirecting concept-bearing visual states.

Critical weaknesses
  1. Counterpart construction requires concept-specific editing and selection. The reported 14.2-minute training time excludes this preparation, leaving the end-to-end efficiency advantage unclear. The different supervision budgets—1,000 pairs from 100 Van Gogh sources, 50 identity pairs, and 109 nudity pairs—also lack a data-efficiency analysis.

  2. All experiments use SD1.4. Applicability to DiT-based models such as PixArt or FLUX remains unverified.

Questions for authors
  1. Can you report end-to-end costs, including counterpart generation, filtering, and any manual selection, under a comparable timing protocol to the baselines?

  2. What is the rationale for the chosen number of source images and counterparts per source in each task? Is there experimental evidence that the same data budgets would suffice for erasing other concepts within the same domain or across different domains?

  3. Can the method be validated on a DiT backbone, and how would the target derivation and dual-branch supervision be adapted to a flow-matching model such as FLUX or PixArt?

  4. Several outputs of VisualErase in Figure 3 appear blurred or oversmoothed. The style-erasure, nudity-erasure FID also trail most listed baselines, leaving the robustness–quality trade-off insufficiently characterized. Can you report CPBD(A No-Reference Image Blur Metric Based on the Cumulative Probability of Blur Detection), NIQE(Making a “Completely Blind” Image Quality Analyzer), and MUSIQ(MUSIQ: Multi-Scale Image Quality Transformer) metric separately for target-conditioned and attack-generated outputs?

  5. Quantitative experiments cover only one artist, one celebrity, and nudity. Results across additional concepts and joint or sequential erasure are needed to establish generality and scalability. Can VisualErase erase multiple styles or identities, or concepts across categories, jointly or sequentially without interference, quality degradation, or recovery of previously erased concepts?

Reviewer_NQjkOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes VisualErase, which derives denoising targets from concept-removed image counterparts to supervise both conditional and unconditional predictions. A counterpart retention loss is introduced to preserve general generation quality. Experiments on SD1.4 demonstrate strong adversarial robustness across style, celebrity, and nudity erasure tasks.

Critical strengths
  1. The paper addresses an important problem of preventing visual concept recovery beyond text-based suppression.

  2. The proposed method demonstrates impressive robustness against seven attacks, including TINA+, while largely preserving generation quality in terms of FID and CLIP scores. The ablation studies also support the effectiveness of the key components.

Critical weaknesses
  1. Limited methodological novelty.

The paper presents trajectory redirection toward explicitly defined visual outcomes as a new paradigm. However, DUO (NeurIPS 2024) has already explored visual unlearning using concept-removed image pairs, ACE (CVPR 2025) incorporates erasure supervision into both conditional and unconditional predictions, and SDErasure (ICLR 2026) investigates concept-specific trajectory shifting. Although the proposed counterpart-derived targets differ from existing objectives, the methodological distinctions and novel technical contributions relative to these works are not sufficiently established. In particular, the characterization of prior methods in Section 3 does not adequately account for these related approaches.

  1. Insufficient evaluation of non-target concept preservation.

While the paper reports strong erasure performance and maintains general generation quality on COCO-30K, it does not evaluate whether semantically related non-target concepts are preserved. For instance, when erasing Van Gogh's style, it remains unclear whether other artistic styles, such as Monet or Renoir, can still be generated faithfully. Since the proposed method directly modifies visual denoising predictions, evaluating potential collateral effects on related concepts is important to establish the selectivity of the erasure.

  1. Limited generalization and practical applicability.

All main experiments are conducted on SD1.4, leaving the applicability of the proposed method to modern DiT and flow-matching-based models unverified. Moreover, the quantitative evaluation focuses on a single artistic style, a single celebrity identity, and nudity, providing limited evidence of generalization across target concepts. In particular, the absence of multi-concept erasure experiments leaves it unclear whether adversarial robustness and non-target generation quality can be maintained when multiple concepts are erased simultaneously.

Questions for authors
  1. What is the key technical contribution of the proposed counterpart-derived dual-branch supervision compared to DUO, ACE, and SDErasure? Beyond differences in the training objectives, what substantive advantages does the proposed formulation offer? Could the authors also provide quantitative comparisons with these methods?

  2. When erasing a target artistic style or celebrity identity, does the proposed method preserve closely related non-target styles or identities? Could the authors provide a targeted preservation evaluation beyond the general COCO FID/CLIP metrics?

  3. Does the proposed method maintain concept-specific robustness and overall generation utility when multiple concepts are erased simultaneously or sequentially? Furthermore, can the authors evaluate its applicability to modern DiT- or flow-matching-based generative models and provide comparative results?

[1] Park et al., "Direct Unlearning Optimization for Robust and Safe Text-to-Image Models." NeurIPS, 2024.

[2] Wang et al., "ACE: Anti-Editing Concept Erasure in Text-to-Image Models." CVPR, 2025.

[3] Miao et al., "SDErasure: Concept-Specific Trajectory Shifting for Concept Erasure via Adaptive Diffusion Classifier." ICLR, 2026.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.