- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper deals with concept erasure which aims to remove target concepts from diffusion models without harming their ability to generate unrelated concepts. Prior works used external information / other concepts in order to define what should be retained. This paper defines the surplus of the concept which is the portion of the concept’s activation that the span of its parts cannot explain. Contrary to previous works, this surplus is retrieved from the concept itself. Thus, erasing only this surplus prevents the model from assembling the concept while different concepts sharing its parts are still available for generation.
Critical strengths
- The positioning of the paper in the context of the related work and its idea is well motivated and clear.
- The proposed erasure method is training-free.
Critical weaknesses
- The results are not compelling. Table 1 is not entirely clear. What are data req.? gates? what is strong/mid/weak? Robustness? Qualitative comparison values would be appreciated. In Table 2, ExCISE looks only marginally better compared to CASteer. Showcasing part erasure only on a single domain quantitatively is not enough. A qualitative example of another domain (cowboy hat) could be cherry-picked. A further qualitative superiority demonstration could convince effectiveness. Figure 3: on y axis, CASteer is only ~0.016 behind in CLIP score, which is marginal. On Identity Erasure task, again, only a single domain is demonstrated. What about erasure of other concepts such as garbage truck that was mentioned previously? Animals?
Questions for authors
- Please see Weakness #1. How would ExCISE perform on other domains compared to the other approaches?
6 comments
Response to Reviewer 9ko7 (2/3)
(continued)
[W1-2, Q1] Differentiation from CASteer
Answer: We agree with the reviewer that, in some standard benchmark settings, the performance differences between ExCISE and CASteer are marginal. In particular, we acknowledge the reviewer's concern regarding the nudity erasure results in Table 2, where a difference of a single detected image does not provide sufficient evidence of a meaningful improvement.
However, we emphasize that ExCISE's contribution extends beyond strong performance on standard erasure benchmarks. Our method specifically addresses nearby-concept preservation by protecting visual components. Our extended evaluations demonstrate that this mechanism leads to meaningful improvements over CASteer in nearby-concept preservation and adversarial robustness, while consistently maintaining strong target erasure across tasks. We provide quantitative evidence of these differences below:
- Nudity Erasure and Robustness: While both methods achieve the strongest erasure on standard I2P [G14] prompts (Table 2, Lines 324-336), ExCISE shows greater erasure robustness under UnlearnDiffAtk (official default setup, 118 prompts), achieving an attack success rate of 10.2%, compared with 19.5% for CASteer and 39.0% for ReCARE [G4], showing that ExCISE leaks the erased nudity on roughly half as many attack prompts as CASteer (Section 5.1, Lines 352-372).
- Celebrity Erasure: Under pairwise celebrity prompts (Table 3, Lines 397-408), ExCISE substantially improves non-target identity preservation over CASteer (54.55% vs. 15.05% Acc_s) while maintaining near-zero target recognition (0.50% vs. 0.05%). This results in a higher harmonic mean (70.34 vs. 25.61), showcasing that in contrast to CASteer, ExCISE maintains a strong erasure-preservation trade-off even under more complex prompts. NLCE preserves more on pairwise prompts; combining our direction with NLCE’s gate gives the best pairwise result (Acc_s 67.20, H 80.30).
- Object Erasure: To further extend the comparison to object erasure, we evaluate retention on nearby concepts (Section 5.3, Lines 505-510). On SDXL [G11], ExCISE improves preservation across 11 neighboring concepts under garbage truck erasure, reducing FID from 134.0 to 65.8 and ΔCLIP (×100) from 5.54 to 1.18 compared with CASteer, while maintaining low target leakage (2.6% vs. 0.0%). The figure and corresponding quantitative table have been added to the main paper and Appendix, respectively (Figure 6; Tables E.1 and E.2).
[W1-3, Q1] Part Erasure: Clarification and extension.
Answer: To clarify, nudity erasure serves as our primary evaluation of part-level concept removal, where the goal is to suppress targeted visual components while preserving the remaining parent concept (the person and its benign characteristics). To further demonstrate the generality of this setting, we have extended our qualitative examples of cowboy hat erasure (Figure 4b, Lines 338-346) to address part erasure tasks beyond the common nudity evaluations. We additionally evaluated cowboy-hat erasure quantitatively with the same protocol as for objects (800 images per prompt set from 80 ImageNet templates × 10 seeds). Since "cowboy hat" is an ImageNet class, it is detected with the same classifier as our object leakage. Top-1 detection on cowboy hat drops from 73.4% to 2.3% on prompts naming the hat and from 66.0% to 1.3% on prompts naming only the parent ("cowboy"), while COCO [G8] preservation is maintained (caption CLIP unchanged at 0.78; FID 11.52). Since the compared methods do not report part erasure, we evaluate it against the unedited model.
[W1-4, Q1] Extended Qualitative Evaluation
Answer: We have extended the qualitative results to include additional generated examples across different concepts, prompts, and random seeds (Figures 4(b,c) and 5; Appendix Figures C.1–C.3).
These additions include per-method comparisons and aim to provide a broader view of the method's behavior across tasks, including variations in target erasure and nearby-concept preservation, while the quantitative evaluations provide a more systematic comparison.
[W1-5, Q1] Figure 3.
Answer: In Figure 3 (now Figure 7(a), Lines 486-495), CASteer was included as a reference point on a limited prompt subset rather than for comprehensive comparison. Measured from the unedited model, CASteer loses three times more bystander CLIP than ExCISE (0.023 vs. 0.007) at similar target suppression. Our extended evaluations provide stronger evidence of ExCISE's advantages in nearby-concept preservation and adversarial robustness.
Response to Reviewer 9ko7 (3/3)
(continued)
[W1-6, Q1] Celebrity Erasure: Per-Concept Results
Answer: We clarify that the celebrity results presented in the main paper (Table 3, Lines 397-408) are averaged across four celebrities: Taylor Swift, Andrew Garfield, Elon Musk, and Emma Watson. The preservation accuracy (Acc_s) measures how often the non-target celebrities of the full MACE [G7] retain set (100 celebrities) are still identified.
To improve transparency, we have further added complete per-celebrity comparisons for all baseline methods in Appendix E.1 (Table E.3) and specified the evaluated celebrities in the experimental section (Section 5.2, Lines 422-423) to facilitate comparisons.
[W1-7, Q1] Extended object erasure evaluation
Answer: To address concerns about the limited scope of object erasure experiments and follow the request to provide results on garbage truck and an animal example (English springer), we have extended our SDXL evaluation with additional quantitative and qualitative results on object erasure in Section 5.3 of the revised manuscript (Lines 477–510).
The extended evaluations provide a broader comparison across methods and concept categories with different characteristics across three new objects: garbage truck, golf ball, and English springer.
| Method | GT T1↓ | GT COCO CLIP↑ | GT COCO FID↓ | Vehicle FID↓ | GB T1↓ | GB COCO CLIP↑ | GB COCO FID↓ | Ball FID↓ | ES T1↓ | ES COCO CLIP↑ | ES COCO FID↓ | Dog FID↓ |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Orig | 86.9 | 0.78 | – | – | 78.9 | 0.78 | – | – | 91.5 | 0.78 | – | – |
| UCE | 59.6 | 0.78 | 15.81 | 41.3 | 56.0 | 0.75 | 26.62 | 79.0 | 7.4 | 0.77 | 18.39 | 69.7 |
| AdaVD | 27.0 | 0.78 | 0.00 | 0.0 | 71.1 | 0.78 | 0.00 | 0.0 | 1.0 | 0.78 | 0.00 | 0.0 |
| NLCE | 18.9 | 0.78 | 9.92 | 64.1 | 17.2 | 0.78 | 9.52 | 78.2 | 5.9 | 0.79 | 8.59 | 54.0 |
| ESD | 6.8 | 0.77 | 18.26 | 117.2 | 10.5 | 0.78 | 13.95 | 156.9 | 2.9 | 0.77 | 19.10 | 108.7 |
| ReCARE | 6.5 | 0.75 | 27.63 | 130.3 | 11.1 | 0.76 | 17.64 | 108.9 | 0.0 | 0.76 | 19.69 | 107.6 |
| CASteer | 0.0 | 0.77 | 11.80 | 79.8 | 0.0 | 0.78 | 7.87 | 129.9 | 0.0 | 0.78 | 8.29 | 71.4 |
| Ours | 2.6 | 0.79 | 10.53 | 55.5 | 1.0 | 0.79 | 9.61 | 79.5 | 0.1 | 0.79 | 9.84 | 77.2 |
GT: garbage truck, GB: golf ball, ES: English springer; T1: ImageNet top-1 leakage (%); superclass FID on Vehicle, Ball and Dog prompts. Best per column in bold; AdaVD leaves non-target prompts unchanged (FID 0.0). ESD is ESD-x-strict.
On SDXL, across all three object categories, ExCISE demonstrates a consistently strong erasure–preservation trade-off, without ranking among the two worst-performing methods on any metric. It achieves near-zero target leakage (0.1–2.6%) while maintaining the highest COCO CLIP preservation (0.79, tied on English springer) across all three concepts and competitive FID scores. These results highlight the robustness of ExCISE's erasure–preservation performance across visually distinct object categories. The full table is inserted into the manuscript as Table 4 (Lines 432-449). A garbage truck erasure evaluation on FLUX.1 dev and full comparisons on SD v1.4 can be found in the revised appendix (C.2 and E.2).
As discussed in point [W1-2, Q1] above, we further extended the garbage truck evaluation to include additional visual and semantic neighbors. Complete quantitative results and comparisons across object categories are provided in the newly added object erasure subsection (Section 5.3).
Overall, our extended evaluations demonstrate ExCISE's key strength: a consistently strong erasure–preservation trade-off across diverse concept categories without training or manually constructed retain sets. Its advantages are particularly evident in nearby-concept preservation and adversarial robustness, while its applicability extends beyond U-Net architectures to FLUX.1 dev [G13].
We hope that these additional results and clarifications adequately address the reviewer's concerns and kindly invite the reviewer to reconsider their assessment in light of the provided evidence. We are happy to provide further details if needed.
Response to Reviewer 9ko7 (1/3)
We thank the reviewer 9ko7 for the careful review and for recognizing the training-free nature of our method, the clarity of our motivation and its positioning within the existing literature. Below, we address each point with new quantitative results.
Answer:We thank the reviewer for highlighting the need for additional experimental results and clearer comparisons with existing baseline methods. Accordingly, we extend our quantitative and qualitative evaluations across multiple concept categories and provide further clarification for several aspects of our experimental evaluation. We address each of the concerns raised under W1 in detail below.Answer:We have clarified the table caption to highlight its purpose: summarizing the erasure–preservation trade-off across tasks, data requirements, and inference-time complexity. We removed the unclear erasure-performance column (strong/mid/weak); quantitative comparisons are reported in the experimental section (Section 5).Data Requirements and Gates: Data requirements refer to manually selected or externally retrieved information that is needed to improve the erasure-retain trade-off through a constructed retain space. CASteer [G2] requires prompt pairs, NLCE [G3] an external retain set, and ExCISE only the target concept to improve utility preservation. Gates indicate the number of inference-time gating mechanisms a method uses for the same purpose of refining erasure-retain trade-off. Unlike CASteer, ExCISE applies cross-attention output steering without spatial clipping, using a uniform per-layer direction and strength across all spatial positions. ExCISE's reduced dependence on external data matters, as manually selected data (e.g., prompt pairs) reduce efficiency while retrieved sets are heavily dependent on retrieval success and concept neighborhoods as seen in the varying performance of NLCE [G3] on SDXL. Under the same method settings, NLCE showcases near-zero concept leakage on celebrity erasure (Table 3, Lines 397-408), while ranging from 5.9% on English springer to 18.9% leakage on garbage truck. Inference-time gates often improve preservation, but add computational overhead.
Robustness: Robustness is quantitatively evaluated in the experimental section (Sections 5.1 and 5.2) for nudity and celebrity erasure, with additional paraphrasing results provided in Appendix E.3. In Table 1, a checkmark is assigned to methods that do not rank among the two worst-performing methods on any metric within the respective task. ExCISE is the only method satisfying this criterion across all three robustness settings: nudity erasure under Ring-A-Bell [G6] and UnlearnDiffAtk [G5], celebrity erasure with pairwise prompts, and garbage truck erasure under paraphrased prompts. Robustness matters because an erased concept should not be recoverable through adversarial or paraphrased prompts. Under 58 paraphrases of garbage truck (580 images; Appendix E.3, Figure E.1), ExCISE leaks 9.1% (unedited: 88.1%), compared with 38.1% for NLCE and 70.5% for UCE; CASteer (1.7%) and ReCARE (7.1%) leak less but degrade the neighboring concepts far more (neighbor FID 134.0 and 142.1 vs. 65.8).