- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CRET, a representation-level concept erasure method for text-to-image diffusion models. The method freezes the parameters of the pretrained text encoder and diffusion model, first mapping text representations into a low-dimensional latent space through an encoder–decoder architecture, and then learning a rank-constrained orthogonal concept subspace. This subspace captures the target concept by exploiting the differences between risky and safe prompts, while suppressing the projections of safe and preservation representations onto the editable subspace. During inference, CRET removes target-related components and transfers the corresponding residual changes back to the original text representations.
Experiments are conducted primarily on Stable Diffusion models, covering standard prompts, multiple adversarial attack benchmarks, and general image generation on COCO. The results show that CRET suppresses target concepts more effectively than several existing concept erasure methods while maintaining competitive generation quality on benign prompts. Experiments on SDXL further demonstrate its potential for cross-model applicability. Ablation studies on projection construction, loss components, projection rank, intervention scope, and representation mapping provide additional evidence for the effectiveness of the learned target subspace, the preservation of benign information, and the post-decoding preservation constraint. The paper also presents qualitative results involving violence, ordinary objects, and artistic styles to illustrate the method's potential applicability across diverse concepts.
Critical strengths
-
CRET obtains the lowest mean adversarial ASR among the listed methods and substantially outperforms competing model-preserving methods on Ring-A-Bell, MMA-Diffusion, and UnlearnDiffAtk. This is consequential because robustness to prompt attacks is central to the paper’s stated goal.
-
The target-inclusion, benign-exclusion, and decoded-preservation objectives directly encode the desired safety–utility tradeoff. The ablations support their functional roles, especially the large degradation in erasure after removing the target-inclusion term and the large FID degradation after removing benign exclusion. The rank-matched comparison against direct SVD, QR-span, and preserve-constrained SVD isolates the learned projector as the main source of improvement. This strengthens the methodological claim.
Critical weaknesses
-
The paper’s quantitative validation is largely limited to nudity and violence. Object erasure and style erasure are demonstrated only through qualitative examples, despite the paper claiming applicability beyond unsafe concepts. Without quantitative target-erasure and preservation metrics for these settings, it is difficult to determine whether CRET generalizes beyond the primary safety benchmarks or whether the visual examples are representative.
-
The scalability of CRET to multi-concept erasure is not evaluated. The experiments appear to learn a concept-specific projector for one target at a time, leaving unanswered whether a single projector can capture several target concepts or whether multiple learned projectors can be composed without causing interference and excessive degradation of benign generation.
Questions for authors
-
Can the authors provide quantitative results for objects and styles erasure, including target-recognition success, preservation of unrelated objects and generation quality?
-
Is CRET trained separately for each target concept in all experiments? Have the authors tested multi-concept joint training or composition of multiple target projectors, and how does this affect attack success, preservation quality, and computational cost?
5 comments
Response to Reviewer pnkM(2/2)
[W2 , Q2]: Multi-Concept Erasure
Thank you for this suggestion. In all original experiments, CRET was trained separately for each target concept. To evaluate multi-concept erasure, we added two compositions of independently trained target projectors, applied in the orders nudity then violence and violence then nudity, as well as joint training on both concepts. Compared with OCE, DP, and ScaPre, all three CRET configurations achieve lower P4D, UDA, and Q16 ASRs and match OCE on RAB ASR, but have slightly lower CLIP scores and longer erasure preparation times. Among the CRET configurations, sequential composition gives lower P4D and Q16 ASRs, whereas joint training provides better FID and CLIP scores and reduces inference time from 1.2 to 0.8. These results illustrate the trade-off between suppression effectiveness, generation quality, and inference efficiency (Tables R4–R6).
Q16 ASR is the proportion of images generated on the I2P violence subset that Q16 classifies as inappropriate; lower is better. Across the three CRET configurations, Q16 ASR ranges from 5.29% to 9.25%, below OCE's 13.1%, the lowest among the compared baselines (Table R5).
Table R4. Multi-concept erasure and generation quality (Table 4 of the revised PDF).
| Method | RAB ASR ↓ | P4D ASR ↓ | UDA ASR ↓ | FID ↓ | CLIP ↑ |
|---|---|---|---|---|---|
| OCE | 0.01 | 0.139 | 0.042 | 46.83 | 31.0 |
| DP | 0.241 | 0.453 | 0.127 | 37.67 | 31.2 |
| ScaPre | 0.165 | 0.358 | 0.078 | 43.18 | 30.5 |
| CRET: Sequential composition (N → V) | 0.01 | 0.093 | 0.014 | 48.56 | 29.5 |
| CRET: Sequential composition (V → N) | 0.01 | 0.066 | 0.007 | 50.10 | 29.0 |
| CRET: Joint training (N + V) | 0.01 | 0.119 | 0.014 | 45.78 | 29.8 |
Table R5. Q16 ASR for multi-concept erasure (Table 4 of the revised PDF).
| Method | Q16 ASR (%) ↓ |
|---|---|
| OCE | 13.1 |
| DP | 16.0 |
| ScaPre | 20.5 |
| CRET: Sequential composition (N → V) | 7.93 |
| CRET: Sequential composition (V → N) | 5.29 |
| CRET: Joint training (N + V) | 9.25 |
Table R6. Multi-concept erasure computation time (Table 14, App. B.7 of the revised PDF).
| Method | Edit time ↓ | Infer time ↓ |
|---|---|---|
| OCE | 2.48 | 1.1 |
| DP | 4.00 | 1.1 |
| ScaPre | 23.14 | 0.8 |
| CRET: Sequential composition (N → V) | 57 | 1.2 |
| CRET: Sequential composition (V → N) | 57 | 1.2 |
| CRET: Joint training (N + V) | 58 | 0.8 |
N denotes nudity and V denotes violence.
Response to the Rebuttal
Thank you for your detailed responses and the effort devoted to the additional experiments and clarifications. I will also carefully consider the other reviewers’ comments to identify any aspects I may have overlooked and support a balanced assessment. Thank you again for your time and effort.
Response to Reviewer pnkM(1/2)
We appreciate the reviewer's thoughtful comments and helpful suggestions. Your concerns about generalization beyond unsafe concepts and multi-concept erasure motivated additional quantitative evaluations. We present these results below and have incorporated them into the revised manuscript to better clarify the scope and scalability of CRET.
[W1 , Q1]: quantitative results for objects and styles erasure
Thank you for raising the question of CRET's applicability beyond unsafe concepts. To provide quantitative evidence in this setting, we evaluate object erasure separately on ten categories from an ImageNet subset, treating one category as the target and using the other nine to assess preservation in each experiment. Specifically, we use a pretrained ResNet-50 for top-1 classification and report the target-category recognition rate on generated images and the average decrease in non-target recognition rate relative to the original model. Lower values indicate more effective target suppression and better non-target preservation, respectively.
CRET reduces the average target-category recognition rate from 85.6% to 1.5%, close to OCE (1.8%) and ScaPre (2.4%), while the average non-target recognition rate decreases by only 4.5 percentage points (Tables R1 and R2). On the ten evaluated object categories, these results show that CRET effectively suppresses the target concept while retaining most of the recognition performance on non-target categories, providing quantitative support for its applicability beyond unsafe concepts.
Table R1. Target recognition after object erasure (left half of Table 3 in the revised PDF).
Table R2. Non-target preservation after object erasure (right half of Table 3 in the revised PDF).
For style erasure, we separately erase the styles of Van Gogh and Picasso. We evaluate style suppression using CLIP Score (CS), the average CLIP matching score between generated images and their corresponding target-style prompts. A lower CS indicates more effective style suppression; it is not a classification accuracy. To assess preservation, we report FID, which measures how closely the generated image distribution matches that of the original model, and CLIP, which measures how well generated images retain the semantics specified by the content prompts.
As shown in Table R3, CRET achieves a Van Gogh CS of 23.26, close to OCE (23.89) and Receler (22.09), and the lowest Picasso CS of 23.86 among the compared methods. Compared with OCE, CRET also obtains lower FID for both styles, although its CLIP scores are slightly lower. These comparisons indicate competitive style suppression, with a trade-off between distributional preservation and content alignment.
Table R3. Style erasure and preservation (Table 13, App. B.6 of the revised PDF).