acceptodds
Withdrawn from ICLR 2027

Pristine Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Training on broad, unfiltered data enables high-quality image generation, but can also teach models concepts that should not be generated; restricting the training data ensures safety, but reduces quality because examples containing unsafe concepts also provide useful knowledge that is not itself unsafe. Existing approaches largely address this trade-off post hoc through concept unlearning, which can lead to catastrophic forgetting as the number of concepts to remove grows. We instead ask whether unsafe concept knowledge can be compartmentalized during pretraining so that access to it can be disabled at inference. We introduce Pristine Models, a novel text-to-image generation architecture with an attention mechanism that encourages the separation of safe and unsafe knowledge into independently parameterized branches. The branch-specific parameters are learned during training, while the unsafe branch can be detached at inference. On ImageNet, Pristine Models enable safer generation while retaining much of the utility of full-data pretraining, achieving higher image quality and better text alignment than training only on safe data or applying post-hoc concept removal. We further scale our approach to CC12M, demonstrating that it remains effective in a substantially larger and more diverse open-world pretraining setting.

Primary area:
generative models
Keywords:
diffusion, safety, unlearning, pretraining

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3QcQOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Pristine Models, a text-to-image generation architecture for improving safe image generation with high image quality during model pretraining. Authors utilize MLLMs to construct training prompts from image datasets by using the predefined “safe” and “unsafe concepts to train Safe and Unsafe Models via the proposed Pristine Attention.

Critical strengths

None

Critical weaknesses

The novelty of the proposed method is very limited only replacing the attention layers with the Pristine Cross Attention. There are only few explanation of the structure of Pristine Cross Attention without empirical or theoretical justification.

Authors’ goal is to training a “safe” text-to-image model by pretraining via defined “safe” data pairs. However, authors didn’t not define the concept between “safe” and “unsafe” according to actual semantic of text prompt or image content. This leads to whole experiments fail to support the central claim of this paper. Moreover, authors didn’t show how the proposed model works under well-know “unsafe” prompt benchmark like I2P [1] and adversarial benchmarks such as Ring-A-Bell [2] and P4D [3].

[1] Schramowski, Patrick, et al. "Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.

[2] Tsai, Yu-Lin, et al. "Ring-a-bell! how reliable are concept removal methods for diffusion models?." International Conference on Learning Representations. Vol. 2024. 2024.

[3] Chin, Zhi-Yi, et al. "Prompting4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts." arXiv preprint arXiv:2309.06135 (2023).

Questions for authors

In prompt decomposition, what is the method used to extract “safe” part from a unsafe prompt? What is the rule to decide “safe” and “unsafe” concept from the prompt? Does author consider there are overlap between “safe” and “unsafe” in different contexts?

2 comments
AuthorsOpenReview ↗

Author Rebuttal

Novelty

We respectfully disagree with the reviewer's assessment of the novelty of our approach and would like to clarify our contributions. Pristine Models introduce a mechanism for isolating unsafe knowledge into a detachable module during pretraining in the context of text-to-image generation, rather than attempting to remove such knowledge after training. Importantly, Pristine Attention also enables selective control over how knowledge learned from unsafe images is utilized, allowing useful attributes to be retained without generating the unsafe concepts themselves. This is demonstrated in Section 3.3, where attributes such as bronze and algae, learned from unsafe images, can be transferred to safe categories to generate novel compositions such as a bronze dog.

Thus, our approach goes beyond simply isolating unsafe knowledge by enabling the selective learning of useful knowledge from unsafe training data. We also note that the novelty of our approach has been recognized by reviewer fMFj. We hope this clarifies the distinction between our proposed method and existing approaches.

We would appreciate further clarification on the reviewer's assessment of limited novelty, as it is unclear why “only replacing the attention layers with Pristine Cross Attention” diminishes the novelty of the proposed mechanism or its capabilities.

Empirical Justification

Furthermore, the reviewer states that the paper lacks theoretical and empirical justification. We respectfully disagree with the assessment regarding empirical evidence, as the paper provides extensive empirical validation across multiple datasets, architectures, baselines, and experimental settings, including,

  1. Suppression of unsafe concepts, Improvement in generative performance: Demonstrated across ImageNet Dogs (Sec 3.1) and ImageNet Random (Appendix A.3.2) with both MMJiT and UNet architectures. We also demonstrate this on the open-world CC12M dataset (Sec 3.5).

  2. Explicit Transfer of Intended Properties: In Sec 3.3 we show that Pristine models are able to inherit the attributes of the unsafe concepts, without inheriting the unsafe concept identities themselves. For instance, a Pristine model is able to learn details about bronze and algae and compose this with dogs to create images which are not present in the training distribution. We also qualitatively demonstrate the same trend with MNIST (Sec 2.3)

3/ Additionally, we also compare with several other baselines from concept erasure literature (Sec 3.4) as well as alternate design choices (Sec 3.2) to further provide evidence differentiating Pristine Models from other techniques.

Given this breadth of empirical evaluation, we find the characterization of the paper as lacking empirical justification difficult to reconcile with the presented results. We also note that the empirical evidence has been acknowledged by reviewer Byaq. We would therefore appreciate clarification on which specific claims the reviewer believes remain insufficiently supported by experiments.

Explanations of the structure of Pristine Attention

Furthermore, the reviewer states that the paper provides “very few” explanations of the Pristine Cross Attention architecture. We would like to note that Section 2.1 provides explicit mathematical formulations, accompanied by an architectural illustration in Figure 3, while Appendix A.4 further extends the discussion to Pristine Self-Attention. Given these detailed explanations, we would appreciate clarification on which specific aspects of the architecture the reviewer finds insufficiently explained.

AuthorsOpenReview ↗

Rebuttal (continued)

Clarifying the Concept-Based Definition of Safe and Unsafe Data.

We would like to clarify our definition of safe and unsafe data, as described in Lines 99–103. Our framework adopts a concept-based definition of safety, where the model developer specifies a set of restricted concepts (e.g., Mickey Mouse, guns, or copyrighted logos) that should not appear in generated images. Training images containing any of these concepts are considered unsafe, while the remaining images are considered safe. Importantly, our method is not tied to any particular set of concepts or semantic categories; the restricted set is user-specified, allowing the same framework to be applied to arbitrary safe and unsafe concepts.

We believe this is a practically meaningful formulation, particularly for copyrighted characters, logos, and artworks, where specific concepts must be excluded from generation. Our goal is not to establish a universal semantic definition of safety, but to investigate whether models can learn useful knowledge from restricted images without generating the restricted concepts themselves. We believe our experiments directly evaluate this objective and would appreciate clarification on which aspect of this formulation the reviewer finds insufficient.

We therefore find the reviewer's statement that our definition does not account for the “actual semantics of text prompts or image content” unclear, since the presence or absence of restricted concepts in image content is precisely what defines our safety distinction. We would appreciate clarification on what additional semantic distinction the reviewer believes is missing and why this would invalidate our experimental conclusions.

Adversarial Prompts and Well-known Benchmarks

We thank the reviewer for suggesting these benchmarks. We would like to clarify that benchmarks such as I2P and P4D evaluate categories including nudity and other NSFW content, which are not the focus of our unsafe concepts. Notably, the original CC12M paper ([4], Sec. 2.2) explicitly states that images flagged by pornography detectors are excluded during dataset construction. Consequently, evaluating knowledge isolation for such concepts would be less meaningful in our setting, as the model has limited exposure to them during pretraining, making it difficult to distinguish successful isolation from a lack of learning in the first place.

Nevertheless, we have conducted additional adversarial evaluations on both ImageNet and CC12M, which demonstrate that Pristine substantially suppresses unsafe concepts even under adversarial prompting. For CC12M, we use the established HUB benchmark (ICCV 2025), which evaluates copyrighted characters using both direct prompts and adversarial prompts generated using the reviewer-requested Ring-A-Bell technique. Across all 10 restricted concepts, Pristine achieves an overall leakage rate of 5.22%, compared with 33.03% for Full and 1.17% for Safe, representing a substantial reduction in unsafe-concept generation.

Specifically, under Ring-A-Bell adversarial prompts, Pristine reduces leakage from 14.33% (Full) to just 3.00%. Additionally, on ImageNet (Dog setting), Pristine reduces leakage from 33.14% to 3.91% on Ring-A-Bell and from 22.67% to 2.30% on MMADiffusion. These results demonstrate that Pristine effectively suppresses restricted concepts even under adversarial prompting, approaching the safety of models trained exclusively on safe data. We refer the reviewer to our general response for further details on these additional evaluations.

References

[4] Changpinyo, S., Sharma, P., Ding, N., & Soricut, R. (2021, June). Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3557-3567). IEEE.

Reviewer_ByaqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose the Pristine Models. It is a pretraining-time approach for suppressing restricted concepts in text-to-image generation while preserving useful knowledge from unsafe training data.

The method introduces a modular attention mechanism that separates safe, unsafe, and shared knowledge through prompt decomposition, allowing the unsafe branch to be removed at inference.

Experiments on ImageNet and CC12M demonstrate improved safety–quality trade-offs compared with safe-only training and several post-hoc unlearning baselines.

Critical strengths

The paper proposes an interesting perspective on concept suppression. The paper approaches concept suppression from a pretraining perspective rather than relying entirely on post-hoc unlearning. The proposed modular attention mechanism is intuitive and provides a potentially useful way to retain transferable knowledge while suppressing restricted concepts.

The paper incorporates relative Reasonable experimental analysis. The authors evaluate across different attention architectures and provide several ablation studies investigating prompt decomposition and branch specialization. The comparisons with existing unlearning methods and additional CC12M experiments provide encouraging evidence for the proposed approach.

Critical weaknesses
  1. What about some latest large-scale text-to-image models?

Feel a little bit concerned about using on more latest text-to-image generation. Experiments on CC12M partially resolve my concerns, but recent text-to-image models and training pipelines have advanced considerably in generation quality. Although the authors consider both U-Net and Transformer architectures, stronger modern text-to-image models, such as Stable Diffusion 3.5, and Qwen-Image, would provide more convincing evidence of practical applicability. Glad to see how's the performance by applying the method on those architectures.

  1. Unclear practical advantages and robustness of knowledge isolation.

The proposed method requires architectural modifications and pretraining from scratch, which may limit its applicability to existing pretrained foundation models, which needs more explanations.

Moreover, the CC12M experiments still show noticeable restricted-concept leakage (13.29%, compared with 5.87% for the safe-only baseline), suggesting that removing the unsafe branch does not completely isolate restricted knowledge, which also needs more explanations.

Questions for authors
  1. Could the authors clarify why stronger modern text-to-image architectures and more recent concept-erasure methods are not included in the evaluation? How might the proposed approach scale to substantially larger pretrained generative models?

  2. How robust is the proposed knowledge isolation under implicit or adversarial prompts, particularly in the CC12M setting where the restricted-generation rate remains higher than that of the safe-only baseline?

2 comments
AuthorsOpenReview ↗

Author Rebuttal

Evaluation on Modern Architectures and Larger-Scale Data.

We would like to clarify that our experiments already include a modern MMDiT-based architecture. Specifically, MMJiT [1] builds upon the MMDiT architecture introduced in Stable Diffusion 3 [2], using the recent JiT [4] formulation with flow matching. Both Stable Diffusion 3.5 and Qwen-Image [3] similarly employ MMDiT-based architectures. Thus, our experiments evaluate Pristine within the same broad architectural family underlying these large-scale models, in addition to the previously conventional U-Net architecture.

Furthermore, we extend our experiments beyond ImageNet to CC12M, a substantially larger and more diverse open-world dataset. Notably, the original Stable Diffusion 3 paper [2] (Sec. 5.1, Table 2 etc) also uses CC12M as well as ImageNet for controlled experiments to help navigate their design space, recognizing them as a realistic text-to-image training setting. This further motivates CC12M as a meaningful large-scale setting for evaluating architectural contributions. While we agree that experiments at even larger foundation-model scales would strengthen our findings, pretraining such models from scratch across multiple configurations requires substantial computational resources beyond our available budget. Furthermore, commercial models such as Qwen-Image [7], FLUX [6] and Stable Diffusion 3.5 [8], do not provide the official training code/data making it hard to perform a faithful reproduction even at scale. We believe our evaluations across distinct architectures and at CC12M scale provide encouraging evidence of the method's applicability and potential for further scaling. We hope this addresses the reviewers' concerns.

Recent Concept-Erasure Baselines.

In response to the reviewer's concern regarding the inclusion of recent concept-erasure baselines, we would like to clarify that Section 3.4 already includes comparisons with several recently published methods, notably ScaPre (2026) and SPEED (2026), alongside UCE (2024) and SAFREE (2025). In particular, ScaPre and SPEED were both published this year. Additionally, we also compare Pristine Models with respect to a concept erasure baseline (lines 301-318) from the recent LLM literature [5] (arXiv 9th July 2026). We therefore respectfully note that our evaluation already includes recent concept-erasure methods and hope this clarification addresses the reviewer's concern regarding the recency of our baselines, if not we request the reviewer to suggest baselines which they feel would make our evaluation more complete.

Scope and Practical Applicability to Existing Foundation Models.

Regarding the concern about the practical applicability of Pristine to existing pretrained foundation models, we clarify the intended scope of our approach. Pristine Models is intentionally a pretraining-time approach, rather than a post-hoc unlearning technique. As explicitly discussed in the Introduction (Lines 51–53, 67, 91–93) and Conclusion (Lines 477, 484–485), our method targets unsafe concepts known before pretraining and is complementary to existing unlearning approaches, which remain necessary when new restrictions arise after pretraining (line 393-394, line 484-485). Therefore, the application to pretrained foundation models, is not the setting that we are working with. We do explore a Pristine (Unlearn) setting, which tries using Pristine Models as a post-training based approach (lines 319-323), we observe that it can improve the safety of the model, but the empirical data strongly suggests that a pretraining based approach works better.

From a practical perspective, we believe this approach can contribute meaningfully to the development of safer generative models. Many concepts that model developers may wish to suppress, such as copyrighted characters, weapons, and NSFW content, can be identified prior to pretraining. For such concepts, incorporating safety directly into pretraining provides an opportunity to isolate unsafe knowledge as it is learned, rather than attempting to remove it afterward. Importantly, our approach does not preclude post-hoc unlearning methods, which can complement Pristine Models by addressing newly identified restrictions after pretraining.

AuthorsOpenReview ↗

Rebuttal (continued)

Robustness to Implicit and Adversarial Prompts in the CC12M Setting.

We thank the reviewer for raising this concern. We emphasize that Pristine's robustness extends beyond explicit prompting, as our evaluation also considers implicit prompts that describe restricted concepts without directly naming them (Lines 297–298). Furthermore, we have conducted additional evaluations using adversarial prompts on both ImageNet and CC12M, which demonstrate Pristine's ability to suppress restricted concepts even under adversarial prompting. We refer the reviewer to our general response and our response to reviewer fMFj for detailed results and discussion.

Regarding CC12M, while Pristine does not yet match the unsafe-generation rate of the Safe-only baseline, it achieves a substantial reduction from 56.42% (Full) to 13.29%, while improving generation quality over Safe (82.31 vs. 79.30 on DPG-Bench and 5.02 vs. 4.79 on PRISM). These findings highlight the promising safety–quality trade-off achieved by Pristine at scale, while acknowledging opportunities for further improving knowledge isolation.

References

[1] https://peppaking8.github.io/minit2i/

[2] Esser, P., Kulal, S., Blattmann, A., Entezari, R., Müller, J., Saini, H., ... & Rombach, R. (2024, July). Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning.

[3] Wu, C., Li, J., Zhou, J., Lin, J., Gao, K., Yan, K., ... & Liu, Z. (2025). Qwen-image technical report. arXiv preprint arXiv:2508.02324.

[4] Li, T., & He, K. (2026). Back to basics: Let denoising generative models denoise. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 36115-36125).

[5] Roland, E., Cubuktepe, M., Martinez, E., Servaes, S., Pepper, K., Vaiana, M., ... & Cloud, A. (2026). Modular Pretraining Enables Access Control. arXiv preprint arXiv:2607.08077.

[6] https://github.com/black-forest-labs/flux

[7] https://github.com/QwenLM/Qwen-Image

[8] https://github.com/Stability-AI/sd3.5

Reviewer_fMFjOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Pristine Models, a text-to-image generation architecture that separates safe and unsafe knowledge into independently attention branches during pretraining, allowing the unsafe branch to be removed at inference.

Critical strengths

A new, well-motivated way to remove unsafe concepts during pretraining. This is a native way to remove unsafe concepts, offering an alternative to conventional post-hoc concept unlearning.

Critical weaknesses

No robustness tests

The evaluation primarily considers explicit requests for restricted concepts, without systematically testing adversarial prompts or incidental unsafe generation. The unsafe concepts can be potentially still in the models.

Safety improvements may be confounded by inference prompt sanitization

In Section 2.2, "Prompts are decomposed in the same way as during training", does it mean the model only receives the anonymized description without the restricted concept identity? If that's the case, the model's failure to generate an assault rifle may simply result from the absence of the corresponding textual instruction, rather than successful isolation of unsafe knowledge in the removed parameters. A missing experiment is to feed the original, undecomposed prompt directly into the retained shared branch and evaluate whether the restricted concept can still be generated. Another useful control would be to apply equivalent prompt sanitization to the Full baseline.

The eval is not that convincing

The primary ImageNet benchmark is overly artificial and may favor trivial suppression. In that setting, only 131 dog-related classes as safe and the remaining 869 ImageNet classes as unsafe. This setting can reward a generator that mostly produces dogs, rather than one that selectively suppresses unsafe concepts. In the more realistic CC12M setting, the proposed method still largely underperform

Questions for authors

See above.

3 comments
AuthorsOpenReview ↗

Author Rebuttal

Pristine Maintains Strong Suppression Under Adversarial Prompts.

We thank the reviewer for raising this concern. We would like to clarify that our evaluation is not limited to explicit prompts. As reported in Lines 297–298, we also evaluate implicit prompts that describe restricted concepts without explicitly mentioning their names (e.g., “a small red beetle with black spots” for a ladybug). Pristine achieves an unsafe-concept generation rate of 2.82% on U-Net (compared with 2.80% for Safe and 32.30% for Full) and 6.20% on MMJiT (compared with 40.21% for Full and 1.2% Safe). These results demonstrate that Pristine maintains effective suppression even when restricted concepts are invoked indirectly.

To further address the reviewer's concern, we additionally evaluate Pristine on adversarial prompts for both ImageNet and CC12M. On ImageNet (Dog setting), Pristine reduces leakage rates from 33.14% to 3.91% on Ring-A-Bell and from 22.67% to 2.30% on MMADiffusion, achieving performance comparable to the Safe model. On CC12M, using the existing HUB benchmark for copyrighted characters, Pristine reduces leakage from 14.33% to 3.00%, approaching the performance of the Safe model. These results further demonstrate Pristine's robustness to adversarial attempts to elicit restricted concepts. Additional details are provided in our general response.

Pristine's Suppression Is Not Merely an Effect of Prompt Sanitization.

We thank the reviewer for suggesting this important control experiment. To assess whether the suppression of unsafe concepts depends on prompt decomposition, we bypass decomposition and feed the entire prompt directly into the shared branch of the Pristine Model (based on the reviewer's suggestion). We then evaluate the model using the same unsafe-concept metrics. We observe that even without prompt decomposition, Pristine largely retains its ability to suppress unsafe concepts, with unsafe precision increasing only slightly from 0.54 to 0.58, unsafe recall from 0.22 to 0.26, and unsafe-concept generation accuracy from 0% to just 0.52%. These results suggest that the model's ability to suppress unsafe concepts does not rely solely on prompt decomposition, but is also learned through the Pristine architecture and training procedure. In other words, even when unsafe concepts are explicitly provided to the shared branch, the model remains largely unable to generate them.

SettingUnsafe PrecisionUnsafe RecallUnsafe Accuracy (%)
With Test-time Prompt Decomposition0.540.220
Without Test-time Prompt Decomposition0.580.260.52
Trained on Unsafe Anonymized Prompts0.760.443.37

Furthermore, to disentangle the effects of prompt anonymization from those of our proposed architecture, we conduct an additional control experiment. Specifically, we train a standard T2I model (analogous to Full), but use only anonymized prompts for unsafe training examples, such that unsafe concept identity names are never explicitly provided during training. Despite this, the model exhibits substantially higher unsafe-concept generation than Pristine, with unsafe precision/recall of 0.76/0.44 versus 0.54/0.22, and unsafe accuracy of 3.37% versus 0%, respectively. These results demonstrate that prompt anonymization alone cannot account for the observed suppression, highlighting the importance of explicitly isolating unsafe concept information within a dedicated, removable attention module.

AuthorsOpenReview ↗

Rebuttal (continuation)

Promising Results Beyond the Controlled Dog Setting.

We agree that ImageNet Dogs is a controlled setting, but this is intentional. The Dog Setting allows us to explicitly study a key advantage of Pristine Models, which is whether useful knowledge from restricted examples transfers to safe generation. For instance, attributes such as bronze and algae, rarely associated with dogs in safe training data, can be learned from unsafe examples and composed to create unseen images such as a bronze dog. Section 3.3 demonstrates this transfer, with Pristine achieving a compositional TPIPS of 0.7404 (measuring similarity in terms of object identity and material; higher is better), outperforming Safe (0.6890) and approaching Full (0.7705).

Importantly, the current method is not restricted to a particular definition of safe and unsafe. To demonstrate this, we also evaluate a Random Setting (Appendix A.3.2, Table 5), where 150 randomly selected safe classes and 850 unsafe classes create substantial semantic overlap between the two partitions. For example, a soccer ball is safe while a basketball is unsafe. This setting tests selective concept suppression beyond the semantically coherent Dog Setting. Despite this challenge, Pristine achieves unsafe accuracy of 0.01 on MMJiT (matching Safe, vs. 0.53 for Full), while substantially improving safe-concept recall from 0.39 (Safe) to 0.61 (Pristine). Crucially, Pristine achieves the best balanced accuracy (0.56), precision (0.43), and recall (0.60), outperforming both Safe (0.40/0.42/0.54) and Full (0.51/0.31/0.32). These results demonstrate that Pristine achieves a superior safety–quality trade-off even when safe and unsafe concepts are closely related.

We would also like to emphasize that Pristine demonstrates a promising safety–quality trade-off on CC12M, substantially reducing unsafe generation from 56.42% (Full) to 13.29%, while significantly improving generation quality over Safe (82.31 vs. 79.30 DPG-Bench and 5.02 vs. 4.79 PRISM). While we acknowledge that there remains room for further improving knowledge isolation, we believe these results provide encouraging evidence that Pristine can retain useful knowledge from unsafe training data while substantially suppressing unsafe concepts at scale.

Thank you for the detailed rebuttal and additional experiments. My concerns are kind of addressed, but I think a version that can address these in detail can really strengthen the paper. But I still share the other reviewers' concerns, like residual unsafe-concept leakage on CC12M, etc.

I am therefore raising my score from 2 to 3 as I like the idea of using a native way to remove unsafe concepts, as an alternative to conventional post-hoc concept unlearning. But I don't mind if this version gets rejected because I think there are indeed many aspects that can be improved.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for taking their time to provide constructive feedback. We are glad that the reviewers found the pretraining time approach to concept suppression as well-motivated (fMFj) as well as interesting (Byaq). We also find it encouraging that our research has been acknowledged as novel (fMFj). Furthermore, the reviewers found the paper writing clear and easy to follow (fMFj, Byaq).

A major concern was the lack of experiments with studying the robustness to adversarial prompting techniques. To that end, we have tested our ImageNet-based models with Ring-A-Bell [1] and MMA-Diffusion [2]. Based on the suggestion provided by reviewer fMFj, we give the entire prompt to the shared branch (which is trained on both safe and unsafe images). Furthermore, we also evaluate our CC12M model on adversarial prompts from Ring-A-Bell.

For ImageNet (Dog setting), the Full model exhibits high leakage rates of 33.14% and 22.67% under Ring-A-Bell and MMADiffusion adversarial prompts, respectively. In contrast, Pristine Models reduce these rates to just 3.91% and 2.30%, respectively, achieving safety levels comparable to the Safe model trained exclusively on safe images. These results demonstrate that Pristine Models can effectively suppress unsafe concepts even when they are elicited implicitly through adversarial prompting.

ImageNetRing-A-Bell (%)MMA-Diffusion
Safe1.731.73
Full33.1422.67
Pristine3.912.30

For CC12M, we also evaluate our model on adversarial prompts designed to elicit copyrighted characters using the existing HUB benchmark [3], which employs Ring-A-Bell [1] to generate adversarial prompts. The Full model exhibits a leakage rate of 14.33%, whereas Pristine reduces this rate to just 3%, achieving performance close to that of the Safe model.

CC12MRing-A-Bell (%)
Safe1.09
Full14.33
Pristine3.00

References

[1] Tsai, Y. L., Hsu, C. Y., Xie, C., Lin, C. H., Chen, J. Y., Li, B., ... & Huang, C. Y. (2024, May). Ring-a-bell! how reliable are concept removal methods for diffusion models?. In International Conference on Learning Representations (Vol. 2024, pp. 41543-41554).

[2] Yang, Y., Gao, R., Wang, X., Ho, T. Y., Xu, N., & Xu, Q. (2024). Mma-diffusion: Multimodal attack on diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 7737-7746).

[3] Moon, S., Lee, M., Park, S., & Kim, D. (2025, October). Holistic unlearning benchmark: A multi-faceted evaluation for text-to-image diffusion model unlearning. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV) (pp. 16356-16366). IEEE.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.