- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes Pristine Models, a text-to-image generation architecture for improving safe image generation with high image quality during model pretraining. Authors utilize MLLMs to construct training prompts from image datasets by using the predefined “safe” and “unsafe concepts to train Safe and Unsafe Models via the proposed Pristine Attention.
Critical strengths
None
Critical weaknesses
The novelty of the proposed method is very limited only replacing the attention layers with the Pristine Cross Attention. There are only few explanation of the structure of Pristine Cross Attention without empirical or theoretical justification.
Authors’ goal is to training a “safe” text-to-image model by pretraining via defined “safe” data pairs. However, authors didn’t not define the concept between “safe” and “unsafe” according to actual semantic of text prompt or image content. This leads to whole experiments fail to support the central claim of this paper. Moreover, authors didn’t show how the proposed model works under well-know “unsafe” prompt benchmark like I2P [1] and adversarial benchmarks such as Ring-A-Bell [2] and P4D [3].
[1] Schramowski, Patrick, et al. "Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.
[2] Tsai, Yu-Lin, et al. "Ring-a-bell! how reliable are concept removal methods for diffusion models?." International Conference on Learning Representations. Vol. 2024. 2024.
[3] Chin, Zhi-Yi, et al. "Prompting4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts." arXiv preprint arXiv:2309.06135 (2023).
Questions for authors
In prompt decomposition, what is the method used to extract “safe” part from a unsafe prompt? What is the rule to decide “safe” and “unsafe” concept from the prompt? Does author consider there are overlap between “safe” and “unsafe” in different contexts?
2 comments
Rebuttal (continued)
Clarifying the Concept-Based Definition of Safe and Unsafe Data.
We would like to clarify our definition of safe and unsafe data, as described in Lines 99–103. Our framework adopts a concept-based definition of safety, where the model developer specifies a set of restricted concepts (e.g., Mickey Mouse, guns, or copyrighted logos) that should not appear in generated images. Training images containing any of these concepts are considered unsafe, while the remaining images are considered safe. Importantly, our method is not tied to any particular set of concepts or semantic categories; the restricted set is user-specified, allowing the same framework to be applied to arbitrary safe and unsafe concepts.
We believe this is a practically meaningful formulation, particularly for copyrighted characters, logos, and artworks, where specific concepts must be excluded from generation. Our goal is not to establish a universal semantic definition of safety, but to investigate whether models can learn useful knowledge from restricted images without generating the restricted concepts themselves. We believe our experiments directly evaluate this objective and would appreciate clarification on which aspect of this formulation the reviewer finds insufficient.
We therefore find the reviewer's statement that our definition does not account for the “actual semantics of text prompts or image content” unclear, since the presence or absence of restricted concepts in image content is precisely what defines our safety distinction. We would appreciate clarification on what additional semantic distinction the reviewer believes is missing and why this would invalidate our experimental conclusions.
Adversarial Prompts and Well-known Benchmarks
We thank the reviewer for suggesting these benchmarks. We would like to clarify that benchmarks such as I2P and P4D evaluate categories including nudity and other NSFW content, which are not the focus of our unsafe concepts. Notably, the original CC12M paper ([4], Sec. 2.2) explicitly states that images flagged by pornography detectors are excluded during dataset construction. Consequently, evaluating knowledge isolation for such concepts would be less meaningful in our setting, as the model has limited exposure to them during pretraining, making it difficult to distinguish successful isolation from a lack of learning in the first place.
Nevertheless, we have conducted additional adversarial evaluations on both ImageNet and CC12M, which demonstrate that Pristine substantially suppresses unsafe concepts even under adversarial prompting. For CC12M, we use the established HUB benchmark (ICCV 2025), which evaluates copyrighted characters using both direct prompts and adversarial prompts generated using the reviewer-requested Ring-A-Bell technique. Across all 10 restricted concepts, Pristine achieves an overall leakage rate of 5.22%, compared with 33.03% for Full and 1.17% for Safe, representing a substantial reduction in unsafe-concept generation.
Specifically, under Ring-A-Bell adversarial prompts, Pristine reduces leakage from 14.33% (Full) to just 3.00%. Additionally, on ImageNet (Dog setting), Pristine reduces leakage from 33.14% to 3.91% on Ring-A-Bell and from 22.67% to 2.30% on MMADiffusion. These results demonstrate that Pristine effectively suppresses restricted concepts even under adversarial prompting, approaching the safety of models trained exclusively on safe data. We refer the reviewer to our general response for further details on these additional evaluations.
References
[4] Changpinyo, S., Sharma, P., Ding, N., & Soricut, R. (2021, June). Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (pp. 3557-3567). IEEE.
Author Rebuttal
Novelty
We respectfully disagree with the reviewer's assessment of the novelty of our approach and would like to clarify our contributions. Pristine Models introduce a mechanism for isolating unsafe knowledge into a detachable module during pretraining in the context of text-to-image generation, rather than attempting to remove such knowledge after training. Importantly, Pristine Attention also enables selective control over how knowledge learned from unsafe images is utilized, allowing useful attributes to be retained without generating the unsafe concepts themselves. This is demonstrated in Section 3.3, where attributes such as bronze and algae, learned from unsafe images, can be transferred to safe categories to generate novel compositions such as a bronze dog.
Thus, our approach goes beyond simply isolating unsafe knowledge by enabling the selective learning of useful knowledge from unsafe training data. We also note that the novelty of our approach has been recognized by reviewer fMFj. We hope this clarifies the distinction between our proposed method and existing approaches.
We would appreciate further clarification on the reviewer's assessment of limited novelty, as it is unclear why “only replacing the attention layers with Pristine Cross Attention” diminishes the novelty of the proposed mechanism or its capabilities.
Empirical Justification
Furthermore, the reviewer states that the paper lacks theoretical and empirical justification. We respectfully disagree with the assessment regarding empirical evidence, as the paper provides extensive empirical validation across multiple datasets, architectures, baselines, and experimental settings, including,
Suppression of unsafe concepts, Improvement in generative performance: Demonstrated across ImageNet Dogs (Sec 3.1) and ImageNet Random (Appendix A.3.2) with both MMJiT and UNet architectures. We also demonstrate this on the open-world CC12M dataset (Sec 3.5).
Explicit Transfer of Intended Properties: In Sec 3.3 we show that Pristine models are able to inherit the attributes of the unsafe concepts, without inheriting the unsafe concept identities themselves. For instance, a Pristine model is able to learn details about bronze and algae and compose this with dogs to create images which are not present in the training distribution. We also qualitatively demonstrate the same trend with MNIST (Sec 2.3)
3/ Additionally, we also compare with several other baselines from concept erasure literature (Sec 3.4) as well as alternate design choices (Sec 3.2) to further provide evidence differentiating Pristine Models from other techniques.
Given this breadth of empirical evaluation, we find the characterization of the paper as lacking empirical justification difficult to reconcile with the presented results. We also note that the empirical evidence has been acknowledged by reviewer Byaq. We would therefore appreciate clarification on which specific claims the reviewer believes remain insufficiently supported by experiments.
Explanations of the structure of Pristine Attention
Furthermore, the reviewer states that the paper provides “very few” explanations of the Pristine Cross Attention architecture. We would like to note that Section 2.1 provides explicit mathematical formulations, accompanied by an architectural illustration in Figure 3, while Appendix A.4 further extends the discussion to Pristine Self-Attention. Given these detailed explanations, we would appreciate clarification on which specific aspects of the architecture the reviewer finds insufficiently explained.