acceptodds
Under review as a conference paper at ICLR 2027

Being Trapped then Mitigated: Defending Malicious Jailbreak Prompts via Embedded Honeypots in Text-to-Image Models

Abstract

The rapid popularity of Text-to-Image (T2I) models has raised widespread concerns about their non-ethical application for generating inappropriate or harmful content. To prevent their misuses, several methods for safety alignment, also known as concept erasure, have been developed, yet vulnerable to adversarial attacks, which exploit optimized jailbreak prompts to break the safety barriers, regenerating undesired content. In this paper, distinct from previous arts that patch model vulnerabilities via adversarial training and external defensive tools, we introduce a proactive defense framework, HoneyT2I, which exploits embedded honeypots as artificial vulnerabilities to manipulate jailbreak optimization, making the jailbreak behavior of resulting prompts to be detected and mitigated easier. Specifically, the honeypots of unsafe concepts are embedded into pretrained T2I models with the backdoor learning paradigm, ensuring target concepts are regenerated once the corresponding trigger emerges. Moreover, Singular Vector Decomposition is utilized to extract honeypot and difference signatures from prompt collections containing target concepts and their backdoor versions, which further detect and mitigate jailbreak behavior at the inference time. Extensive experiments show that HoneyT2I effectively defend jailbreak prompts generated by various attacks without sacrificing generation quality, while possessing the promising scalability, i.e., being integrated with existing alignment methods and applied to advanced T2I models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Text-to-Image Generation, Concept Erasure, Jailbreak Defense, Honeypot Defense

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Pd2JOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HoneyT2I, a proactive defense framework against jailbreak attacks on text-to-image models. The framework embeds artificial vulnerabilities, termed honeypots, into T2I models to attract jailbreak optimization toward detectable representation patterns. It then extracts honeypot and difference signatures through SVD to identify and mitigate jailbreak prompts during inference. Experiments demonstrate improvements in jailbreak robustness while largely preserving benign image generation quality.

Critical strengths
  • Clear writing and well-organized presentation. The paper is well written and clearly presents the research motivation, proposed methodology, and experimental evaluation.

  • Interesting detection and mitigation framework. HoneyT2I combines honeypot-based jailbreak identification with representation-level steering, enabling the detection and mitigation of adversarial prompts while preserving image generation capabilities.

Critical weaknesses
  • Unclear necessity of the two concept-erasure objectives. Both LeL_e and LheL_{he} aim to suppress unsafe concepts, but LheL_{he} additionally preserves other prompt semantics. Their optimization targets may differ even for the same prompt, potentially introducing conflicting supervision. The authors should clarify their complementary roles and provide independent ablations.

  • Insufficient comparison with embedding-steering defenses. Beyond SAFREE, the paper omits relevant methods such as Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions [1] and SteerDiff [2], which directly manipulate text embeddings to suppress unsafe generation. Comparisons with these methods would better demonstrate the advantages of HoneyT2I.

  • Limited utility evaluation and potential training–evaluation overlap. COCO captions are used for both utility preservation during training and FID/CLIP evaluation, but it is unclear whether the samples overlap. Moreover, the evaluation lacks analysis of benign prompts related to erased concepts or incorrectly identified as jailbreaks and subjected to representation steering.

  • Limited evaluation across models and unsafe concepts. The main experiments focus on SD v1.4 and nudity, leaving generalization to modern T2I models and other harmful concepts insufficiently established. Moreover, it remains unclear whether concept-specific steering directions can be combined or applied jointly without interference.

  • Missing ablation on loss-weight sensitivity. The ablations examine trigger length, detection threshold, and steering strength, but omit the loss weights λ1\lambda_1, λ2\lambda_2, and λ3\lambda_3. Their effects on jailbreak robustness, concept erasure, and generation utility remain unclear.

[1] Li, Zhiwen, et al. "Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions." Proceedings of the ACM Web Conference (WWW), 2025.

[2] Zhang, Hongxiang, Yifeng He, and Hao Chen. "SteerDiff: Steering towards Safe Text-to-Image Diffusion Models." arXiv preprint arXiv:2410.02710, 2024.

Questions for authors
  • Concept-erasure objectives. Could the authors explain the distinct roles of LeL_e and LheL_{he} and why both are necessary despite their potentially overlapping or conflicting objectives?

  • Baseline comparison. Could the authors compare HoneyT2I with additional embedding-steering defenses, particularly Responsible Diffusion Models [1] and SteerDiff [2]?

  • Utility evaluation. Are the COCO training and evaluation prompts disjoint? How does HoneyT2I affect benign prompts related to erased concepts or falsely detected as jailbreaks?

  • Generalization and scalability. How does HoneyT2I perform on modern T2I models and other unsafe concepts? Can multiple concept-specific steering directions be combined without interference?

  • Loss-weight sensitivity. How do different values of λ1\lambda_1, λ2\lambda_2, and λ3\lambda_3 affect jailbreak robustness, concept erasure, and generation utility?

[1] Li, Zhiwen, et al. "Responsible Diffusion Models via Constraining Text Embeddings within Safe Regions." Proceedings of the ACM Web Conference (WWW), 2025.

[2] Zhang, Hongxiang, Yifeng He, and Hao Chen. "SteerDiff: Steering towards Safe Text-to-Image Diffusion Models." arXiv preprint arXiv:2410.02710, 2024.

1 comment

Thanks for your time and effort in reviewing this paper. Your comments are valuable to improve the quality of this paper. As the author-reviewer discussion period will end in few days, we gently request that you read the rebuttal and check whether your concerns are addressed. Thanks again for your time and help in improving the quality of this paper.

Regards from the authors.

Reviewer_AdG2OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a novel architecture called HoneyT2I, which injects a “honeypot” backdoor into pretrained T2I models. When an attacker optimizes a jailbreak prompt to exploit this shortcut, the optimized prompt leaves a detectable signature in the textual representation space. HoneyT2I then uses the extracted honeypot signature to identify such jailbreak prompts, and applies the difference signature to remove the jailbreak-related direction from the representations, and steers generation away from the target concept. Overall, HoneyT2I improves robustness against jailbreak attacks while preserving generation quality.

Critical strengths

Overall, the proposed method is sound, and its design is quite comprehensive, as it mitigates jailbreak prompts while preserving generation quality. In addition, the experimental design is thorough and provides evidence to support most of the claims.

Critical weaknesses

The design of this method is quite similar to that of Trapdoor [1]. Although Trapdoor was originally proposed for image classification, its core idea of using trapdoors to attract adversarial examples is very similar to the honeypots used in this work. The authors should explicitly compare this work with Trapdoor and clarify the differences beyond the application domain. Otherwise, readers may easily regard this work as merely transferring a similar idea from image classification to T2I. [1] Shan, Shawn. "Using honeypots to catch adversarial attacks on neural networks." Proceedings of the 8th ACM Workshop on Moving Target Defense. 2021.

Questions for authors
  1. Refer to the weakness.
  2. The experiments in the paper consider only “nudity” as the target concept for jailbreak evaluation. Could the authors further test other concepts, such as “violence” from the I2P subset? This would further strengthen the evaluation and support the paper’s claims.
  3. The paper claims that the method can mitigate jailbreak prompts while preserving image generation quality. However, the CLIP scores and FID are computed only on the COCO dataset, which does not contain harmful prompts. If possible, could the authors evaluate generation quality by reporting CLIP and FID on the images generated after mitigating the jailbreak prompts.
  4. Here are some suggestions for improving the presentation and fixing typos: (1) Lines 044–045: Please check/correct the citation format. (2) Figure 1 is not very informative. Although it is intended to show the differences between prior methods and the proposed method, it is difficult to understand how the prior methods actually work. The authors should consider adding more information to this figure. (3) Lines 244–245: The format of the symbol Le (4) Figure 2 has a very similar problem to Figure 1. It would be better to include more technical details in the figure so that readers can grasp the idea more quickly.
Reviewer_KbuaOpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes HoneyT2I, a proactive defense framework against jailbreak attacks on T2I models. Unlike existing defenses that focus on patching model vulnerabilities, HoneyT2I intentionally introduces controlled vulnerabilities, called honeypots, into the text encoder through backdoor learning. These honeypots guide jailbreak optimization toward predictable representation patterns. The framework then uses SVD to extract honeypot and difference signatures, which are used to detect and mitigate jailbreak prompts during inference. Experimental results demonstrate that HoneyT2I effectively defends against various jailbreak attacks while largely preserving concept erasure effectiveness and benign image generation quality.

Critical strengths
  • The proposed approach proactively reshapes the attack surface by embedding controlled honeypots, offering an alternative to conventional reactive jailbreak defenses.
  • The framework combines signature-based jailbreak detection with representation-level mitigation to suppress unsafe generation without directly rejecting suspicious prompts.
Critical weaknesses
  • Unclear Role of Honeypots in Black-Box Defense. Appendix D.4 reports low ASRs against several black-box jailbreak attacks, suggesting promising defense performance. However, the contribution of honeypot-induced optimization to these results remains unclear. Unlike local white-box attackers targeting the honeypot-embedded model, black-box attackers interact with the fully protected service and observe outputs after detection and mitigation. As a result, the artificial vulnerabilities may no longer serve as attractive optimization targets. The reported low ASRs do not establish whether black-box attacks are guided toward honeypots or whether the defense performance mainly comes from concept erasure and inference-time mitigation.

  • The threat model allows attackers to access the parameters and gradients of the honeypot-embedded model while keeping deployment-specific defense states private. Under this setting, attackers could optimize jailbreak prompts locally and then refine their strategies based on feedback from the protected T2I service. Although the paper evaluates signature-aware evasion and trigger removal, these experiments do not fully capture attackers who iteratively adapt their strategies after observing failures against the complete detection and mitigation pipeline. Such attackers may discover alternative exploitation paths that bypass honeypot-based detection.

Questions for authors
  1. In the existing jailbreak experiments, what proportion of successful attacks against the honeypot-embedded model are detected by the honeypot signature before mitigation?

  2. Do the reported adaptive attacks involve repeated queries to the fully protected T2I service, with attackers updating their strategies based on the observed outputs?

  3. In Appendix D.4, are the black-box jailbreak prompts optimized directly against the complete HoneyT2I inference pipeline, including detection and mitigation? Do the existing results also show how frequently these prompts trigger honeypot-based detection?

  4. I would consider raising my score if the authors adequately address this concern.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.