- Rating
- 2(+1)
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper focuses on the safety issue of the conversational image-generation systems. The authors proposed an attack method, called I-Con, which supplies a harm-space scene and a coarse category, then uses three fixed text turns before a single image request. They claim that prior work carries the harmful intent in their prompts or requests, requiring the attacker to specify a harm event or leaving the target system to elaborate on it. I-Con requires no external attacker model, adaptive search, repeated image attempts, or access beyond the ordinary chat interface. And I-Con achieves a higher attack success rate on GPT serial models and Gemini serial models compared with MMA-Diffusion, DACA, CoJ-Attack, etc.
Critical strengths
- Conversational image-generation systems are widely used and raise some critical safety issues, for example, the harmful content generation. This paper propose a jailbreak attack whose inputs and requests do not contain harmful informantion but can guide the models generate harmful contents. This is a more covert method, which exposes the potential safety risks and ethical issues when we deploy the generation models and provide servies to public.
- The paper conducts experiments with cutting-edge models such as GPT and Gemini, showing the attack effectiveness against latest technologies.
Critical weaknesses
- The proposed method makes sense but what challenge does this paper aims to address is unclear. And what key contribution does this paper make?
- The paper proposes neither a new algorithm nor a new model. It is just about a kind of prompt refining. The novelty is limited.
- What is the main contribution of this paper? A item-by-item list is good to show the contributions.
- The dataset used in this paper is created by themselves. So it is hard to say if there are bias or fairness issues because the dataset is not publicly available.
Questions for authors
All four items in the weaknesses form.
5 comments
Response to Reviewer Sx3y [2/2]
[W2] Novelty beyond prompt refinement.
Answer: The simplicity of I-CON exposes a practical safety weakness: fixed scene-development requests can induce the system to supply the concrete harmful event itself. A malicious user can execute the attack using a starting scene, a coarse category, three fixed text requests and one image request. No external attacker model, search or retries are needed, making the attack straightforward to execute and the safety weakness exploitable through a simple, fixed procedure. Our contribution is to study this use of scene construction and measure how the resulting content reaches the user.
- Event construction at the same call budget. On GPT-Image-1.5, I-CON reaches 33.21% ASR vs. 10.71% for refinement that only adds visual detail, with the same inputs and three text calls. Replacing only the Concretize request with a neutral continuation gives 25.71% (Holm-adjusted p = 0.0418; Sec. 4.2, Table 3(b) of the revised manuscript). Prior attacks given the same harm-sparse inputs (RICE [G9], Crescendo [G6] and Inception [G7]) reach at most 6.07% across the four systems. The gain thus comes from the instruction content, not from the inputs or the number of calls.
- A safety exposure that image ASR misses. On GPT-Image-1.5, 40.54% of the sessions that end in a provider block or refusal contain Intermediate Leakage, and resubmitting the last released description alone yields a detector-flagged unsafe image in 9.46% of the sessions that returned none (Sec. 4.3; App. D.6.1 of the revised manuscript). A session counted as a failure by image ASR can thus still expose a description that can be reused to elicit an unsafe image.
- What the analysis adds. Self-prompting [G2] already delegates concrete content generation to the target ([W1-1]). We examine harmful event construction within a supplied visual scene and trace who introduces the event, when its description reaches the user and whether it appears in the image.
Our original submission included the equal-budget refinement control, the matched-input comparisons, and the leakage and standalone-rendering results (Sec. 4.2, Tables 2 and 3, Sec. 4.3, and App. D.6.1). In the revised manuscript, we have added the fixed-turn stage-replacement control to Sec. 4.2 and Table 3 and added a comparison with self-prompting and implicit-prompt attacks to App. J.
[W4] Dataset availability, bias, and fairness.
Answer: Our 280 harm-sparse scenes are derived from the public I2P benchmark [G8] (MIT license), and we will release all of them with the camera-ready version.
- Construction. I2P is also used to evaluate prior attacks, including RICE [G9], the metaphor-based attack [G5] and Inception [G7]. The 280 cases contain 40 per I2P category. An LLM drafted harm-sparse scenes under a fixed rule that removes harmful actions, relations, instruments and outcomes, and the authors reviewed them. In a two-annotator audit, the rewrites keep the subjects and, in most cases, the setting of the source scene and almost always remove its harmful content (App. I.2 of the revised manuscript).
- Examples and release. App. I.1 of the revised manuscript lists example scenes from every category. With the camera-ready version, we will release all 280 scenes together with each scene’s I2P source prompt and the frozen rewrite rule, so readers can inspect each source prompt and the construction criteria.
- Composition and bias. Gendered terms appear in 34.29% of the scenes: 15.71% use only male terms, 17.86% only female terms, and 0.71% both. The I2P-derived sexual cases are skewed toward female terms (62.50%), which we now report in App. I.1. Because all methods are compared on identical scenes and categories (Table 2), the composition is the same for every method, and the release lets readers examine bias and fairness directly.
Our original submission described the dataset construction and the comparisons on identical harm-sparse scenes and categories (Sec. 3.1 and 4.1, Table 2). In the revised manuscript, we have added the dataset composition, the observed gender skew, and the release plan to App. I.1.
We hope these clarifications address your concerns about the contributions and dataset. If there is any remaining question, please let us know; we would be glad to clarify it during the discussion period.
Dear Reviewer Sx3y,
We are grateful for the time the reviewer has spent on our manuscript. The review asked us to state explicitly what problem the paper addresses and what it contributes, and answering these questions has helped us present the work more clearly.
We would appreciate hearing whether our responses have resolved the concerns raised in the review. If any point is still unclear or needs more detail, we would be happy to provide further explanation during the discussion.
We thank the reviewer for considering our responses.
Best regards,
Authors
Response to Reviewer Sx3y [1/2]
Dear Reviewer
Sx3y, We sincerely thank you for your feedback. We particularly appreciate your recognition of a more covert attack that exposes safety risks in deployed systems, and experiments with cutting-edge models such as GPT and Gemini. Your comments on the problem statement, contributions, and dataset have helped us present our work more clearly. We address your concerns and questions in detail in the responses below.Answer:We address a safety weakness of conversational image generation: an attacker can leave the design of a concrete harmful event to the system itself.We have clarified how a harmful direction differs from a concrete harmful event and how I-CON reduces the need for attacker-authored harmful details in Sec. 1 of the revised manuscript (L078–080 and L089–090).
Answer:Our four contributions address the attack procedure, the resulting exposure, the ability of existing safeguards to detect it, and a mitigation:j3Afandos3Nrecognized this analysis as offering "useful insights beyond final-image ASR" and showing "how harmful content develops throughout a conversational generation process".j3Af[W3, Q3]).Our original submission described the attack procedure and evaluation measures, which appear in Sec. 3 and 3.3 of the revised manuscript. It also reported the primary attack comparison, event-level analysis and offline safeguard evaluation, which appear in Table 1 and Sec. 4.3 and 4.4, respectively. The gate is added in the revision and appears in Sec. 4.5. Sec. 1 summarizes the four contributions in Lines 104–107.