acceptodds
Under review as a conference paper at ICLR 2027

Which Image Property Carries The Jailbreak? A Controlled Dissection Of Image-To-Text Jailbreaks

Abstract

We ask which image-side property causes safety failure in multimodal image-to-text jailbreaks, and answer with a controlled dissection rather than another leaderboard entry. We hold the harmful instruction byte-identical, probe four published attack families across five victims (GPT-4o, Claude S.4.5, Qwen3-VL-8B, Gemma-3-4B, GLM-4.1V) on a 313-prompt benchmark slice, and score every response with an automated rubric judge; the paired ablations are sampled draws per prompt and the baseline matrix is single-draw, as declared in the method. The image is the carrier of the attack: a bare harmful query and a benign unrelated image sit at or near zero on every victim, while attaching an attack composite lifts success substantially. Three findings are robust across the panel. Per-tile density screening cannot separate attack tiles from size-matched benign distractors, so density-only screening is not a viable filter. The compositional ladders are either confounded by payload-text visibility or null, rather than rising with a clean image-side descriptor. On the headroom model with the most room to move, the topical relation between a generated scene and the harmful query raises refusal by about one tenth of the ASR scale (rubric-judged, three draws per prompt, single-victim), an observation that is bounded and descriptive rather than causal: it is attributed under the measured-descriptor rule to the relatedness manipulation, with image-text congruence unmeasured, and reproduced by a within-category harm-salience control on overlapping stimuli. The API and reasoning-augmented rows are floored or saturated, which bounds where an effect can be measured. The dissection is scoped to composite and typographic-layout families; every claim is conditional on our rubric judge, and we rely on signs and orderings rather than on magnitudes inside the roughly one-tenth cross-judge band.

open until 14 Dec 2026

est. 32% chance this paper gets accepted at ICLR 2027.

Reject 68%Accept 32%

What do you think this paper will get?

All positions stay anonymous.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.