acceptodds
Under review as a conference paper at ICLR 2027

Seeing Is Not Recognizing: Exact Target Identification as the Bottleneck for Toxicity Detection in AI-Generated Visual Illusions

Abstract

Controllable diffusion models can embed specified targets into natural scenes with benign surface content to create visual illusions. Toxic visual illusions pose new challenges for multimodal content moderation. Existing benchmarks have limited coverage and lack non-toxic controls and non-illusion images, while end-to-end evaluation makes it difficult to localize errors. We introduce ToxicIllusion, a human-annotated benchmark comprising 14,940 images generated from 166 toxic and non-toxic targets across four target types: English text, Chinese text, gesture, and symbol. We propose a three-stage evaluation framework covering illusion perception, target identification, and toxicity classification to localize errors across these stages. We examine the relationship between target identification and toxicity detection through output state decomposition and conditional toxicity analysis. We further analyze visual representations and relevance maps to examine the representational basis of target identification failures and models' spatial responses to surface scenes and targets. Our results show that current vision-language models do not reliably connect illusion perception, target identification, and toxicity classification. Target identification is a key bottleneck in toxicity detection, and correct target identification is consistently associated with more reliable toxicity classification. The visual representations of pretrained visual encoders are dominated by surface scenes. Relevance maps further show that the evaluated open-weight models primarily attend to surface scenes rather than targets.

Primary area:
datasets and benchmarks
Keywords:
AI-generated visual illusions, vision-language models, target identification, toxicity detection, multimodal content moderation, datasets and benchmarks

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_Bc2BOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes ToxicIllusion, a benchmark with 14,940 synthetic images with both toxic and non-toxic targets in common natural scenes, which includes English text, Chinese text, gestures, and symbols. It suggests a three-stage assessment scheme that tests illusion perception, accurate target identification, and toxicity classification separately and compares eight vision-language models based on different APIs and open-weight models to a single baseline with zero-shot prompting. The results indicate that identification of the target is very low even though it is perceived by models with accuracy ranging from 1.1-4.4%. Proper identification of the targets is that is related to a significantly higher toxicity-classification performance. Representation-similarity analysis and relevance maps also indicate that models are more concerned with features of the surface, as opposed to the embedded targets. The authors believe that one of the main challenges in AI-generated visual illusions is the lack of precise identification.

Critical strengths

[S1] Divides the process of a toxicity evaluation into perception, precise identification of the target and classification of toxicity, allowing for more precise error analysis.

[S2] Introduces a large data set of 14,940 images of toxic and non-toxic targets with good inter-annotator agreement k = 0.765 and also failed generations.

[S3] Assesses eight models that are based on APIs and open weighted, using task-specific metrics and supplemental representation analyses.

[S4] Recognises that detection of target recognition is an important bottleneck in the process of detecting visual-toxicity, and that this can be a limitation of end-to-end toxicity scores.

Critical weaknesses

[W1] For instance, the accuracy of the target identification depends on the ability to match the predicted target with the ground truth; how the alternative name, synonym, or variations in the gesture/symbol description will be addressed is not clearly stated in this paper. This could be punishing for semantically correct identifications that are incorrectly worded.

[W2] Toxicity classification is ‘gated’ by the model's prediction of the visibility, such that non-visible prediction is mapped to non-toxic. This is a mixture of perceptiveness and classification errors, which can make the overall toxicity score hard to interpret without the context of the individual error scores.

[W3] Lacks task-specific baselines such as OCR systems for text targets, but also task-specific gestures and symbols recognition. Confidence intervals or statistical tests would also be useful in determining the strength of the model comparisons reported.

[W4] The representation and relevance-map analyses would be improved by providing more detail in the methodology and reproducibility information. The analysis of local or multi-scale visual properties could also be used to further investigate whether the observed scene dominance extends beyond global representations.

[W5] The benchmark is based on synthetic images; this may restrict the findings of the benchmark to real-world visual illusions. The results would have improved validity if there was evaluation on a smaller, representative sample of real and/or photo edited examples.

[W6] While the paper references related multimodal safety research, a more robust evaluation of the proposed benchmark and the evaluated models may be possible by comparing these with other systems designed to detect harmful content (either implicit or visually embedded).

Questions for authors

[Q1] What is the method for scoring exact target identification for multilingual text, gestures and symbols? Are other names, different words, or slight variations of wording OK? Does Chinese text output work for Chinese character and pinyin? Would the reported results be different with less stringent matching criteria?

[Q2] Will the data set, the evaluation code, the structured prompts, output parsing scripts, and the generation of the relevance maps be made publicly available? Can the output of models be cached for API-based models for reproduction?

[Q3] Can you report toxicity classification based on ground truth visibility and existing protocol (model predicted visibility)? This comparison would facilitate the separation of the errors in illusion perception from the errors in toxicity classification.

[Q4] What is the sensitivity of the representation-similarity results to the visual encoder, target type and the degree of human visibility (V1 or V2)? These breakdowns would aid in the assessment of the claim of the dominance of surface-scene features in target representations.

[Q5] Could you share more detail of the approach to developing the spatial relevance maps and how the accuracy of the maps was determined? This would allow readers to judge more accurately the support of the claim for embedded targets versus surface scenes afforded by the visualisations.

Reviewer_QTF7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper examines the limitations of VLM in identifying harmful content hidden in AI-generated visual illusion images. The author has constructed a benchmark dataset named "ToxicIllusion", covering four target types: English text, Chinese text, gestures, and symbols. An evaluation framework consisting of three stages - visual illusion perception, specific target recognition, and toxicity classification - has been designed to analyze the specific reasons for the failure of model security detection. By evaluating eight visual language models such as GPT, the author found that although some models can perceive the existence of hidden content in the images, their ability to accurately identify specific targets is generally weak. When the model correctly identifies the target, the performance of toxicity classification significantly improves. Further visual feature analysis and spatial correlation analysis indicate that the existing visual encoders mainly focus on the surface scene of the image and have difficulty effectively extracting the overall structure and semantic information of the hidden target. The core contribution of this paper lies in proposing a more granular visual illusion security assessment framework and revealing that accurately identifying hidden targets is the key bottleneck affecting the toxicity detection performance of visual language models, providing a new research direction for improving multimodal security detection.

Critical strengths

This paper proposes a three-stage assessment framework with clear diagnostic value, which decomposes the toxicity detection in visual illusions into Illusion Perception, Target Identification, and Toxicity Classification. Different from the traditional assessment methods that only focus on the final classification accuracy, this framework can distinguish the errors of the model at different stages. Through conditional toxicity analysis and visual representation analysis, it further reveals that the insufficient target identification ability is an important bottleneck affecting the toxicity detection.

Critical weaknesses
  1. Although the paper manually annotated the entire generated images for their visual visibility, the annotation process mainly focused on determining whether there were complete and perceptible hidden structures in the images, without further requiring the annotators to independently identify their specific contents. Since the targets generated by the diffusion model may undergo deformation or semantic distortion, a structure being visible to humans does not necessarily mean it can be accurately identified as the preset target. Therefore, directly using the specified targets at the time of generation as the Ground Truth for the recognition task may not fully reflect the true semantic content in the final image. The core finding of the paper is that the target recognition accuracy of VLM is only 1.1%–4.4%, and based on this, target recognition is regarded as the key bottleneck in toxicity detection. If the targets in some images cannot be accurately identified by humans, then the low accuracy of the model may partly result from the ambiguity of the data itself, rather than completely reflecting the deficiency in the model's capabilities. This directly affects the credibility of the core experimental results.
  2. The paper discovered through conditional toxicity analysis that when the model can correctly identify the hidden targets, its toxicity classification performance significantly improves. However, this result is mainly based on correlation analysis and cannot fully prove that the insufficient target recognition ability is the main reason for the failure of toxicity detection. For example, samples with clearer visual structures and more explicit semantics may be more easily identified and classified, thereby generating a strong correlation. Additionally, the three tasks are jointly completed in the same model response, lacking a control experiment that can independently control the target recognition information.
Questions for authors
  1. The experimental results show that the recognition accuracy of all models for Chinese text and gesture targets is generally lower than that for English text and symbols. Can the author evaluate the recognition ability of the model on the original target images without the interference of visual illusions to distinguish the original text and gesture recognition capabilities of the model from the additional difficulties caused by visual illusions?

  2. The paper adopts a unified Zero-shot Prompt, requiring the model to complete three tasks - illusion perception, target recognition, and toxicity classification - in a single response. Has the author considered evaluating the three tasks separately or adopting different prompt strategies? In particular, explicitly guiding the model to focus on the overall structure of the image, or providing auxiliary views that have been reduced in size or blurred, can this improve the performance of target recognition? This analysis helps to determine to what extent the current results are influenced by the evaluation strategy.

1 comment

I have read the author's rebuttal and have basically resolved my issue. I will keep my score.

Reviewer_neLEOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper uses SD1.5 and the QR Code Monster ControlNet to generate 14,940 visual illusion images. Targets fall into four types, namely English text, Chinese text, gestures, and symbols, with roughly half toxic and half non-toxic. Three annotators labeled each image with a human visibility level (V0/V1/V2). The authors split the evaluation into three steps, illusion perception, target identification, and toxicity classification, and test 8 VLMs. Target identification accuracy ranges from 1.1% to 4.4%. Toxicity predictions are much more accurate when the target is identified correctly. Encoder representations reflect the surface scene more than the target. From this, the authors argue that exact target identification is the bottleneck for toxicity detection.

Critical strengths
  1. The dataset includes non-toxic controls and keeps V0 images where no illusion formed. This makes it possible to separate "did not see the target" from "saw it but judged it wrongly". It is the most practical improvement over Qu et al. (2025b).

  2. The annotation process is careful. Annotators did not see the targets or generation settings, the auxiliary views use fixed settings, and agreement is reported for each target type.

Critical weaknesses

1. Limited novelty and missing reference. SemVink / HC-Bench (Li et al., EMNLP 2025) uses the same SD + ControlNet pipeline to test hidden content recognition. It also finds near-zero zero-shot accuracy for VLMs. More importantly, it shows that downscaling the image to 32 to 128 pixels raises accuracy above 99%. This paper does not cite it. Qu et al. (2025b) also already showed that vision encoders focus on surface details and miss hidden messages, and they already tested whether models can name the exact hidden message. Most of the main findings here have been reported before.

2. The bottleneck claim is not supported by the evidence. The title presents exact target identification as the bottleneck for toxicity detection. However, Section 4.3 only shows that correct identification and accurate toxicity labels tend to occur together. This is correlation, not causation. To show a bottleneck, an intervention is needed. For example, give the model the clean target image or the ground-truth target name. If toxicity accuracy then rises sharply, identification is the cause.

3. Gaps in the experimental design may underestimate identification accuracy. There is no baseline with downscaled or blurred inputs, so the failure may simply be a resolution issue. V1 targets are only visible to humans in auxiliary views, but models only receive the original image. The paper also does not explain how a correct identification is scored. Names such as "Parteiadler" or "Serbian Three" are hard to produce exactly even if the model sees the shape. Finally, annotators only labeled visibility, not what they saw, so there is no human identification baseline.

Questions for authors
  1. How do identification and toxicity results change if images are downscaled to 64 or 128 pixels, or if models receive the same auxiliary views used for V1?
  2. How is a correct target identification scored? Appendix C.1 describes how visibility and toxicity outputs are mapped, but not how identification is matched. What are the results under a looser matching rule?
  3. What toxicity macro-F1 do models reach when given the clean target image without the surface scene, or the ground-truth target name? Results by target type would be most useful, especially for gestures and symbols.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.