- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes ToxicIllusion, a benchmark with 14,940 synthetic images with both toxic and non-toxic targets in common natural scenes, which includes English text, Chinese text, gestures, and symbols. It suggests a three-stage assessment scheme that tests illusion perception, accurate target identification, and toxicity classification separately and compares eight vision-language models based on different APIs and open-weight models to a single baseline with zero-shot prompting. The results indicate that identification of the target is very low even though it is perceived by models with accuracy ranging from 1.1-4.4%. Proper identification of the targets is that is related to a significantly higher toxicity-classification performance. Representation-similarity analysis and relevance maps also indicate that models are more concerned with features of the surface, as opposed to the embedded targets. The authors believe that one of the main challenges in AI-generated visual illusions is the lack of precise identification.
Critical strengths
[S1] Divides the process of a toxicity evaluation into perception, precise identification of the target and classification of toxicity, allowing for more precise error analysis.
[S2] Introduces a large data set of 14,940 images of toxic and non-toxic targets with good inter-annotator agreement k = 0.765 and also failed generations.
[S3] Assesses eight models that are based on APIs and open weighted, using task-specific metrics and supplemental representation analyses.
[S4] Recognises that detection of target recognition is an important bottleneck in the process of detecting visual-toxicity, and that this can be a limitation of end-to-end toxicity scores.
Critical weaknesses
[W1] For instance, the accuracy of the target identification depends on the ability to match the predicted target with the ground truth; how the alternative name, synonym, or variations in the gesture/symbol description will be addressed is not clearly stated in this paper. This could be punishing for semantically correct identifications that are incorrectly worded.
[W2] Toxicity classification is ‘gated’ by the model's prediction of the visibility, such that non-visible prediction is mapped to non-toxic. This is a mixture of perceptiveness and classification errors, which can make the overall toxicity score hard to interpret without the context of the individual error scores.
[W3] Lacks task-specific baselines such as OCR systems for text targets, but also task-specific gestures and symbols recognition. Confidence intervals or statistical tests would also be useful in determining the strength of the model comparisons reported.
[W4] The representation and relevance-map analyses would be improved by providing more detail in the methodology and reproducibility information. The analysis of local or multi-scale visual properties could also be used to further investigate whether the observed scene dominance extends beyond global representations.
[W5] The benchmark is based on synthetic images; this may restrict the findings of the benchmark to real-world visual illusions. The results would have improved validity if there was evaluation on a smaller, representative sample of real and/or photo edited examples.
[W6] While the paper references related multimodal safety research, a more robust evaluation of the proposed benchmark and the evaluated models may be possible by comparing these with other systems designed to detect harmful content (either implicit or visually embedded).
Questions for authors
[Q1] What is the method for scoring exact target identification for multilingual text, gestures and symbols? Are other names, different words, or slight variations of wording OK? Does Chinese text output work for Chinese character and pinyin? Would the reported results be different with less stringent matching criteria?
[Q2] Will the data set, the evaluation code, the structured prompts, output parsing scripts, and the generation of the relevance maps be made publicly available? Can the output of models be cached for API-based models for reproduction?
[Q3] Can you report toxicity classification based on ground truth visibility and existing protocol (model predicted visibility)? This comparison would facilitate the separation of the errors in illusion perception from the errors in toxicity classification.
[Q4] What is the sensitivity of the representation-similarity results to the visual encoder, target type and the degree of human visibility (V1 or V2)? These breakdowns would aid in the assessment of the claim of the dominance of surface-scene features in target representations.
[Q5] Could you share more detail of the approach to developing the spatial relevance maps and how the accuracy of the maps was determined? This would allow readers to judge more accurately the support of the claim for embedded targets versus surface scenes afforded by the visualisations.
I have read the author's rebuttal and have basically resolved my issue. I will keep my score.