acceptodds
Under review as a conference paper at ICLR 2027

Explainable AI-Generated Image Detection with Multimodal Large Language Models

Abstract

Detecting AI-generated images has become increasingly challenging as modern generative models produce highly realistic content. While existing detectors achieve reasonable classification accuracy, they often suffer from limited interpretability and poor generalization to unseen generators, especially in real-world scenarios. Recently, multimodal large language models (MLLMs) have shown promise for explainable AI-generated image detection; however, their effectiveness is constrained by the lack of large-scale, high-quality explanatory datasets. In this work, we introduce Fake-Reason, a large-scale and high-quality dataset for AI-generated image detection with 80K grounded samples provide fine-grained artifact explanations and precise bounding boxes. Unlike datasets dominated by recent generators, Fake-Reason deliberately spans early GANs and diffusion models as well as modern generators. Based on Fake-Reason, we fine-tune a lightweight open-source MLLM and identify an off-region visual shortcut in standard supervised training: the model can rely on unrelated image regions to infer the global label and then fabricate a plausible explanation for the target region. To alleviate this problem, we introduce region-level attention supervision that aligns the attention of each explanation with its annotated bounding box. Extensive generalization experiments and real-world evaluations demonstrate that our resulting detector, RAFE, achieves state-of-the-art detection performance while providing accurate localization and human-verifiable explanations. We believe that Fake-Reason and the proposed training strategy offer a strong foundation for advancing explainable and robust AI-generated image detection.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
AI-generated Image Detection, MLLM

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Vy28OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces Fake-Reason, with 80K region-annotated images and 10K label-only samples, and RAFE, which adds region-level attention supervision to multimodal supervised fine-tuning. It aims to improve detection while grounding explanations in localized visual evidence.

Critical strengths
  1. The dataset design is thoughtful: real images also receive regional explanations, while modern samples without clear local artifacts receive label-only supervision. This reduces the risk of explanations becoming a fake-image-only output pattern.
  2. Detection results are competitive across diverse benchmarks. A same-backbone ablation reports a Synthbuster improvement from 92.8% to 98.3% with attention supervision, providing useful evidence for the auxiliary loss.
Critical weaknesses
  1. The central explanation and localization claims lack direct quantitative evaluation. Attention concentration does not establish factual correctness or dependence on the highlighted evidence. Dataset quality also lacks a quantitative audit; Figure 8 pairs a dog-region annotation with a date-stamp description, illustrating a relevant alignment error.
  2. Evaluation independence and reporting need clarification. Appendix B.4 reports hyperparameter selection using Synthbuster results without identifying a separate validation split, and GenImage training/test separation is insufficiently documented. Table 1's four Synthbuster entries average 98.3%, not the reported 94.6%; Tables 4/6 report 98.3%, whereas Table 7 repeats 94.6%. These issues complicate interpretation of the detection gains.
Questions for authors
  1. Can an independent audit compare explanation factuality, region alignment, and hallucination rates against ordinary SFT, including the mismatch in Figure 8?
  2. Were hyperparameters selected on separate validation data, how was GenImage overlap prevented, and which Synthbuster average is correct?
  3. Can matched cross-domain ablations separate attention supervision from the early/modern training-data composition?
  4. How does RAFE relate to FakeXplain [1], LEGION [2], LTE [3], and ReAlign [4]? Could suitable comparisons clarify detection generalization, and grounding or explanation quality where applicable?

References: [1] Ji et al. FakeXplain: AI-Generated Image Detection via Human-Aligned Grounded Reasoning. ICLR 2026. [2] Kang et al. LEGION: Learning to Ground and Explain for Synthetic Image Detection. ICCV 2025. [3] Ji et al. Locate-Then-Examine: Grounded Region Reasoning Improves Detection of AI-Generated Images. CVPR 2026. [4] Huang et al. ReAlign: Generalizable Image Forgery Detection via Reasoning-Aligned Representation. CVPR 2026.

Reviewer_W9BxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors introduce Fake-Reason, a balanced dataset of 90K real and generated images. Of these, 80K have descriptions and bounding boxes for regions that support the real or fake label. The remaining 10K have labels only and cover recent generators. Gemini produces the annotations, which are then checked by another model and manually filtered. The authors fine-tune Qwen3-VL-8B on this dataset to build RAFE. They add an attention loss to encourage each description to use the region it refers to. RAFE achieves strong detection results across several generator and social media benchmarks. The paper also shows examples of predicted boxes, explanations, and attention maps.

Critical strengths
  1. Fake-Reason provides descriptions and boxes for real images as well as fake ones. I also like the decision described in Section 3 to use labels alone for the 10K examples covering recent generators. A reliable local explanation may be difficult to give for these images, so it makes sense to avoid forcing one. The combination of descriptions, boxes, and authenticity labels makes this a useful dataset design for training detectors whose outputs people can inspect.

  2. In Table 4, the attention loss improves Synthbuster accuracy from 92.8% to 98.3% on the same Qwen3-8B backbone. This is a meaningful gain over ordinary SFT and gives direct support for the training method. Tables 1 and 2 also show strong results of 97.1% on New-Generator and 97.2% on WildRF. The range of external generators and social media images is a substantial strength. It shows that the method works beyond a single test distribution.

Critical weaknesses
  1. The evidence for explanation quality is thinner than the evidence for detection accuracy. Sections 5.5 and 5.6 give useful examples and show better attention alignment, but do not measure how often the descriptions are correct or the boxes identify the relevant artifacts. Appendix A describes model-assisted and manual filtering without reporting an independent quality check. A small human evaluation would help establish how reliable these outputs are. This limits the claims about explanation quality, although the detection gains remain useful on their own.

  2. I am less sure about the claim that mixing generator eras explains the generalization results. The earlier-generator examples and the extra 10K samples are not studied separately, and the MLLM baselines use different training data. These comparisons show the value of the full system, but leave the contribution of each data choice open. The attention ablation does isolate one useful part of the method. Appendix B.4 also needs clarification because it selects hyperparameters using Synthbuster without stating whether the final evaluation uses separate examples.

Questions for authors
  1. Please clarify which source splits were used for training and testing, and how duplicate images were excluded, especially for GenImage. Were the Synthbuster images used for hyperparameter selection separate from those used for final evaluation? Also, Table 1 reports a RAFE average of 94.6%, although its four entries average 98.3%, as reported in Tables 4 and 6. Which value is correct?

  2. Do you have results from an independent check of annotation or explanation quality? Even a small random sample with human judgments of the descriptions and boxes would help assess how often they are correct.

The data mixture ablations and tests of whether changing a described region changes the explanation would be useful follow-up work. I do not regard a full study of these questions as necessary to establish the detection contribution in this submission.

Reviewer_xSkSOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper introduces Fake-Reason, a 90K-sample dataset, and proposes a multimodal detection model RAFE based on regional attention supervision, enabling the model to simultaneously complete AI image detection, forgery area localization, and natural language explanation, while improving the model's generalization ability to unknown generators.

Critical strengths
  1. This paper proposed a technical solution for solving explainability detection through a regional attention supervision mechanism.
  2. Fake-Reason retained obvious structural anomalies generated by early generators and provided regional-level interpretive annotations, which helped the model learn forged features with semantic interpretability.
Critical weaknesses
  1. The experiment mainly measures the accuracy of binary classification, while the quality of explanation and region localization largely depend on qualitative cases and attention map visualization. Alignment of Attention with annotated regions does not necessarily mean that the model actually makes judgments based on the evidence in that region. The paper lacks independent experiments on causal verification, such as the accuracy of explanations, the IoU of localization, or region occlusion.
  2. The explanations for the 80K samples are mainly generated by Gemini, and then assisted by the model and manually filtered, but there is a lack of systematic evaluation of the consistency and accuracy of manual annotations and the contribution of attention supervision to the dataset composition, annotation quality, and individual contributions.
  3. In my opinion, Figure 2 clearly shows signs of AI synthesis, yet the author did not provide any explanation regarding the use of AI painting.
Questions for authors
  1. Does the paper's method enable the model to learn to make credible explanations based on local forged evidence, or does it merely make the attention distribution more in line with the manually specified area?
  2. Can the authors provide a detailed manual selection process, including annotation guidelines, the number of manual reviewers, specific criteria for identifying errors, rejection rates at each filtering stage, and accuracy assessment of bounding boxes?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.