acceptodds
Under review as a conference paper at ICLR 2027

ClaimEval: Atomic-Level Evaluation of Explanations for AI-Generated Image Forensics

Abstract

Recent work leverages MLLMs to build explainable AI-generated image detectors that output authenticity predictions and natural language explanations. However, these explanations are often unreliable—frequently generic, hallucinated, or forensically insufficient. Existing evaluation methods, including text similarity metrics, image-text similarity, and VLM-as-a-Judge, primarily produce holistic quality scores and cannot determine whether specific claims within an explanation are supported by visual evidence in the image, nor whether those claims constitute valid forensic evidence. We propose ClaimEval, a fine-grained evaluation framework that formulates explanation assessment as atomic-level forensic reasoning. Given an image-explanation pair, we decompose the explanation into atomic claims—the smallest independently verifiable semantic propositions—and evaluate each through a structured pipeline: language evaluation, claim grounding, evidence grouping, and forensic evidentiality assessment. We train a task-specific evaluator based on Qwen3-VL-8B with supervised fine-tuning followed by Structure-Guided GRPO (SG-GRPO). It produces structured claim-level judgments that are aggregated into an overall explanation score. To support this task, we construct a dataset with claim-level annotations and expert quality ratings. Experiments show that ClaimEval achieves substantially better alignment with human judgments than existing metrics. As a diagnostic tool, it reveals that classification accuracy does not imply explanation credibility, and serves as an effective inference-time reranker for improving detection reliability.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
AI-generated image forensics, Explainable forgery detection, Claim-level evaluation, Visual grounding, Multimodal large language models

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_utpbOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

Recent methods can detect AI-generated images and explain their decisions, but their explanations are not always supported by the image or based on meaningful forensic evidence. The paper introduces ClaimEval, which breaks explanations into the smallest independently verifiable atomic claims. It assesses language quality, checks whether the claims are supported by the image, groups related claims, and evaluates their value as forensic evidence. The authors build an expert-reviewed dataset and train an evaluator using supervised fine-tuning followed by Structure-Guided GRPO. Comparisons with the evaluated baselines show better agreement with expert judgments, while experiments removing individual reward components help establish their contributions.

Critical strengths
  1. I found the distinction between visual faithfulness and AI generation detection very useful. Grouping related claims also matter for combined evidence, dependencies and overlapping evidence.
  2. The experiments give convincing support for the approach. The authors compare against models following the same evaluation structure, check agreement with independent expert ratings, and test performance on unseen image and explanation sources. The reward ablations help show what the training components contribute.
Critical weaknesses

The main idea is clear, but I found myself going back and forth between the main text and appendix to follow some important details. The annotation rules, full reward definitions, and calculations behind the final explanation score are spread across the appendix. A compact summary of the key definitions and scoring rules in the main text would make the method easier to follow.

Questions for authors

Could you include a compact summary of the key annotation rules, training rewards, and final scoring calculations in the main text? A small table would help readers follow how these parts fit together without repeatedly referring to the appendix.

Reviewer_MQgxOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies explanation quality in explainable AI-generated image detection and proposes ClaimEval, an evaluation framework. ClaimEval decomposes an explanation into independently verifiable atomic claims and organizes related claims into evidence units. It then assesses whether the explanation contains obvious textual problems, such as mechanical repetition, whether each observation claim is grounded in the image, and whether each evidence unit provides sufficient support for the authenticity judgment. Claims concerning low-level image properties that cannot be reliably assessed through visual inspection alone are routed to tool-based verification. Experiments show that ClaimEval aligns more closely with human judgments of explanation quality than direct evaluation by general-purpose MLLMs and other baselines. Furthermore, using ClaimEval for Best-of-N reranking improves the detection performance of existing explainable detectors at inference time.

Critical strengths
  • Novelty: The paper addresses a longstanding gap in explainable AI-generated image detection by providing a reliable framework for evaluating explanation quality.
  • Comprehensive evaluation: The framework assesses explanation quality from multiple perspectives, including visual grounding and forensic evidentiality, providing a relatively comprehensive evaluation approach.
  • Substantial effort: The work includes a reasonably sized dataset with human involvement in its construction and review.

These strengths represent substantive contributions to the field.

Critical weaknesses

The overall system appears rather complex, and the necessity of this complexity is not adequately supported by module-level ablations of human alignment.

  • The paper does not clearly specify how tool-based verification selects reference images from the real-image gallery or which regions of those images are used. The construction and size of this gallery are also unspecified.
  • The paper lacks ablations that establish how individual evaluation modules contribute to alignment with human ratings. For example, how does Spearman correlation with human ratings change when tool-based verification, language evaluation (the Lᵢ factor), or claim grounding is removed? The reward-component ablations do not directly answer these questions.

These limitations reduce confidence in the framework’s reliability.

Questions for authors
  • Do you plan to publicly release the trained evaluator weights and the complete dataset?
  • Could you provide ablations of the evaluation modules, using metrics such as Spearman correlation with human expert ratings?
Reviewer_MTJYOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes ClaimEval, a framework for evaluating explanations from explainable AI-generated image detectors. It breaks each explanation into atomic claims and checks them for language errors, visual grounding, evidence grouping and forensic evidentiality. Claims that need low-level signal checks go to a statistical verifier, and all judgments are combined into a score. The evaluator is Qwen3-VL-8B, trained with LoRA SFT and then GRPO with a matching-based structural reward (SG-GRPO). It is trained on 6,150 image-explanation pairs whose annotations were proposed by models and refined by experts. It beats zero-shot MLLMs on structured metrics, aligns better with expert ratings (Spearman 0.587 vs. 0.366), and improves detection accuracy through Best-of-N reranking.

Critical strengths
  1. Separating visual grounding from forensic evidentiality is a useful idea, since a true observation is not always valid evidence of manipulation.

  1. The reward keeps unmatched predictions in every denominator and gates category credit on status agreement, which blocks easy reward hacking.

  1. The tool verifier abstains instead of guessing and is evaluated with coverage, selective precision and false-alarm rate together. The human reference is reliable, and the cross-domain design cleanly separates image, explanation-source and joint shift.

  1. Best-of-N reranking shows practical value beyond offline scoring.
Critical weaknesses
  1. DeepfakeJudge [1] and XPlainVerse (ACM-MM-26 Challenge [2]) are not referred to or compared. Both introduce protocols for reasoning evaluation in Deepfake detection. Also, XPlainVerse already decomposes forensic explanations into atomic evidence claims verified by an LLM judge, so the novelty is narrower than claimed.

  1. All baselines are zero-shot without schema-constrained decoding, and Ground Acc. appears tied to claim-splitting conventions (Claude scores 0.549 where always predicting SUPPORTED gives about 0.77).

  1. Table 2 omits Claude Opus 4.6, an SFT-only row and a no-tool row, even though the tool verifier touches at most about 1.3% of claims.

  1. GPT-5.4 and Gemini-3 Pro generate explanations, label them and serve as baselines; expert edit rates are not reported, and rater independence from annotation is unclear.

  1. The score uses hand-set constants with no recall term, so averaging penalizes corroborating weak evidence and missing the decisive artifact goes unpunished.

[1] Pixels Don't Lie (But Your Detector Might): Bootstrapping MLLM-as-a-Judge for Trustworthy Deepfake Detection and Reasoning Supervision (CVPR-2026) [2] Explainable Deepfake Detection Challenge (https://arxiv.org/pdf/2607.21007)

Questions for authors

Please check the weakness section.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.