ClaimEval: Atomic-Level Evaluation of Explanations for AI-Generated Image Forensics

Recent work leverages MLLMs to build explainable AI-generated image detectors that output authenticity predictions and natural language explanations. However, these explanations are often unreliable—frequently generic, hallucinated, or forensically insufficient. Existing evaluation methods, including text similarity metrics, image-text similarity, and VLM-as-a-Judge, primarily produce holistic quality scores and cannot determine whether specific claims within an explanation are supported by visual evidence in the image, nor whether those claims constitute valid forensic evidence. We propose ClaimEval, a fine-grained evaluation framework that formulates explanation assessment as atomic-level forensic reasoning. Given an image-explanation pair, we decompose the explanation into atomic claims—the smallest independently verifiable semantic propositions—and evaluate each through a structured pipeline: language evaluation, claim grounding, evidence grouping, and forensic evidentiality assessment. We train a task-specific evaluator based on Qwen3-VL-8B with supervised fine-tuning followed by Structure-Guided GRPO (SG-GRPO). It produces structured claim-level judgments that are aggregated into an overall explanation score. To support this task, we construct a dataset with claim-level annotations and expert quality ratings. Experiments show that ClaimEval achieves substantially better alignment with human judgments than existing metrics. As a diagnostic tool, it reveals that classification accuracy does not imply explanation credibility, and serves as an effective inference-time reranker for improving detection reliability.