- Rating
- 1
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper targets a critical limitation of existing Multimodal Large Language Models (MLLMs) in AI-generated image (AIGI) forensics: current MLLMs can effectively detect fake images via low-level pixel artifacts but fail to translate these subvisual, pixel-domain forgery traces into coherent, object-level natural language explanations.
To address this gap, the work proposes a complete and systematic forensic explanation pipeline consisting of four core components. First, it defines Subvisual Forgery Semantics (SFS), which reinterprets low-level pixel residuals, texture distortions, and frequency artifacts as semantic impacts on specific visual objects, rather than describing isolated pixel differences. Second, it designs the Semantic Differencing (SemDiff) paradigm, which derives forgery clues by parsing paired original and reconstructed images, extracting object-level pixel discrepancies, and structuring differences into standardized semantic descriptions with object names, visual phenomena, semantic impacts, and severity levels. Third, the paper constructsTRACE-Bench, a dedicated benchmark built on COCO images and three VAE reconstruction models, with multi-MLLM annotation and judgment-based quality control, to evaluate the low-level forgery semantic explanation capability of MLLMs. Fourth, it develops ForgeryTracer, a Qwen3-VL-8B-based MLLM optimized with dual text-residual token supervision and a two-stage training strategy, to simultaneously maintain low-level forgery perception and SFS generation ability.
Critical strengths
-
Novel and Practical Research Motivation: The paper addresses a genuine, under-explored pain point in MLLM-based image forensics. Most existing studies focus on binary fake/real classification or high-level semantic anomaly explanation, while ignoring the interpretability of low-level pixel-wise forgery artifacts. This work shifts the research focus from "forgery detection" to "explainable low-level forgery reasoning", which aligns well with the future development trend of trustworthy AI forensics and has significant research value.
-
Intuitive and Effective Core Idea (SemDiff): The proposed SemDiff paradigm provides a natural and elegant solution to the incompatibility between low-level pixel residuals and MLLM semantic reasoning. It successfully converts uninterpretable pixel-level differences into MLLM-friendly object-level semantic differences, enabling MLLMs to understand, summarize, and describe subvisual forgery artifacts through comparative analysis of original and reconstructed object pairs. This core design is the most insightful contribution of the work.
-
Complete Research System with Long-Term Benchmark Value: Different from most single-model forensic studies, this work builds a closed-loop research ecosystem: SFS definition → SemDiff extraction pipeline → TRACE-Bench evaluation platform → ForgeryTracer verification model. Among them, the manually curated and quality-controlled TRACE-Bench fills the gap of dedicated low-level forgery semantic explanation benchmarks, which can serve as a fundamental evaluation resource for subsequent related research and brings longer-term academic value than the ForgeryTracer model itself.
-
Competitive Experimental Performance: ForgeryTracer achieves state-of-the-art results on TRACE-Bench across three VAE reconstruction sources, with high SemF1 scores (83.5, 88.4, 87.8) and forgery detection accuracy (91.2%, 94.9%, 89.4%). It also obtains an average detection accuracy of 93.6% on the unseen AIGI-Now cross-generator dataset, demonstrating decent generalization ability against unknown generative models.
-
Convincing Ablation Study for Core Supervision Designs: The ablation experiments clearly verify the conflicts and coordination between semantic explanation learning and low-level forgery perception. The results prove that pure SFS semantic supervision severely degrades detection performance, while residual-token supervision can effectively restore and further boost low-level detection capability. This conclusion solidifies the rationality of the dual-supervision design and provides clear experimental support for the model training strategy.
Critical weaknesses
-
Inherently Unreliable SFS Ground Truth: The SFS semantic labels serving as ground truth are fully generated and filtered by MLLMs (via SemDiff + multi-MLLM judges) with only limited human spot-checks. This creates a closed-loop circular logic: MLLMs generate SFS labels → MLLMs are trained on these SFS labels → MLLMs evaluate SFS quality. Consequently, the experiment only proves that ForgeryTracer can well reproduce the MLLM-defined semantic descriptions, rather than truly perceiving and explaining real low-level forgery semantics. Human inspection cannot completely eliminate the inherent bias of MLLM-generated pseudo-labels.
-
Confusion Between Reconstruction Artifacts and Real Forgery Traces: The entire SemDiff pipeline relies on pixel residuals between original images and VAE-reconstructed images. However, VAE reconstruction itself inevitably introduces extra artifacts including texture blurring, frequency distortion, sharpness change, and color deviation. The extracted residuals thus mix real AIGI forgery artifacts and reconstruction-induced noise. Using three different VAEs cannot fundamentally eliminate this confounding factor, making it unclear whether the derived SFS semantics correspond to true forgery traces or model-specific reconstruction biases.
-
Subjective and Incomplete Definition of SFS: The paper defines SFS as the semantic degradation of visual objects caused by low-level forgery artifacts. This definition only covers artifacts that bring perceivable semantic impacts on human vision, while ignoring subtle low-level traces that do not affect object semantic perception. Therefore, the proposed SFS cannot represent the full spectrum of low-level forensic evidence, leading to incomplete forgery semantic modeling.
-
Limited Model Architecture Innovation: The technical innovation of this work is mainly reflected in the novel SFS semantic representation, SemDiff extraction pipeline, dual supervision strategy, and benchmark construction. The ForgeryTracer model is built based on the existing Qwen3-VL-8B backbone with conventional module modifications (DeepStack, token mask decoder). There is no breakthrough or novel design in model architecture, resulting in limited model-level innovation.
-
Insufficient Real-World Scenario Generalization: The TRACE-Bench dataset is constructed based only on COCO images and VAE reconstruction pipelines, which cannot fully simulate real-world AIGI scenarios. Actual AI image forgery includes diverse generation and editing modes such as diffusion-based generation, local inpainting, outpainting, and regional replacement, which produce completely different low-level artifact distributions. Cross-VAE generalization experiments cannot prove the model’s robustness against diverse real-world forgery mechanisms.
-
Loose and Inaccurate Evaluation Metrics for Forensic Explanation: The paper adopts BERTScore, SemAP, SemAR, and SemF1 to evaluate semantic explanation quality. However, these text similarity-based metrics cannot reflect forensic correctness. Semantically similar sentences may correspond to completely different forensic artifacts, and linguistically close descriptions can mask factual forensic errors. Although LLM-as-judge is supplemented, the evaluation system still lacks strict forensic-grounded criteria, failing to accurately judge whether the model’s explanation truly matches real forgery characteristics.
-
No Causal Proof of SFS’s Positive Impact on Detection Performance: The paper’s motivation implies that semanticization of low-level artifacts can improve forgery detection, but no experimental evidence supports this causal relationship. On the contrary, the ablation results show that pure SFS semantic supervision significantly reduces detection accuracy (from 81.4 to 67.6). The final performance improvement is entirely attributed to residual-token supervision, rather than SFS semantic learning. The work only verifies that semantic explanation and low-level detection can coexist, but fails to prove that SFS benefits detection tasks. The two capabilities are relatively decoupled, without verified causal interaction.
Questions for authors
Please refer to the weaknesses.