acceptodds
Under review as a conference paper at ICLR 2027

From Low-level Artifacts to Forgery Knowledge via Semantic Differencing

Abstract

Making forgery evidence explainable is fundamental to trustworthy AI-Generated Image (AIGI) detection, yet low-level forgery artifacts remain semantically elusive to Multimodal Large Language Models (MLLMs), which can hardly articulate such artifacts in language. We present the first study on semanticizing low-level forgery artifacts. At its core, we introduce Subvisual Forgery Semantics (SFS), describing how forgery traces below salient perception manifest as object-grounded semantic effects, and develop Semantic Differencing (SemDiff), a multi-agent pipeline that contrasts an image with its reconstruction and translates pixel residuals into structured semantic descriptions. With SemDiff, we construct TRACE-Bench, the first benchmark for assessing MLLMs on subvisual forgery semantics, curated from multi-source annotations over three reconstruction models, where state-of-the-art MLLMs struggle to describe them. As a strong baseline, we present ForgeryTracer, which preserves forgery-related low-level cues through residual-token supervision and learns to articulate them as SFS through semantic supervision, showing that SFS can be learned while maintaining detection accuracy. Our work makes low-level forgery evidence not only detectable but explainable, opening a low-level perspective for interpretable AIGI forensics. Data and models will be released.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
AIGI Detection, VLM

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FNF4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper targets a critical limitation of existing Multimodal Large Language Models (MLLMs) in AI-generated image (AIGI) forensics: current MLLMs can effectively detect fake images via low-level pixel artifacts but fail to translate these subvisual, pixel-domain forgery traces into coherent, object-level natural language explanations.

To address this gap, the work proposes a complete and systematic forensic explanation pipeline consisting of four core components. First, it defines Subvisual Forgery Semantics (SFS), which reinterprets low-level pixel residuals, texture distortions, and frequency artifacts as semantic impacts on specific visual objects, rather than describing isolated pixel differences. Second, it designs the Semantic Differencing (SemDiff) paradigm, which derives forgery clues by parsing paired original and reconstructed images, extracting object-level pixel discrepancies, and structuring differences into standardized semantic descriptions with object names, visual phenomena, semantic impacts, and severity levels. Third, the paper constructsTRACE-Bench, a dedicated benchmark built on COCO images and three VAE reconstruction models, with multi-MLLM annotation and judgment-based quality control, to evaluate the low-level forgery semantic explanation capability of MLLMs. Fourth, it develops ForgeryTracer, a Qwen3-VL-8B-based MLLM optimized with dual text-residual token supervision and a two-stage training strategy, to simultaneously maintain low-level forgery perception and SFS generation ability.

Critical strengths
  • Novel and Practical Research Motivation: The paper addresses a genuine, under-explored pain point in MLLM-based image forensics. Most existing studies focus on binary fake/real classification or high-level semantic anomaly explanation, while ignoring the interpretability of low-level pixel-wise forgery artifacts. This work shifts the research focus from "forgery detection" to "explainable low-level forgery reasoning", which aligns well with the future development trend of trustworthy AI forensics and has significant research value.

  • Intuitive and Effective Core Idea (SemDiff): The proposed SemDiff paradigm provides a natural and elegant solution to the incompatibility between low-level pixel residuals and MLLM semantic reasoning. It successfully converts uninterpretable pixel-level differences into MLLM-friendly object-level semantic differences, enabling MLLMs to understand, summarize, and describe subvisual forgery artifacts through comparative analysis of original and reconstructed object pairs. This core design is the most insightful contribution of the work.

  • Complete Research System with Long-Term Benchmark Value: Different from most single-model forensic studies, this work builds a closed-loop research ecosystem: SFS definition → SemDiff extraction pipeline → TRACE-Bench evaluation platform → ForgeryTracer verification model. Among them, the manually curated and quality-controlled TRACE-Bench fills the gap of dedicated low-level forgery semantic explanation benchmarks, which can serve as a fundamental evaluation resource for subsequent related research and brings longer-term academic value than the ForgeryTracer model itself.

  • Competitive Experimental Performance: ForgeryTracer achieves state-of-the-art results on TRACE-Bench across three VAE reconstruction sources, with high SemF1 scores (83.5, 88.4, 87.8) and forgery detection accuracy (91.2%, 94.9%, 89.4%). It also obtains an average detection accuracy of 93.6% on the unseen AIGI-Now cross-generator dataset, demonstrating decent generalization ability against unknown generative models.

  • Convincing Ablation Study for Core Supervision Designs: The ablation experiments clearly verify the conflicts and coordination between semantic explanation learning and low-level forgery perception. The results prove that pure SFS semantic supervision severely degrades detection performance, while residual-token supervision can effectively restore and further boost low-level detection capability. This conclusion solidifies the rationality of the dual-supervision design and provides clear experimental support for the model training strategy.

Critical weaknesses
  • Inherently Unreliable SFS Ground Truth: The SFS semantic labels serving as ground truth are fully generated and filtered by MLLMs (via SemDiff + multi-MLLM judges) with only limited human spot-checks. This creates a closed-loop circular logic: MLLMs generate SFS labels → MLLMs are trained on these SFS labels → MLLMs evaluate SFS quality. Consequently, the experiment only proves that ForgeryTracer can well reproduce the MLLM-defined semantic descriptions, rather than truly perceiving and explaining real low-level forgery semantics. Human inspection cannot completely eliminate the inherent bias of MLLM-generated pseudo-labels.

  • Confusion Between Reconstruction Artifacts and Real Forgery Traces: The entire SemDiff pipeline relies on pixel residuals between original images and VAE-reconstructed images. However, VAE reconstruction itself inevitably introduces extra artifacts including texture blurring, frequency distortion, sharpness change, and color deviation. The extracted residuals thus mix real AIGI forgery artifacts and reconstruction-induced noise. Using three different VAEs cannot fundamentally eliminate this confounding factor, making it unclear whether the derived SFS semantics correspond to true forgery traces or model-specific reconstruction biases.

  • Subjective and Incomplete Definition of SFS: The paper defines SFS as the semantic degradation of visual objects caused by low-level forgery artifacts. This definition only covers artifacts that bring perceivable semantic impacts on human vision, while ignoring subtle low-level traces that do not affect object semantic perception. Therefore, the proposed SFS cannot represent the full spectrum of low-level forensic evidence, leading to incomplete forgery semantic modeling.

  • Limited Model Architecture Innovation: The technical innovation of this work is mainly reflected in the novel SFS semantic representation, SemDiff extraction pipeline, dual supervision strategy, and benchmark construction. The ForgeryTracer model is built based on the existing Qwen3-VL-8B backbone with conventional module modifications (DeepStack, token mask decoder). There is no breakthrough or novel design in model architecture, resulting in limited model-level innovation.

  • Insufficient Real-World Scenario Generalization: The TRACE-Bench dataset is constructed based only on COCO images and VAE reconstruction pipelines, which cannot fully simulate real-world AIGI scenarios. Actual AI image forgery includes diverse generation and editing modes such as diffusion-based generation, local inpainting, outpainting, and regional replacement, which produce completely different low-level artifact distributions. Cross-VAE generalization experiments cannot prove the model’s robustness against diverse real-world forgery mechanisms.

  • Loose and Inaccurate Evaluation Metrics for Forensic Explanation: The paper adopts BERTScore, SemAP, SemAR, and SemF1 to evaluate semantic explanation quality. However, these text similarity-based metrics cannot reflect forensic correctness. Semantically similar sentences may correspond to completely different forensic artifacts, and linguistically close descriptions can mask factual forensic errors. Although LLM-as-judge is supplemented, the evaluation system still lacks strict forensic-grounded criteria, failing to accurately judge whether the model’s explanation truly matches real forgery characteristics.

  • No Causal Proof of SFS’s Positive Impact on Detection Performance: The paper’s motivation implies that semanticization of low-level artifacts can improve forgery detection, but no experimental evidence supports this causal relationship. On the contrary, the ablation results show that pure SFS semantic supervision significantly reduces detection accuracy (from 81.4 to 67.6). The final performance improvement is entirely attributed to residual-token supervision, rather than SFS semantic learning. The work only verifies that semantic explanation and low-level detection can coexist, but fails to prove that SFS benefits detection tasks. The two capabilities are relatively decoupled, without verified causal interaction.

Questions for authors

Please refer to the weaknesses.

Reviewer_EwfnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how to translate low-level forgery artifacts, which are difficult for multimodal large language models to perceive and describe, into natural-language explanations. It introduces Subvisual Forgery Semantics (SFS) to characterize how these artifacts manifest in specific objects and affect their semantic content. One key contribution is SemDiff, a multi-agent pipeline that converts pixel differences between original real images and their reconstructions into semantic annotations through object pairing, difference analysis, and structured formatting. Using COCO images and reconstructions from three VAEs, the authors construct TRACE-Bench. They further introduce ForgeryTracer, which combines textual semantic supervision with token-level supervision over high-residual regions in a two-stage training procedure, enabling the model to learn both authenticity classification and forgery artifact description.

Critical strengths

The problem appears to be real and is consistent with intuition. Current VLMs seem to have difficulty explaining subtle low-level forgery artifacts despite their strength in high-level semantic understanding. The ablation offers a related observation: adding semantic description supervision without residual-token guidance appears to reduce detection accuracy. Together, these findings suggest that connecting low-level visual evidence with natural-language descriptions remains challenging for VLMs.

Critical weaknesses

Weaknesses

  1. The reliability of VLM-generated annotations and explanations remains insufficiently validated. Providing reference images and object-level comparisons reasonably reduces the annotation difficulty. However, identifying a difference does not necessarily mean that the VLM can accurately explain the corresponding low-level artifact or its relevance to forgery. Since annotation and cross-checking mainly rely on VLMs, it remains unclear whether the checking agents can reliably identify errors in the generated descriptions, given the difficulty these models have with low-level artifacts. Independent validation would help establish that the pipeline produces explanations grounded in observable evidence.

  2. The validity and usefulness of the explanations without reference images are not sufficiently demonstrated. Descriptions derived from paired comparisons may be difficult to verify when only the query image is available. Since explaining low-level artifacts is a key contribution, more examples should demonstrate whether humans can locate and recognize the described low-level phenomena without a reference image, rather than relying on obvious high-level semantic anomalies. A human study would further clarify whether these explanations help readers assess detection results, particularly whether they help identify model errors or instead encourage acceptance of incorrect judgments. The current text-matching and model-based evaluations do not directly establish this aspect of interpretability.

Questions for authors

Questions

  1. How is annotation reliability independently validated, given that annotation and cross-checking mainly rely on VLMs? In particular, please clarify the relationship between the human spot-checks mentioned in the text and the “Human-simulated QC audit” in Table 8.

  2. In Table 4, semantic supervision without residual-token supervision reduces detection accuracy relative to Stage 1. The authors attribute this to semantic targets overwhelming the sparse binary signal. What evidence supports this explanation, and could the decline also reflect difficulty in learning low-level artifacts from textual descriptions alone?

  3. Could the authors disclose the detailed evaluation protocols, including evaluator inputs, prompts, and scoring criteria? In particular, do evaluators receive only the query image, or also reference images, residual maps, or ground-truth descriptions?

  4. Is there a human study showing whether the explanations help users verify low-level artifacts and assess detection results without reference images? In particular, do explanations help users identify model errors, or make them more likely to accept incorrect judgments?

  5. Could the authors provide more successful and unsuccessful examples, particularly for subtle low-level artifacts? Presenting the query image and explanation without a reference image would help clarify when the described evidence is verifiable and when the framework struggles to produce a reliable explanation.

Reviewer_kfu7OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how low-level artifacts in AI-generated images can be translated into interpretable semantic explanations. It introduces Subvisual Forgery Semantics (SFS), which describes subtle forgery traces through their effects on object-level visual properties. To construct such descriptions, the authors propose SemDiff, a multi-agent pipeline that compares real images with VAE reconstructions and generates structured semantic annotations. Based on this pipeline, they build TRACE-Bench and develop ForgeryTracer, which combines residual-token supervision with semantic instruction tuning. Experiments demonstrate improved semantic explanation scores and strong cross-generator detection performance.

Critical strengths
  1. The paper addresses an important limitation of explainable AIGI detection: existing MLLMs primarily reason about high-level semantic anomalies, while low-level forgery artifacts remain difficult to articulate. I think the motivation is clear and easy to follow.

  2. SemDiff provides a reasonable approach to bridging low-level visual differences and natural language explanations. Comparing original and reconstructed images allows MLLMs to leverage their existing visual comparison capabilities, and the multi-agent pipeline offers a scalable way to construct structured annotations.

  3. Experiments show that ForgeryTracer improves SemF1 from 72.7–73.2 to 83.5–88.4 across three TRACE-Bench sources (Table 1) and achieves 93.6% average detection accuracy on nine unseen generators (Table 3). Ablations further show that residual-token supervision improves detection accuracy while maintaining semantic explanation quality (Table 4).

Critical weaknesses
  1. SFS is constructed from differences between real images and VAE reconstructions. However, many identified artifacts, such as blurring and texture smoothing, can also arise from ordinary image processing. The paper does not sufficiently establish whether SFS captures forgery-specific evidence or general visual degradation, raising concerns about its validity and applicability.

  2. Insufficient evidence for faithful and generalizable explanations. Strong detection accuracy and semantic scores do not necessarily show that the generated explanations reflect the actual forgery evidence. Semantic evaluation relies mainly on VAE-based annotations, whose independent validation remains unclear (Table 8), while cross-generator experiments primarily assess detection rather than explanation quality (Tables 3 and 10). This leaves the paper's central claim of grounded forgery explanations insufficiently supported.

Questions for authors
  1. How do the authors distinguish SFS derived from VAE reconstructions from ordinary visual degradation, such as compression-induced blurring or texture loss? What evidence supports treating these differences as forgery-specific artifacts?

  2. Tables 3 and 10 demonstrate detection generalization, but do the generated SFS explanations remain accurate on unseen reconstruction models or generators? Is there direct evidence that these explanations correspond to the actual low-level visual artifacts?

  3. The paper mentions human validation, but Table 8 describes a "human-simulated QC audit" using an MLLM also involved in annotation filtering. Could the authors clarify whether independent human evaluation was conducted and how the reliability of TRACE-Bench ground truth was established?

  4. Tables 2 and 4 appear to swap the detection accuracies for SDXL and Qwen-Image. Additionally, the SDXL Stage 2 SemF1 in Table 4 appears inconsistent with its SemAP and SemAR. Could the authors verify these results?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.