- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper targets the core limitation of traditional Image Forgery Localization (IFL) methods, which heavily rely on task-specific training data and suffer from poor generalization to emerging and unseen forgery manipulations (e.g., GAN editing, diffusion inpainting, and new MLLM-based image tampering). The authors formally define the Zero-shot Image Forgery Localization (ZIFL) task, which aims to achieve pixel-level forgery localization on unknown manipulation types without any task-specific training or parameter fine-tuning.
To address this task, the paper proposes a Collaborative LEAder-worker discrimiNator (CLEAN) framework, which innovatively combines frozen pre-trained Multimodal Large Language Models (MLLMs) and LangSAM. The core insight is to convert traditional low-level forensic artifact detection into prior-violation analysis: leveraging MLLMs’ learned visual, semantic, and physical world priors to identify inconsistent and anomalous content in forged images, rather than relying on manually defined forgery traces.
The CLEAN framework consists of two core modules. First, the Coarse-Grained Suspect Exploration (CGSE) module generates structured suspicious region proposals via multi-dimensional forensic reasoning (semantic consistency, digital integrity, and physical-environment consistency), and adopts a context-aware bounding box expansion strategy to avoid missing forgery boundaries. Second, the Fine-Grained Iterative Refinement (FGIR) module verifies and optimizes initial proposals through confidence pre-judgment, leader-worker clue decomposition, LangSAM segmentation, and critic-guided iterative prompt refinement, finally outputting accurate pixel-level forgery masks.
Extensive experiments are conducted on 7 mainstream IFL benchmarks covering both traditional Photoshop-based forgery and AIGC-generated tampering, with six representative MLLMs adopted for evaluation. The results demonstrate that CLEAN achieves consistent and significant performance gains in IoU and F1 scores over vanilla MLLM baselines, and even outperforms some supervised training-based SOTA IFL methods. Ablation studies further validate the indispensable roles of the CGSE proposal generation and FGIR refinement modules, as well as the effectiveness of the three-dimensional forensic reasoning dimensions.
Critical strengths
-
Meaningful and practical problem formulation: The ZIFL task defined in this paper is of high research and application value. It effectively addresses the critical pain point of traditional supervised IFL models, i.e., severe generalization degradation when facing out-of-distribution and newly emerging forgery manipulations caused by the rapid iteration of generative image editing technologies. This problem definition is more forward-looking than incremental architectural improvements of existing IFL methods.
-
Reasonable and interpretable cross-model collaboration paradigm: The paper designs a clear logical division of labor between MLLMs and SAM-based segmentation models: MLLMs undertake high-level forensic reasoning, suspicious region mining, and clue verification, while LangSAM is responsible for precise pixel-level localization. Different from simple model stacking, the proposed pipeline forms a closed-loop system ofreasoning-evidence-segmentation-refinement, which effectively bridges the gap between open-ended visual anomaly reasoning and precise segmentation localization.
-
Innovative coarse-to-fine iterative refinement mechanism: The FGIR module is the core contribution of the system design. The leader-worker decomposition strategy successfully converts abstract forensic anomaly descriptions into concrete visual segmentation prompts, building a reliable semantic bridge between forensic evidence and visual entity segmentation. Moreover, the critic-guided boundary evolution mechanism iteratively optimizes segmentation masks by checking evidence completeness and boundary rationality, effectively solving the mismatch between natural object contours and real forgery boundaries.
-
Comprehensive and robust experimental validation: The experimental setup is rigorous and sufficient. The method is validated on six diverse MLLM backbones and seven benchmarks covering both traditional manual forgery and cutting-edge AIGC tampering scenarios. Consistent performance improvements across all settings prove that CLEAN’s effectiveness is not a backbone-specific prompt trick, reflecting good generalizability. In addition, detailed ablation experiments on core modules and forensic reasoning dimensions fully verify the rationality of each design component.
-
Insightful prior-violation analysis perspective: The core observation of replacing traditional low-level forensic artifact detection with MLLM prior-violation analysis is the most valuable conceptual insight of this work. It provides a novel paradigm for zero-shot IFL by exploiting the inherent visual, semantic and physical knowledge of pre-trained MLLMs, rather than relying on task-specific supervised learning.
Critical weaknesses
- Overly loose and controversial zero-shot definition: The paper claims a pure zero-shot setting, but the setting is essentiallytraining-free/task-adaptation-free rather than genuine zero-shot forgery localization. All adopted MLLMs and LangSAM are powerful pre-trained foundation models that have already absorbed massive internet data and potential knowledge of AIGC generation models involved in test datasets (e.g., GPT-4o, Diffusion models). The claim that the model has never seen unseen manipulations is not strictly proven, which overstates the technical contribution.
- Core performance heavily relies on manual prompt engineering: The so-called forensic reasoning capability of MLLMs is essentially driven by manually designed forensic prompt checklists (including explicit definitions of semantic, digital, and physical anomalies such as unnatural edges, texture discontinuity, and inconsistent lighting). The method injects prior forensic domain knowledge into MLLMs via handcrafted prompts, rather than mining inherent forensic awareness from MLLM pre-trained representations. This weakens the scientific novelty of the work.
- Limited algorithmic innovation (mostly agentic prompting pipeline): The proposed CLEAN framework has no trainable parameters or novel network architectures. The leader-worker decomposition, evidence verification, and critic refinement modules are essentially multi-stage, structured MLLM prompt orchestration and iterative inference. From the algorithmic novelty perspective, this work is a well-designed agentic reasoning pipeline rather than a novel learning-based IFL algorithm, which is a critical limitation for top conference acceptance. Unconvincing theoretical justification: The theoretical convergence analysis in the appendix is based on an unvalidated core assumption (0 < ρ < 1). The paper assumes that critic feedback can continuously reduce localization errors, but there is no experimental evidence to support this assumption. In practice, the critic mechanism may fail or even degrade mask quality due to incorrect prior evidence, making the theoretical guarantee conditional and superficial.
- Unresolved fundamental mechanism gap in forensic localization: The paper fails to explore the internal mechanism of how MLLMs perceive low-level forensic residuals (noise, edge, texture anomalies). As semantic-dominant visual models, MLLMs are inherently insensitive to signal-level subtle variations, and the improvement purely relies on customized prompts. There is a lack of feature-level, frequency-domain analysis and residual visualization experiments to verify the reliability of the transformation from low-level forensic residuals to high-level semantic anomalies.
- Cascading error propagation and confirmation bias risks: The entire pipeline suffers from severe error accumulation problems. Incorrect suspicious region proposals and wrong forensic clues generated in the CGSE stage will be continuously verified and amplified by the worker modules, with no independent error correction mechanism. The critic module only checks mask evidence coverage instead of re-judging the authenticity of forgery clues, which cannot rectify initial reasoning errors and leads to inherent confirmation bias.
Questions for authors
Please refer to the weaknesses.