acceptodds
Withdrawn from ICLR 2027

ForRAIL: Forensic Retrieval-Augmented Visual In-Context Learning for Training-Free Image Forgery Localization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Image forgery localization (IFL) has become increasingly challenging as modern image editing models produce realistic and semantically coherent manipulations, while existing methods often rely on task-specific training and struggle to generalize to unseen manipulation techniques and generators. Multimodal large language models (MLLMs) offer broad visual reasoning capabilities and cross-domain knowledge, yet their potential for precise, training-free forgery localization remains largely unexplored. In this work, we investigate whether a frozen MLLM can perform pixel-level IFL through visual in-context learning. We propose Retrieval-Augmented Visual In-Context Learning for IFL (ForRAIL), a training-free framework that retrieves informative forgery demonstrations for each query to guide localization without updating model parameters. To improve demonstration selection, we further introduce CoSFoR, a coarse-to-fine semantic–forgery retrieval strategy that jointly considers semantic relevance and manipulation-sensitive characteristics. Extensive experiments across diverse out-of-domain datasets, generators, and multiple MLLM backbones show that ForRAIL substantially outperforms vanilla MLLMs and existing training-free MLLM-based approaches and exceeds the best specialized trained forensic models 0.03 IoU on OOD average. It also maintains strong localization performance across different tampered-region sizes and under common image perturbations, including Gaussian blur, Gaussian noise, JPEG compression, and scale. These results establish retrieval-augmented visual in-context learning as a promising training-free paradigm for generalizable image forgery localization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Forgery Localization, Image Forensics, Multimodal Large Language Models, In-Context Learning, Retrieval-Augmented Generation, Training-Free Learning

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_hsCfOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This manuscript proposes a training-free image forgery localization method that integrates Retrieval-Augmented Visual In-Context Learning with existing LLMs. The proposed method first retrieves the most relevant examples from a pre-constructed offline example repository. It further introduces CoSFoR, a coarse-to-fine semantic–forgery retrieval strategy that jointly considers semantic relevance and manipulation-sensitive characteristics. Experimental results demonstrate the effectiveness of the proposed components.

Critical strengths

The proposed retrieval-augmented visual in-context learning method for image forgery localization is interesting and shows enhanced IFL performance.

Critical weaknesses

Methodology: The novelty of the proposed method is limited. The motivation for adopting a training-free LLM-based approach to IFL is not sufficiently elaborated. Furthermore, the rationale behind the designed components is not clearly explained. For example, it remains unclear why and how M_s is converted into a set of polygons. Similarly, the justification for selecting 1k instances from SID-Set and MagicBrush to construct the example pool is insufficient. The authors should also clarify the rationale for choosing GME as the feature extractor and explain the selection rationales of the residual maps. In addition, the notation r_{s|b} is not clearly defined. The claim that "d_{b,s} cancels the camera and compression baseline shared by all blocks" is overly strong and lacks supporting theoretical justification or empirical evidence.

Experiments: Although the proposed method is training-free, it introduces additional inference time and computational overhead. Therefore, experiments evaluating inference efficiency and computational cost are necessary. In Table 1, the training datasets used by the compared methods are not consistent, potentially affecting the fairness of the comparisons. Furthermore, additional unseen IFL datasets, such as NIST, CASIA, and Columbia, should be included to provide a more comprehensive evaluation of the generalizability of the proposed method.

Questions for authors

Please see the critical weaknesses.

Reviewer_HmwDOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ForRAIL, a training-free framework for image forgery localization that leverages visual in-context learning and query-specific demonstration retrieval. It introduces CoSFoR, a coarse-to-fine retrieval strategy that combines semantic similarity with manipulation-sensitive forensic features to select relevant demonstrations.

Critical strengths

The paper explores a training-free approach to image forgery localization by leveraging visual in-context learning and query-specific demonstration retrieval. Combining semantic similarity with manipulation-sensitive forensic features is a reasonable design choice.

Critical weaknesses

The effectiveness, key design choices, and generalization ability of the proposed retrieval strategy require stronger justification. Moreover, the main components are built upon existing techniques, and the paper lacks substantial methodological novelty. The overall presentation quality of the figures and tables is poor.

Questions for authors
  1. How sensitive is CoSFoR's performance to the different similarity components and their weights?
  2. The main contribution appears to lie in combining semantic similarity, manipulation-sensitive forensic features, and visual in-context learning. However, these components are based on existing techniques. What are the substantive novel contributions of ForRAIL and CoSFoR?
  3. The formatting and readability of some figures and tables need improvement. For example, some labels in Figures 1 and 2 are too small to read clearly, and the formatting styles are inconsistent across tables. Please improve the readability and formatting consistency of the figures and tables.
Reviewer_DpDUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ForRAIL to solve Image Forgery Localization in a training-free framework by reranking. Specifically, it proposes ForRAIL, which uses ground truth mask and original image to construct a library to record the details of an example of manipulation. Then it further designs CoSFoR for coarse-to-fine retrieval strategy. Extensive experiments shows impressive improvements compared with existing methods.

Critical strengths
  1. The model effectively transfers existing VLM to a forgery detection expert without training process.

  2. The ablation results prove that the design of CoSFoR is better for simply retrieving across the library.

Critical weaknesses
  1. The cost during inference. A complete inference costs nearly 1 minute for a 31B model, which limits the application in the reality.

  2. Another issue is that the robustness of this method with respect to real images remains unknown. The overall architecture does not appear to include any specific provisions or evaluations for handling such images.

  3. In essence, training-free actually rely on a reference library. The performance when faced with new generation techniques remains unknown. Existing out-of-distribution (OOD) evaluations fail to distinguish between the domain of the image content and the generation method, while the latter of which typically has a more significant impact on the reference library.

Questions for authors
  1. How about the performance on BR-Gen [1], which is a harder test set with more morden generation method and detailed manipulated. I wonder the OOD results, which prove that if the library is enough for a completely different domain.

  2. TCS employs residual energy statistics. These may reflect traces of editing, but they could also stem from compression, resolution, camera-specific processing, or variations in image content. The paper should more directly disentangle these factors through controlled experiments and report the impact of potential failures in residual extraction or differences in input pre-processing.

  3. Only a single backbone configuration was reported for the 60–70 second duration. There is no breakdown of the total cost, such as the respective contributions of retrieval, residual features, SAM2, example context length, API call fees, and retries upon failure.

What I'm most concerned about is the probability to use it in reality, which is more related with the inference cost and the robustness.

[1] Zooming In on Fakes: A Novel Dataset for Localized AI-Generated Image Detection with Forgery Amplification Approach

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.