acceptodds
Withdrawn from ICLR 2027

ForensicZoom: Adaptive Visual Inspection with Multimodal LLMs for Industrial-Grade Face Forgery Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reliable face forgery detection is critical to the security of online identity verification systems, where missed attacks compromise security and excessive false positives disrupt legitimate users. Specialized forensic detectors achieve strong detection performance but provide limited interpretability, while multimodal large language models (MLLMs) offer strong semantic understanding and interpretable reasoning yet remain substantially weaker for face forgery detection. We argue that a key limitation lies in how visual evidence is acquired: subtle forensic artifacts may be poorly represented at standard resolution, while uniformly processing all cases at higher resolution is computationally inefficient. We therefore introduce ForensicZoom, an industrial-grade MLLM framework for adaptive visual inspection. ForensicZoom first equips a general-purpose MLLM with forensic-aware visual representations and aligns the language model with these features. Its central mechanism, NEED_ZOOM, enables the model to autonomously request magnified views of suspicious regions when the initial evidence is insufficient, turning fixed-pass classification into adaptive multi-round forensic reasoning. The zoom behavior is learned through reward shaping that balances detection accuracy with unnecessary visual inspection, concentrating additional computation on difficult cases. A final attribution optimization stage improves natural-language forensic reports while preserving detection performance. On large-scale industrial identity verification data, ForensicZoom achieves over TPR at FPR, substantially outperforming both specialized detectors and existing MLLM-based methods while producing actionable forensic attributions. These results demonstrate that ForensicZoom can provide an effective path toward accurate, interpretable, and scalable MLLM-based face forgery detection. Our scripts and model parameters will be released upon acceptance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Face Forensics

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_zAJsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ForensicZoom, a multimodal large language model framework for industrial face-forgery detection. Starting from Qwen3-VL-8B-Instruct, it progressively (i) adapts the upper vision-encoder layers and adds a Forensic Query Bank, (ii) aligns visual representations with the language model through supervised fine-tuning, (iii) trains a PPO-based NEED_ZOOM policy that optionally triggers a second inference pass with an attention-guided enlarged crop, and (iv) refines natural-language attributions through cascade DPO/GSPO optimization. On a proprietary identity-verification dataset of approximately 2.05 million samples, the paper reports 97.3% TPR at 0.1% FPR on AIGC forgeries, compared with 91.0% for the strongest specialized baseline reported in the paper. It also discusses inference latency and an asynchronous deployment pipeline.

Critical strengths

<1>The paper targets identity-verification security rather than an unconstrained image classification task. The sizable industrial dataset, user-disjoint split, and emphasis on TPR at 0.1% FPR make the evaluation setting practically meaningful. The reported 97.3% TPR is promising, although the private-data setting limits independent verification. <2>The four-stage design connects task-specific perception, language alignment, adaptive inspection, and attribution optimization. Stage-wise ablations and the reported single-/two-pass latency and manual-review efficiency provide a useful account of the accuracy–inspection-cost trade-off, even though the causal benefits of the zoom policy remain to be isolated.

Critical weaknesses

<1>Stage 3 raises TPR from 93.7% to 96.9%, but simultaneously changes the learned policy, available visual evidence, and number of inference passes. The paper does not compare against a second pass using only the original image, random cropping, always zooming, MLLM-predicted regions, or confidence/entropy/logit-margin triggering. Consequently, it is unclear whether the gains arise from meaningful region selection and a learned when-to-zoom policy, rather than repeated inference or a simpler strategy. These omissions directly affect the paper’s central claim.

<2>The method uses the final-layer ViT attention map to choose a crop and aligns a text-derived spatial distribution to that map. Yet a concentrated attention map is not necessarily a faithful localization of manipulation evidence, and text–attention consistency alone does not guarantee factual grounding. The paper does not adequately establish its distinction from related active-visual-inspection and tool-use work, including V*, Active-o3, and ForgeryVCR. Furthermore, detection is evaluated only on proprietary data, without a public benchmark or convincing unseen-attack/temporal-shift evaluation; uncertainty estimates and threshold-calibration details at the stringent operating FPR are also missing. Together, these issues make both the methodological contribution and its generality difficult to assess.

Questions for authors

Q1. Is zooming itself necessary? With the same Stage 2 model and a matched inference budget, what are the results for (a) a second inference pass using the original image only, (b) a random 48×48 crop, (c) always applying attention-guided zoom, and (d) the proposed adaptive zoom policy?

Q2. Are the when/where policies preferable to simple alternatives? At matched zoom rates, how does PPO-based triggering compare with confidence-, entropy-, or logit-margin thresholds? How does attention-guided crop selection compare with directly having the MLLM predict the suspicious bounding box?

Q3. What validates the attention-based reference? What supports the assumption that the last-layer attention map corresponds to actual manipulation regions? In particular, what evidence shows that CoT Attention Alignment improves the factual grounding of the textual region description rather than simply making two model-generated signals agree?

Q4. How robust are the low-FPR results? Can the authors provide public-benchmark or held-out-generator/temporal-shift results, uncertainty estimates, and the precise protocol for selecting and fixing the 0.1%-FPR operating threshold?

Q5. How does the method differ from the closest prior approaches? Please identify the specific methodological improvements over V*, Active-o3, and especially ForgeryVCR, and position the work relative to relevant agentic-forensics and active-visual-search literature (e.g., AIFo, ForenAgent, DeepEyes, and Chain-of-Focus).

Reviewer_T3ApOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

1.The paper proposes ForensicZoom, an adaptive and interpretable face forgery detection framework that achieves over 97% TPR at 0.1% FPR in industrial deployment

Critical strengths

1..The method achieves state-of-the-art performance on large-scale industrial datasets

Critical weaknesses

1.Many components of the proposed method build upon existing research, such as the zooming strategy and techniques like PPO, DPO, and GSPO.The paper should clarify each component’s distinct role and how they interact, rather than merely combining seemingly reasonable components and reporting that the combination works.

2.The role of behavioral signals is unclear. If ForensicZoom uses them while the baselines do not, the comparison may be unfair, and ablations are needed to separate their contribution from that of the proposed method. If they are not essential, their absence does not justify omitting evaluation on public benchmarks.

Questions for authors

Please see the weaknesses

Reviewer_rsbpOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper presents ForensicZoom, a multimodal large language model (MLLM)-based framework for face forgery detection that aims to improve detection accuracy, interpretability, and computational efficiency through adaptive visual inspection. The framework consists of four training stages: forensic visual enhancement using a Forensic Query Bank and attention entropy regularization, cross-modal alignment through supervised fine-tuning, reinforcement learning-based adaptive inspection via the NEED_ZOOM mechanism, and Cascade RL combining DPO and GSPO to improve forensic attribution while preserving detection performance.The authors evaluate the proposed framework on a proprietary industrial identity verification dataset containing approximately 2.05 million samples collected over eight months. Experimental results show that ForensicZoom achieves a true positive rate (TPR) of 97.3% at a false positive rate (FPR) of 0.1%, with an AUC of 0.998, outperforming the reported specialized detectors and MLLM-based baselines. The paper further presents stage-wise ablation studies, evaluations of attribution quality based on human assessments, and deployment statistics to demonstrate the framework's potential for practical identity verification applications.

Critical strengths

None

Critical weaknesses
  1. Limited Methodological Novelty The methodological novelty of this work is insufficiently established. The proposed NEED_ZOOM mechanism relies on a special token to trigger additional visual inspection, with the decision policy optimized through reinforcement learning. However, adaptive visual inspection, active perception, and reinforcement learning-based tool use have already been explored in existing research on multimodal large language models. Similarly, the Forensic Query Bank, attention entropy regularization, supervised fine-tuning, and preference-based optimization largely build upon established techniques. The manuscript primarily combines these components into a unified framework, but does not clearly demonstrate a sufficiently distinctive methodological advance beyond this integration. Moreover, the relationship between the proposed design and existing active visual search approaches is not analyzed in sufficient depth. Without stronger evidence of conceptual novelty or a more fundamental methodological contribution, the claimed innovation appears limited.

  2. Insufficient Experimental Validation and Baseline Comparisons The experimental evaluation is insufficient to substantiate the paper's claims of superior performance and general applicability. All experiments are conducted on a proprietary industrial identity verification dataset, without comprehensive evaluation on established public face forgery detection benchmarks. Although several specialized detectors and MLLM-based methods are included as baselines, the experimental protocols, implementation details, and training configurations are not described sufficiently to establish the fairness and reproducibility of the comparisons. Furthermore, the evaluation does not adequately demonstrate generalization across different forgery generation methods, unseen attack types, or distinct data distributions. Given that generalization is a central challenge in face forgery detection, evaluation on a single private dataset is insufficient to establish the broader effectiveness of the proposed framework. More comprehensive benchmark evaluations and transparent baseline comparisons are necessary to support the reported improvements.

Questions for authors
  1. The paper emphasizes the importance of behavioral signals, such as interaction timing, repeated verification attempts, and device metadata. However, the proposed methodology primarily describes visual feature extraction and adaptive image inspection. How are these behavioral signals incorporated into the model and used for the final prediction? If they are not directly used, how should the claimed multimodal capabilities be interpreted?

  2. Since all reported experiments are conducted on a private industrial dataset, what evidence demonstrates that ForensicZoom generalizes to unseen forgery generation methods and different data distributions? Have the authors evaluated the model on any public face forgery detection benchmarks or conducted cross-domain evaluation? Such evidence could materially affect my assessment of the claimed generalizability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.