acceptodds
Under review as a conference paper at ICLR 2027

Parallel Hypothesis Decisions for Vision-Language Deepfake Face Detection

Abstract

Vision-language models (VLMs) are increasingly asked to judge whether a face image is real or forged, and the default recipe is to let them reason step by step before answering. We argue that authenticity judgment is a perceptual decision that is poorly served by slow, generative reasoning. With the same 4-billion-parameter VLM, the same forensic evidence, and the same images, reasoning for about 998 tokens before answering is not only slow but also inaccurate: on all six external benchmarks it scores below the forensic evidence it was given. We introduce PDM-Face, a detector that answers a panel of eight forensic hypotheses (diffusion, GAN, face swap, real photograph, abnormal spectral energy, up-sampling peaks, and two expert-evidence checks) in parallel and without generating a single token. The image and a compact set of evidence tokens from a gated forensic expert layer are encoded once; the key-value and recurrent states of this shared prefix are then copied into eight branches that are answered in one batched forward pass, and a learned fusion turns the eight three-way answers into a calibrated fake probability. Across six external face benchmarks, PDM-Face outperforms generative reasoning in all 12 comparisons (paired 95% confidence lower bound at least +0.051 AUROC) while being 21 times faster per image. It also significantly outperforms four recent detectors in 19 of 23 benchmark comparisons, improves over its own expert layer on the hardest video benchmarks (+0.034 on FaceForensics++ and +0.021 on DFDC, full test sets), and its parallel hypotheses beat a single-question decision layer on GenImage and 3rdBench for every seed and for a second VLM backbone.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
AI-generated image detection, vision-language models, parallel decision-making, frequency-domain analysis, multimodal learning, evidence verification, uncertainty calibration

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_n4fjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how vision-language models (VLMs) can use forensic evidence to make decisions in face forgery detection. The authors propose PDM-Face, which maps features extracted by forensic experts into continuous evidence tokens and feeds them into a VLM alongside the image. The model answers eight forensic hypotheses in parallel, then combines the answer probabilities with the expert scores to produce a final forgery probability.

Critical strengths

Considering generation mechanisms, spectral characteristics, and expert outputs as complementary sources of evidence and integrating them into a joint decision is a sensible design choice.

Critical weaknesses
  1. The need for the VLM decision layer remains unclear. The current baseline averages the five CNN predictions before combining them with CLIP. A simple router using CLIP features to dynamically weight individual experts could be more competitive. Without this comparison, it is unclear whether the VLM provides enough additional benefit to justify its computational cost.

  2. The performance drops and expert selection need better justification. PDM-Face performs worse than the expert-only baseline on 3rdBench and WildDeepfake, but the causes are not sufficiently explained. The rationale for selecting the five CNNs and CLIP is also unclear. These gaps make it difficult to assess the method’s reliability across different datasets and expert configurations.

Questions for authors
  1. Could you compare PDM-Face with a lightweight router that uses CLIP image features to dynamically weight individual expert predictions, rather than averaging the CNN predictions first? Please use the same experts and training data, control for access to evidence features and auxiliary supervision, and report both AUROC and end-to-end inference cost. This would help establish the additional value of the VLM decision layer.

  2. What criteria guided the selection of the five CNNs and the CLIP classifier, and how complementary are they? Why average the CNN predictions before gating and require the CNN ensemble to receive at least 50% of the weight?

  3. Why does PDM-Face underperform the expert-only baseline on 3rdBench and WildDeepfake? Are these drops consistent across seeds and statistically significant? Please analyze when the decision layer worsens expert predictions and whether the hypothesis branches or final fusion contribute to these errors.

3 comments
AuthorsOpenReview ↗

Thanks for the reviewer's precise and constructive suggestions (Part 1/2)

We thank the reviewer for a precise and constructive review. The review does more than ask whether the VLM is needed. It specifies the experiment that answers the question: the same experts, the same training data, controlled access to evidence features and auxiliary supervision, and both AUROC and cost. We ran this experiment during the discussion period and report it below.

W1 / Q1: We agree that this comparison was missing. We have now run it as a new experiment.

Setup. All baselines train on the same 60,000 faces as PDM-Face. They read the identical 1,315-dimensional evidence vector that feeds PDM-Face's evidence tokens: the CNN features, the CLIP features, the radial spectrum, and the expert scalars. They use the same validation-based early stopping and temperature fitting. Each arm has three seeds, and the model is selected on validation data only. The four arms are:

  • CLIP router. It reads the CLIP image features and dynamically weights the expert predictions, with no 50% floor.
  • Logistic regression on the evidence vector.
  • MLP. A two-layer MLP on the evidence vector.
  • Multi-task MLP. The MLP plus the same eight hypothesis heads, with the same labels and the same loss weight as PDM-Face. This arm controls for auxiliary supervision.

Results. The multi-task MLP was the best arm on validation. The table gives the AUROC of PDM-Face minus this arm on identical images, with paired 95% confidence intervals (seed 0 of each).

BenchmarkPDM-Face minus multi-task MLP95% CIOutcome
DFDC+0.019[+0.005, +0.032]significant
FF+++0.021[+0.011, +0.032]significant
WildDeepfake+0.034[+0.022, +0.046]significant
GenImage+0.003includes zerotied
3rdBench-0.001includes zerotied

Against the CLIP router, PDM-Face is significantly better on five of six benchmarks, for example GenImage (+0.090), FF++ (+0.063), DFDC (+0.037), and WildDeepfake (+0.012). The two are tied on 3rdBench.

Cost. All arms and PDM-Face share the same expert feature extraction, so the difference lies in the decision stage alone. The light fusion heads take under 0.1 ms per image. The VLM decision layer takes 11.7 ms per image (batched, with prefix-state branching, on one H100).

Reading. With the experts, the training data, the evidence features, and the auxiliary supervision all held fixed, the VLM decision layer still adds a significant gain on DFDC, FF++, and WildDeepfake. These are the three benchmarks where the expert layer is weakest (0.622, 0.622, and 0.749 AUROC). The evidence is ambiguous there, and the VLM reads the image jointly with it. On GenImage and 3rdBench, where the experts are already strong, a light head recovers the same accuracy. The added value of the VLM is therefore concentrated on the hard cases, at a cost of 11.7 ms per image. The decision layer also returns the eight hypothesis answers for each image as an auditable readout.

We will add this experiment to the revision (appendix), with all four arms, all seeds, and the cost table, and we will summarize it in the main text.

AuthorsOpenReview ↗

Thanks for the reviewer's precise and constructive suggestions (Part 2/2)

W2 / Q2: Selection criterion. The two experts were chosen for complementary strengths. The CNN ensemble is a low-level forensic expert. It consists of five four-branch networks with an EfficientNetV2-S backbone, and each network fuses RGB, spectral, residual, and self-blending streams. The logistic CLIP ViT-L/14 probe is a semantic expert. A gate reads the CLIP features and weights the two.

Complementarity. Each expert leads on different benchmarks (AUROC):

ExpertIn-domain3rdBenchGenImageFF++DFDCWildDeepfake
CNN ensemble0.9960.9640.7410.5750.5710.724
CLIP probe0.9570.8400.8540.6600.6290.636

The Spearman rank correlation between the scores of the two experts on the external benchmarks is 0.15 to 0.56. The two experts therefore make different errors, which is the condition under which gating helps.

Why the five CNNs are averaged before gating. The five CNNs share the same four-branch architecture, so they read the same kind of evidence. Averaging them, with flip test-time augmentation and per-model temperatures, is standard ensembling that reduces variance before gating. The gate then weighs two different kinds of evidence, forensic and semantic.

Why the 50% floor. The CNN ensemble is the stronger in-domain detector (0.996 against 0.957). The gate reads CLIP features, and these features shift on unseen sources. The floor keeps the forensic CNN in every decision when that happens.

We will add the per-expert AUROC table and a gate-floor ablation to the appendix.

W2 / Q3: Consistency and significance. On 3rdBench the difference is consistent and small. PDM-Face minus the expert layer is -0.011, -0.013, and -0.012 for the three seeds, and it is significant (95% CI [-0.013, -0.008] for seed 0). On WildDeepfake the difference is not consistent. It is -0.005 and -0.002 for two seeds (not significant) and -0.026 for the third. On the full WildDeepfake test set the two are statistically tied (0.742 against 0.746, 95% CI [-0.009, +0.002]).

When the decision layer changes the expert ranking. We ran a new per-image analysis on 3rdBench.

  • By source: on the text-to-image subset both reach 1.000. The small difference comes from the StyleGAN set and the two face-swap collections.
  • By expert confidence: 76% to 85% of the difference comes from the half of the images where the expert layer is least confident. On the images where the expert layer is confident, PDM-Face leaves the ranking essentially unchanged.

The decision layer therefore acts mostly where the experts are uncertain. This is where it gains on FF++ and DFDC (+0.034 and +0.021 on the full test sets), which contain compressed video, face swaps, and reenactment. It is also where it costs about 0.01 on 3rdBench, where the expert layer is already at 0.977 and has little room to improve.

Branches or fusion. The difference does not originate in the hypothesis branches or in the learned fusion. A decision layer with a single question and no fusion is lower on 3rdBench (0.948). The eight parallel hypotheses raise this to 0.965. Replacing the learned fusion with a plain average of the hypothesis answers lowers 3rdBench by 0.015. Both components therefore move the decision toward the expert layer. The remaining difference comes from the re-ranking of low-confidence images described above.

We will add the per-seed intervals and the per-source and per-confidence analysis to the appendix, and we will report both findings in the main text.

We thank the reviewer again. The proposed router experiment and the request for a failure analysis gave the paper two results that it lacked, and both will appear in the revision.

AuthorsOpenReview ↗

Additional results on the expert layer: gate floor and the complete fusion table

This note adds to our earlier response. The gate-floor ablation that we promised there is finished, and the fusion comparison is now run at every seed, which updates the single-seed test given there. Both are in the revised PDF.

Gate floor. The gate gives the CNN ensemble the weight g = c + (1 - c) * sigmoid(MLP(CLIP feature)), so c is the minimum weight of the CNN ensemble. We retrained the gate with c = 0 (no floor), c = 0.25, and c = 0.5 on the 60,000-face training pool, with three seeds each, and evaluated the expert layer alone.

Gate (AUROC, mean of 3 seeds)Celeb-DF3rdBenchGenImageDFDCFF++WildDeepfakeMean
c = 0 (no floor)0.8380.9680.7650.5870.5870.7250.745
c = 0.250.8570.9710.7980.5890.5950.7340.757
c = 0.50.8540.9720.7950.5950.5970.7370.758

With either floor, the three-seed mean is at least as high as without a floor on every external benchmark, and the six-benchmark mean rises from 0.745 to 0.757 and 0.758. In the paired test at seed 0 (last rows of Table 11), the gate with c = 0.5 is significantly higher than the gate without a floor on Celeb-DF, 3rdBench, GenImage, and WildDeepfake (by +0.005 to +0.027), and the two are tied on DFDC and FF++. These gates are a re-implementation of the gate, trained with the recipe of the light fusion heads.

Complementarity within each class. The rank correlation in our earlier response was computed over all images, so it also contains the agreement of the two experts on the label itself. Among real faces alone and among forged faces alone, the Spearman correlation between the two expert scores is 0.01 to 0.36 on the six external benchmarks, so their evidence is largely non-redundant. Each expert alone also leads on different benchmarks: the CNN ensemble on Celeb-DF (0.838 against 0.794), 3rdBench, and WildDeepfake, and the CLIP probe on GenImage, DFDC, and FF++. Each of these six differences is significant in a paired test.

Fusion table. Table 3 of the revision now gives all four heads on all six benchmarks as three-seed means, with a paired test at every seed: against the selected head (the multi-task MLP of our first response), PDM-Face is higher on DFDC, FF++, and WildDeepfake (significant at two of three seeds each) and lower on Celeb-DF (three seeds), 3rdBench (two), and GenImage (one). The decision layer is the stronger of the two where the expert evidence is weakest, at a cost of 11.7 ms per image, and one forward pass also returns its per-hypothesis, auditable outputs.

In the revised PDF, the fusion heads are in Table 3 and Appendix I, the experts and the gate floor are in Appendix E (Tables 10 and 11), and the per-seed and per-confidence analysis is in Appendix F (Tables 12 to 16).

We are glad to clarify any of this.

Reviewer_WDVrOpenReview ↗
Rating
2(+1)
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes PDM-Face, a vision-language-based face forgery detector that replaces autoregressive reasoning with parallel hypothesis classification. The framework first combines an ensemble of forensic CNNs and a CLIP-based detector to extract forensic evidence. A VLM then processes the image and projected evidence tokens, answering eight predefined hypotheses concerning generation mechanisms, spectral artifacts, and expert predictions. The hypothesis outputs are combined through a learned fusion module to produce the final authenticity score.

The authors also introduce a shared-prefix inference strategy that reuses cached states across hypothesis branches to reduce computational overhead. Experiments on six external benchmarks report improvements over a generative reasoning baseline and several existing forgery detectors.The paper argues that fast, non-generative decisions are more appropriate than explicit reasoning for face forgery detection.

Critical strengths
  1. Efficient inference through shared-prefix computation.

The proposed implementation reuses cached representations across multiple hypothesis branches, avoiding redundant computation and autoregressive text generation. The reported latency reductions are practically useful, and the implementation considers the recurrent states of hybrid VLM architectures.

  1. An attempt to examine the role of explicit reasoning in forensic detection.

Whether generative reasoning improves authenticity judgments is an interesting research question. The paper provides controlled comparisons between single-query and multi-hypothesis decision layers, along with evaluations across multiple benchmarks and model configurations.

Critical weaknesses
  1. The central claim about generative reasoning is not supported by a sufficiently fair comparison.

The paper concludes that generative reasoning is an inappropriate decision mode for face forgery detection. However, the comparison underlying this claim is substantially imbalanced.

The generative reasoning baseline uses Qwen3.5-4B in its native thinking mode with a manually designed forensic prompt and numerical expert scores provided as text. In contrast, PDM-Face benefits from task-specific LoRA fine-tuning, continuous forensic evidence tokens, supervised hypothesis predictions, and a learned fusion module.

Consequently, the observed performance differences cannot be attributed specifically to the presence or absence of generative reasoning. They may instead reflect differences in task adaptation, evidence representation, and supervised optimization.

A meaningful comparison would require generative and non-generative variants trained with comparable data, forensic evidence, and optimization budgets. Without such controls, the experiments do not substantiate the paper's central claim that reasoning itself degrades forensic detection performance.

  1. The motivation for using a large VLM is insufficiently established.

The proposed method ultimately performs supervised multi-task classification over eight predefined hypotheses, followed by a lightweight learned fusion. Several hypotheses are directly derived from generator labels, spectral statistics, or existing expert predictions.

Moreover, the forensic expert layer already provides strong detection performance, while removing its evidence causes the VLM-based detector to deteriorate substantially. The improvements over the expert layer are relatively modest and inconsistent across benchmarks.

It therefore remains unclear why a multi-billion-parameter VLM is necessary for this task. The paper does not provide sufficiently strong comparisons with simpler alternatives, such as MLP-based fusion, lightweight feature-level classifiers, or multi-task heads operating directly on the same expert features.

The proposed system may be an effective feature-fusion architecture, but the experiments do not establish that its performance depends on the vision-language reasoning capabilities emphasized in the paper.

  1. Insufficient transparency of training data and limited baseline comparability.

The training corpus is described as a curated collection of real faces and forgeries from diffusion, GAN, face-swapping, and reenactment pipelines. However, the paper does not provide a sufficiently detailed breakdown of the source datasets, generators, sample distributions, or construction procedures.

This limits reproducibility and makes it difficult to assess the claimed cross-generator generalization. The use of quality filtering and identity-aware splitting is reasonable, but these procedures alone do not establish that content, source, and generation-pipeline confounds have been adequately controlled.

The comparison with existing detectors is also limited. NPR and AIDE are retrained on the authors' corpus, whereas FatFormer and RINE use released checkpoints, introducing differences in training distributions and protocols. Furthermore, the main comparison excludes several recent specialized MLLM-based forensic detectors that would be more directly relevant to the paper's motivation.

As a result, the current experimental setting does not provide sufficiently convincing evidence that the proposed decision framework offers a generalizable advantage over established alternatives.

Questions for authors

Fair comparison with generative reasoning: Could the authors evaluate a generative reasoning baseline with task-specific fine-tuning, equivalent forensic evidence, and a comparable training budget? How can the reported performance gap be attributed to reasoning rather than differences in supervision and representation?

Necessity of the VLM: Could the authors compare PDM-Face with simpler discriminative fusion architectures trained on the same CNN, CLIP, and spectral features? What evidence demonstrates that the VLM contributes capabilities beyond conventional supervised feature fusion?

Training data transparency: Could the authors provide the complete composition of the curated training corpus, including dataset sources, generator families, sample counts, and real/fake matching procedures? How is overlap or source correlation between training and external benchmarks assessed?

6 comments
AuthorsOpenReview ↗

Thanks for the reviewer's thorough and principled critique (Part 1/3)

We appreciate the reviewer's thorough and principled critique. The review separates three factors that our comparison had mixed: task adaptation, evidence representation, and supervised optimization. It also sets a clear standard for data transparency and for baseline comparability. We have followed this structure. Below we narrow the central claim, report new experiments, and list the concrete changes to the paper.

W1 / Q1: We agree with the reviewer. The reasoning baseline is zero-shot, and PDM-Face is trained. The gap between them cannot be attributed to generative reasoning alone, and the paper should not claim that reasoning itself degrades detection.

Revised claim. The revision states the result as follows: with the same backbone and the same evidence, a trained parallel decision layer is more accurate than zero-shot generative reasoning, and about 21 times faster per image. We will rewrite the abstract, the contribution list, the title of Section 5.1, the discussion, and the conclusion to match. The phrase "the wrong decision mode" will be removed.

What the current experiments establish. Three results hold under the narrower claim.

  1. Reasoning loses evidence that it is given. With the expert scores in its prompt, the reasoning model ends below those scores on all six benchmarks. For example, 3rdBench drops from 0.983 to 0.864, and Celeb-DF drops from 0.834 to 0.688. This compares the input and the output of the same model, and it involves no trained decision layer. PDM-Face reaches 0.860 on the same Celeb-DF images.
  2. The gap is not a truncation artifact (new analysis). In the paper, 26.9% of the traces reach the token cap. When we keep only the traces that finished below the cap, reasoning with expert scores remains below PDM-Face on every benchmark. The paired 95% confidence intervals of the difference are: Celeb-DF [+0.216, +0.324], 3rdBench [+0.102, +0.155], GenImage [+0.120, +0.190], DFDC [+0.122, +0.223], FF++ [+0.063, +0.157], and WildDeepfake [+0.092, +0.189].
  3. The cost difference is structural. Reasoning writes about 998 tokens per image and takes 2.89 s. PDM-Face generates no token and takes 0.137 s.

Supervision and representation, controlled within trained models. The paper contains one comparison in which everything except the decision layer is identical: the backbone, the data, the evidence tokens, the evidence dropout, and the schedule. This is the single-query decision layer against PDM-Face. There the parallel panel gains +0.040 on GenImage, +0.017 on 3rdBench, and +0.013 on DFDC, each for all three seeds. This isolates the effect of the parallel hypotheses from the effect of training.

Fine-tuned generative reasoning. A reasoning model trained with comparable data, evidence, and budget is a different and valuable study. The revision will present it as complementary future work, and it will make no claim about it. We will add a discussion of RD-MLDG (Xu et al., ICLR 2026, "Reasoning-Driven Multimodal LLMs for Domain Generalization"). That work reports that fine-tuning MLLMs on reasoning chains for classification is harder to optimize than direct label supervision, and that a direct classification path is needed to stabilize it. This places our direct decision layer in context, and it shows what a fair trained comparison has to control.

AuthorsOpenReview ↗

Thanks for the reviewer's thorough and principled critique (Part 2/3)

W2 / Q2: We agree that the paper lacked this comparison. We have now run it as a new experiment, with the three alternatives named in the review: MLP-based fusion, light feature-level classifiers, and multi-task heads on the same expert features.

Setup. Every baseline trains on the same 60,000 faces as PDM-Face. Every baseline reads the identical 1,315-dimensional evidence vector that feeds PDM-Face's evidence tokens: the CNN features, the CLIP features, the radial spectrum, and the expert scalars. Early stopping and temperature fitting use the same validation procedure. Each arm has three seeds and is selected on validation data only.

  • Logistic regression on the evidence vector (light feature-level classifier).
  • MLP. A two-layer MLP on the evidence vector (MLP-based fusion).
  • Multi-task MLP. The MLP plus the same eight hypothesis heads, with the same labels and the same loss weight as PDM-Face (multi-task heads, with auxiliary supervision controlled).
  • CLIP router. It reads the CLIP image features and dynamically weights the expert predictions.

Results. The multi-task MLP was the best arm on validation. The table gives the AUROC of PDM-Face minus this arm on identical images, with paired 95% confidence intervals (seed 0 of each).

BenchmarkPDM-Face minus multi-task MLP95% CIOutcome
DFDC+0.019[+0.005, +0.032]significant
FF+++0.021[+0.011, +0.032]significant
WildDeepfake+0.034[+0.022, +0.046]significant
GenImage+0.003includes zerotied
3rdBench-0.001includes zerotied

Against the CLIP router, PDM-Face is significantly better on five of six benchmarks, for example GenImage (+0.090), FF++ (+0.063), DFDC (+0.037), and WildDeepfake (+0.012). The two are tied on 3rdBench.

What the VLM contributes. The ablation in the paper shows that the VLM needs the forensic evidence. The new experiment shows the converse. On DFDC, FF++, and WildDeepfake, the same evidence with the same multi-task supervision is significantly less accurate without the VLM. These are the three benchmarks where the expert layer is weakest (0.622, 0.622, and 0.749 AUROC). The evidence is ambiguous there, and the VLM reads the image jointly with it. On GenImage and 3rdBench, where the experts are strong, conventional fusion recovers the same accuracy. The revision will state this plainly: the contribution of the VLM is concentrated on the hardest benchmarks.

Cost and model size. All arms share the same expert feature extraction. The light heads take under 0.1 ms per image, and the VLM decision layer takes 11.7 ms per image (batched, on one H100). A larger backbone is not required. With Qwen3.5-2B, PDM-Face stays close to the 4B model: 3rdBench 0.959 against 0.966, GenImage 0.844 against 0.851, DFDC 0.648 against 0.637, and FF++ 0.671 against 0.659.

Wording. We also agree that the gain should not be attributed to vision-language reasoning. PDM-Face generates no text. The claim of the revision is that a VLM used as a fast discriminative decision layer adds value over conventional fusion on hard benchmarks. The full experiment (all arms, all seeds, cost) will be added to the appendix and summarized in the main text.

AuthorsOpenReview ↗

Thanks for the reviewer's thorough and principled critique (Part 3/3)

W3 / Q3: Composition of the training corpus. All faces come from public collections.

  • Real faces: OpenFake (real), FFHQ, FairFace, CelebA-HQ, DF40, DeepFakeFace (real), and UTKFace.
  • Forged faces: OpenFake, PromptHero, OneMillionFaces, SFHQ, DeepFakeFace, DF40, and YouMind.

The decision layer trains on 60,000 faces: 30,000 real and 30,000 forged. The forged faces span 23 generator families. The largest are:

FamilyFacesFamilyFaces
StyleGAN4,957DALL-E1,470
Stable Diffusion4,631Ideogram1,451
Flux3,124Grok1,376
Face swap1,734Mystic1,374
GPT-Image1,639Imagen1,366
Nano Banana1,489Midjourney1,335
HiDream1,482Seedream1,323

The remaining families are smaller.

Construction and matching. Forged faces are spread evenly over the generator families, and each family quota is capped by availability. Real and forged faces pass identical quality gates (detection confidence, native resolution, sharpness, pose, and CLIP faceness), so image quality carries no label information. Splits are group-aware, so one identity never appears in two splits. The revision will add a full composition table to the appendix, with source, generator family, counts, and the matching procedure.

Overlap with the external benchmarks (new experiment). We checked this at two levels.

  1. Files: there is zero file-path overlap between the training faces and the external test images.
  2. Content: we ran a CLIP ViT-L/14 nearest-neighbor search of the external test images against all 60,000 training faces. The fraction of test images with a cosine similarity of at least 0.95 to any training image is 0% on Celeb-DF, GenImage, DFDC, FF++, and WildDeepfake. It is 1.1% on 3rdBench (0.03% at a threshold of 0.98). For reference, the same fraction is 2.6% for our in-domain held-out test set.

By this measure, the external benchmarks are farther from the training faces than our own held-out split. We will add this check to the appendix.

Baseline protocols. NPR and AIDE are retrained on our face corpus. FatFormer and RINE use their released weights. These are the checkpoints validated by their authors, and retraining them risks under-tuning them. We agree that the two protocols must be visible. The revision will label them clearly in Table 2 and in the text.

MLLM-based forensic detectors. We agree that these detectors are directly relevant to our motivation. The revision will add an evaluation of the released FakeShield (Xu et al., ICLR 2025) and SIDA (Huang et al., CVPR 2025) on the six benchmarks. Both are already cited in the paper. If the reviewer has further MLLM-based detectors in mind, we will add them as well.

We thank the reviewer again. The review set a clear standard for the central claim, for the necessity of the VLM, and for the documentation of the training corpus. It has led to a narrower claim, new experiments, and a more transparent paper.

Reviewer_N4MDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper argues that face-forgery authenticity judgment is a perceptual System 1 task poorly served by generative chain-of-thought, and proposes PDM-Face, a VLM decision layer that answers eight fixed forensic hypotheses in parallel without generating text. A shared prefix (image plus four soft evidence tokens from a frozen forensic expert layer) is encoded once, its attention KV cache and delta-rule recurrent state are copied into eight branches answered in one batched pass, and a learned fusion maps the eight three-way answers plus expert scalars to a calibrated fake probability. On six external benchmarks it reports beating generative reasoning in all 12 comparisons at 21x lower latency, gains over its own expert layer on FF++ and DFDC, and wins over four recent detectors in 19 of 23 comparisons.

Critical strengths

The exact prefix-branching mechanism for a hybrid attention plus gated-delta-rule VLM is the real contribution. Copying both the KV cache and the recurrent plus convolution state, and verifying agreement to 2.7e-4 against full recomputation, is a careful and genuinely useful systems result for parallel multi-hypothesis inference. The experimental hygiene is above average, with group-aware identity splits, quality gates that prevent separating real from fake by image quality, path-matched scoring across all methods, paired class-stratified bootstrap intervals, and three seeds on the main models. The reframing of forensic judgment as a panel of mechanism and spectral hypotheses supervised by existing metadata, with no teacher rationales, is clean and the evidence-dropout trick to stop the VLM from copying the expert scores is sensible.

Critical weaknesses
  • The central scientific claim rests on a weak comparison. “Reasoning is the wrong mode” is established against a single 4B model with expert scalars pasted as text and the score read as P(yes) versus P(no) after a 998-token trace, which is a known-poor way to extract a calibrated score from a reasoning model, so the result shows this prompting setup underperforms a trained fusion rather than that System 2 is inherently worse for this task. The improvement over the expert layer is small and not uniform.
  • On the Table 1 samples PDM-Face is below the expert layer on 3rdBench (0.965 vs 0.977) and WildDeepfake (0.738 vs 0.749), and the positive gains elsewhere are +0.025 to +0.034, so the decision layer adds marginal value and sometimes subtracts it.
  • The headline “19 of 23” win over recent detectors is largely inherited from the expert layer, not from the VLM. The expert layer alone already beats all four external detectors on the benchmarks where PDM-Face wins, and on FF++ both FatFormer (0.796) and RINE (0.808) beat PDM-Face (0.651) by a wide margin, so the comparison is favorable mainly on the sets the frozen experts dominate.
  • The absolute numbers on DFDC, FF++, and WildDeepfake (0.62 to 0.74) are well below current deepfake-detection SOTA, so the practical detection claim is weak independent of the efficiency story.
  • The “parallel beats single-query” result holds on only two of six benchmarks (GenImage and 3rdBench) and is tied on the other four, so the eight-hypothesis panel is doing little on most data.
  • Generality is also thin, since the thinking-mode comparison and most ablation rest on one backbone and the second VLM appears only in a narrow check.
Questions for authors
  • Since the reasoning baseline reads P(yes) versus P(no) after a long trace rather than a calibrated parse, how much of the reasoning deficit is the decision mode versus the score-extraction protocol, and does the gap persist if the reasoning model is given the same learned fusion head over its final states?
  • Given that PDM-Face trails the expert layer on 3rdBench and WildDeepfake and gains only marginally elsewhere, what is the evidence that the decision layer adds value rather than mostly passing the experts through?
  • Can you present Table 2 in a way that separates the expert layer’s contribution from the VLM’s, since the wins over NPR, AIDE, FatFormer, and RINE appear to be the expert layer’s rather than PDM-Face’s?
  • Why are FatFormer and RINE much stronger on FF++ while PDM-Face is near chance there, and what does that imply for the “significantly better” framing?
  • Do the parallel-over-single-query gains extend beyond GenImage and 3rdBench on a third backbone, or is the panel’s benefit specific to multi-generator-family sets?
  • How does PDM-Face compare at equal cost against a larger non-reasoning VLM answering a single question, to confirm the gain comes from parallel hypotheses rather than from the added fusion capacity?
5 comments
AuthorsOpenReview ↗

Thanks for the reviewer's rigorous technical assessment (Part 1/3)

We are grateful for this rigorous and technically deep assessment. The review identifies the exact prefix-state branching for hybrid attention and delta-rule models as the core contribution, and it checks our tables number by number. It also shows exactly where our wording went beyond our evidence. We accept these points. We have used them to tighten the claims and to run new analyses.

W1 / Q1: We agree. Our comparison shows that zero-shot reasoning under this protocol underperforms a trained decision layer. It does not show that System 2 is inherently worse for this task, and we will no longer phrase it that way.

Revised claim. The revision states the result as follows: with the same backbone and the same evidence, a trained parallel decision layer is more accurate than zero-shot generative reasoning, and about 21 times faster per image. The abstract, the contribution list, the title of Section 5.1, and the conclusion will be rewritten accordingly.

Decision mode versus score extraction. We ran a new analysis on the existing traces to separate the two. In the paper, 26.9% of the traces reach the token cap, and the answer is then read after a truncated trace. We restricted the evaluation to the traces that finished below the cap. Reasoning with expert scores remains below PDM-Face on every benchmark. The paired 95% confidence intervals of PDM-Face minus reasoning are:

Benchmark95% CI, traces below the cap
Celeb-DF[+0.216, +0.324]
3rdBench[+0.102, +0.155]
GenImage[+0.120, +0.190]
DFDC[+0.122, +0.223]
FF++[+0.063, +0.157]
WildDeepfake[+0.092, +0.189]

The gap is therefore not caused by truncation. We also note that AUROC depends only on the ranking of the scores, so the calibration of the readout does not enter the metric.

Two further results do not depend on how PDM-Face is trained. First, reasoning with the expert scores ends below the expert layer it was given on all six benchmarks. For example, 3rdBench drops from 0.983 to 0.864. This compares the input and the output of the same reasoning model. Second, the cost of about 998 generated tokens per image (2.89 s against 0.137 s) is independent of the readout.

A learned head on the reasoning model. A fusion head trained on the final states of the reasoning model is a trained reasoning variant, like a fine-tuned reasoning baseline. Such a head still follows a generated trace, so the cost result above is unchanged. Under the revised claim, the paper makes no statement about trained reasoning. The revision will present both variants as complementary future work. It will also add a discussion of RD-MLDG (Xu et al., ICLR 2026, "Reasoning-Driven Multimodal LLMs for Domain Generalization"). That work reports that fine-tuning MLLMs on reasoning chains for classification is harder to optimize than direct label supervision, and that a direct classification path is needed to stabilize it. This is consistent with our choice of a direct decision path, and it defines the setting in which trained reasoning should be studied.

AuthorsOpenReview ↗

Thanks for the reviewer's rigorous technical assessment (Part 2/3)

W2 / Q2: Three pieces of evidence show that the decision layer does more than pass the experts through.

1. Significant gains on the full test sets. PDM-Face improves over its expert layer by +0.034 on FF++ (21,144 faces, 95% CI [+0.030, +0.038]) and by +0.021 on DFDC (30,460 faces, [+0.018, +0.024]). On GenImage the gain is positive for all eight generators (+0.010 to +0.034).

2. New experiment: the same evidence without the VLM. We trained light fusion heads on the same 60,000 faces and on the identical 1,315-dimensional evidence vector that feeds PDM-Face's evidence tokens. The arms are a CLIP router over the expert predictions, logistic regression, a two-layer MLP, and the MLP with the same eight hypothesis heads, labels, and loss weight as PDM-Face. Each arm has three seeds and is selected on validation data only. The multi-task MLP was the best arm on validation. PDM-Face is significantly better than this arm on DFDC (+0.019, [+0.005, +0.032]), FF++ (+0.021, [+0.011, +0.032]), and WildDeepfake (+0.034, [+0.022, +0.046]). GenImage (+0.003) and 3rdBench (-0.001) are statistically tied. A layer that only passed the expert evidence through would not be expected to exceed a fusion head trained on that same evidence. The light heads take under 0.1 ms per image, and the VLM decision layer takes 11.7 ms.

3. New analysis: where the ranking changes. On 3rdBench, PDM-Face minus the expert layer is -0.011, -0.013, and -0.012 for the three seeds. The difference is consistent and small, and the expert layer is already at 0.977 there. On the text-to-image subset both reach 1.000. Sorted by expert confidence, 76% to 85% of the difference comes from the half of the images where the expert layer is least confident. On its confident images PDM-Face leaves the ranking essentially unchanged. On WildDeepfake the two are statistically tied on the full test set (0.742 against 0.746, CI includes zero).

Taken together, the decision layer changes the ranking mostly on images where the experts are uncertain. This is where it gains on FF++ and DFDC (compressed video, face swap, reenactment). It is also where it costs about 0.01 on 3rdBench, whose expert score is near the ceiling. We will add the light fusion experiment and the per-seed and per-confidence analysis to the appendix. The main text will describe the value of the decision layer in these terms: its gains are concentrated where the experts are weak.

W3 / Q3: We agree. Much of the margin over the four detectors comes from the expert layer. The expert layer is part of our system, so Table 2 is a comparison between systems, and the revision will say so. Table 2 will be restructured as follows.

  • The expert layer and the full PDM-Face will appear as two labelled stages of one system, and the expert layer will no longer be a reference row only.
  • A new row will give the paired gain of the decision layer over the expert layer on each benchmark, with its significance.
  • The two baseline protocols (retrained on our corpus, or released weights) will be marked in the table.

The sentence on 19 of 23 comparisons will be rewritten as a system-level statement. The contribution of the decision layer will be reported separately, through its paired gains over the expert layer and over the light fusion heads above.

AuthorsOpenReview ↗

Thanks for the reviewer's rigorous technical assessment (Part 3/3)

W4 / Q4: We acknowledge that FatFormer and RINE are stronger than PDM-Face on FF++. A breakdown by forgery type shows where the gap lies. On face swaps, PDM-Face reaches 0.692, RINE 0.871, and FatFormer 0.859. On reenactment all detectors are low (PDM-Face 0.596, RINE 0.683). The gap is therefore a face-swap gap.

The two kinds of detector are complementary. A parameter-free rank fusion of PDM-Face with RINE reaches 0.778 on FF++, which is +0.127 over PDM-Face alone. Because the expert layer enters PDM-Face only through evidence tokens, a CLIP-based detector of this kind can be connected as an additional expert.

On the absolute numbers: the results on DFDC, FF++, and WildDeepfake are cross-dataset numbers. A nearest-neighbor check finds no test image with a CLIP cosine similarity of 0.95 or more to any training face on these three benchmarks. On the same WildDeepfake images, FatFormer and RINE reach 0.662 and 0.665, and PDM-Face reaches 0.738. The revision will state the protocol next to these numbers, so that they are read as cross-dataset results.

On the framing: the revision will replace the blanket phrase "significantly better" with per-benchmark statements. These will say in the main text that FatFormer and RINE lead on FF++.

W5 / Q5: The gain of the panel over the single-query layer is positive for all three seeds on three benchmarks: GenImage (+0.040), 3rdBench (+0.017), and DFDC (+0.013). The single-query layer is a strict control. It has the same backbone, evidence tokens, evidence dropout, data, and schedule. On a second backbone with a different architecture, the full-attention Qwen3-VL-4B, the gain reappears on GenImage (+0.024, 95% CI [+0.019, +0.030]) and on 3rdBench (+0.006, [+0.003, +0.009]).

We agree with the reading offered in the question. The accuracy benefit of the panel is most consistent on benchmarks whose fakes come from several distinct generator families. There the mechanism questions and the spectral questions carry different information, and one holistic question cannot separate them. The revision will state this scope explicitly in the abstract and in Section 5.2. The claim is limited to the two backbone architectures that we tested.

W6 / Q6: Parallel hypotheses or added capacity. The fusion is a two-layer MLP with 64 hidden units. The other trained modules (LoRA, evidence adapter, answer head) are the same in the single-query layer, which uses the same data and schedule. The gain of the panel therefore does not come from a larger trainable model.

Backbone capacity. Backbone size does not drive accuracy either. With the smaller Qwen3.5-2B backbone (one seed), PDM-Face stays close to the 4B model:

Backbone3rdBenchGenImageDFDCFF++
Qwen3.5-2B0.9590.8440.6480.671
Qwen3.5-4B0.9660.8510.6370.659

Capacity is therefore not the limiting factor in this range. A larger single-question VLM would spend its additional cost on capacity. The panel instead reuses one prefix encoding: eight hypotheses cost 11.7 ms per image with branching, against 140.1 ms when they are asked one after another.

Generality. The comparison of decision layers is run on two architectures: the hybrid attention and delta-rule Qwen3.5-4B, and the full-attention Qwen3-VL-4B. The AUROC of PDM-Face differs by at most 0.017 between them on any benchmark. The reasoning comparison uses Qwen3.5-4B, and the revised claim states this scope ("with the same backbone").

We thank the reviewer again for the depth of this assessment. The review separates the mechanism, which it found sound, from the claims, which it found too broad. The revised paper follows the same separation.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Thanks to the SACs, ACs, and All Reviewers

We sincerely thank the SACs, the ACs, and all three reviewers for their time and for the care they put into this submission. The reviews are specific and technically grounded. They have made the paper more precise, and we answer every point in the individual replies.

Contributions and why they matter.

  1. Exact parallel hypothesis inference. PDM-Face encodes the image and the forensic evidence once, copies the prefix state into eight hypothesis branches, and answers all of them in one batched pass without generating text. For hybrid VLMs it copies both the key-value cache and the recurrent state of the gated delta-rule layers. The branched result agrees with full recomputation within 2.7e-4 in float32. It costs 11.7 ms per image, against 25.6 ms with a replicated prefix and 140.1 ms with eight sequential questions (11.9 times faster).
  2. A controlled measurement of generative reasoning on this task. With the same backbone, the same images, and the same expert scores, zero-shot reasoning writes about 998 tokens per image and takes 2.89 s. PDM-Face takes 0.137 s (21 times faster) and is more accurate in all 12 comparisons. Reasoning with the expert scores also ends below the expert layer it was given on all six benchmarks.
  3. Supervision without rationales. The eight hypotheses are supervised by labels that already exist (generator family and spectral statistics). No teacher model and no written explanation is needed.

What the revision will contain. We commit to revising every point raised by the reviewers in the new version. The main changes are as follows.

  • A narrower central claim. The revision states: with the same backbone and the same evidence, a trained parallel decision layer is more accurate than zero-shot generative reasoning, and about 21 times faster per image. We no longer phrase the result as a statement about reasoning in general. Fine-tuned reasoning is presented as complementary future work, and we add a discussion of RD-MLDG (Xu et al., ICLR 2026).
  • New experiment: light fusion baselines. A CLIP router, logistic regression, an MLP, and an MLP with the same eight hypothesis heads were trained on the same 60,000 faces and the same 1,315-dimensional evidence vector. Against the strongest of them, PDM-Face is significantly better on DFDC (+0.019), FF++ (+0.021), and WildDeepfake (+0.034), and statistically tied on GenImage and 3rdBench.
  • New analysis: reasoning traces below the token cap. The gap to reasoning remains on every benchmark when truncated traces are excluded.
  • New analysis: per seed and per expert confidence. It shows where the decision layer changes the expert ranking, and it explains the small difference on 3rdBench.
  • Table 2 as a system-level comparison. The contributions of the expert layer and of the decision layer are separated, the two baseline protocols are labelled, and the wording becomes per benchmark.
  • Training corpus documentation. A full composition table (sources, 23 generator families, counts, matching procedure) and a new nearest-neighbor overlap check against all external benchmarks.
  • More baselines and analyses. An evaluation of FakeShield and SIDA on the six benchmarks, a per-expert AUROC table, and a gate-floor ablation.

Significance and societal impact. Face forgeries are used for identity fraud, non-consensual imagery, and misinformation. Moderation and identity-verification systems need fast and calibrated decisions at scale. A decision that costs one forward pass and no generated text makes VLM-based screening affordable. The hypothesis panel also gives an auditable readout for each image: which generation mechanism and which spectral cue fired. This readout does not rely on free-text explanations that cannot be verified.

Open-source commitment. We will open-source the algorithm and the experimental design. This covers the code of the decision layer, the prefix-state branching for hybrid gated delta-rule models, and the training and evaluation scripts. It also covers the reasoning baseline, the new light fusion baselines, and the full experimental protocol (prompts, hyperparameters, seeds, and statistical tests).

We thank the SACs, the ACs, and the reviewers once more. Their comments led to new experiments and to clearer claims, and the revised paper is stronger because of them.

AuthorsOpenReview ↗

Summary of the Revision and New Experiments

We thank the AC and all three reviewers for their continued attention to this submission. We have uploaded a revised PDF. It implements the changes that we promised in our first responses, and the table at the end of this note lists where each change is. One item goes beyond the promise: fine-tuned reasoning, which we had announced as future work, is now an experiment in the paper. This note summarizes the results finished since the first response and shows where each reviewer question is answered.

New results since the first response.

  1. Fine-tuned reasoning baseline. The reasoning model uses the same Qwen3.5-4B backbone, LoRA on the same target modules, and the expert scores as text. It is trained on its own reasoning chains for 4,000 faces, filtered for correct answers, plus a direct-answer path as in RD-MLDG (21,150 samples, 1,322 steps of batch size 16; PDM-Face trains for 1,250 steps of batch size 16). PDM-Face is significantly higher on all six external benchmarks at each of its three seeds, by +0.056 to +0.176 AUROC.
  2. Released MLLM forensic detectors. FakeShield and SIDA-7B are evaluated with their official weights on the same images. The mean AUROC over the six benchmarks is 0.780 for PDM-Face (three seeds), 0.714 for SIDA-7B, and 0.680 for FakeShield. At every seed, PDM-Face is significantly above both on Celeb-DF, 3rdBench, and GenImage, and above SIDA-7B on WildDeepfake. Both detectors are ahead on FF++ at every seed. On DFDC, FakeShield is tied with PDM-Face, and SIDA-7B is ahead at two of three seeds.
  3. Complete light-fusion table. The table now lists all four fusion heads on all six benchmarks, including Celeb-DF, as three-seed means, with a paired test at every seed. It replaces the single-seed test of our first responses. Against the selected head, PDM-Face is higher on DFDC, FF++, and WildDeepfake (significant at two of three seeds each) and lower on Celeb-DF (three seeds), 3rdBench (two), and GenImage (one). The decision layer is the stronger of the two where the expert evidence is weakest, at a cost of 11.7 ms per image, and one forward pass also returns its per-hypothesis, auditable outputs.
  4. Gate-floor ablation. We re-implemented the gate of the expert layer with floor c in {0, 0.25, 0.5} and three seeds. The mean AUROC over the six benchmarks is 0.745, 0.757, and 0.758.
  5. Tests at every seed. The paired tests against fine-tuned reasoning, the selected fusion head, and the two MLLM detectors are reported for each of the three PDM-Face seeds (Tables 20, 22, and 28).
  6. Larger backbone. A single-question layer on Qwen3.5-9B is significantly below PDM-Face on 4B on five of the six benchmarks and not significantly different on Celeb-DF (seed 0, one run per arm). On 9B, PDM-Face is significantly above the single question on five benchmarks (narrowly on 3rdBench) and below it on Celeb-DF.

Where each question is answered in the revised PDF.

TopicRaised byLocation
Central claim, now limited to the same backbone and the same expert layerN4MD, WDVrAbstract, Sections 1, 5.1, 6, and 7
Fine-tuned reasoning baselineWDVr, N4MDSection 5.1, Table 1, Appendix H (Tables 19 and 20)
Reasoning traces below the token capN4MDSection 5.1, Appendix G (Table 18)
Decision layer against expert layer, per seed and per confidencen4fj, N4MDSection 5.2, Table 2, Appendix F (Tables 12 to 16), Appendix K
Need for the VLM, light router and fusion headsn4fj, WDVrSection 5.3, Table 3, Appendix I (Tables 21 to 25)
Expert selection, complementarity, gate floorn4fjAppendix E (Tables 10 and 11)
System-level comparison with two labelled stages, FF++N4MDSection 5.4, Table 4, Appendices N, O, and P
MLLM detectors and baseline protocolsWDVrSection 5.4, Table 4, Appendix J (Tables 26 to 28)
Training corpus composition and overlap checkWDVrAppendix A (Tables 5 and 7)
Parallel hypotheses against a single question, backbonesN4MDSections 5.2 and 5.5, Appendices L and M
Larger backbone; a larger single-question modelN4MDSection 5.2, Appendix L (Table 30)
Positioning, RD-MLDG, and Venus-DefakerOne2WDVrSections 2 and 6

Two table entries changed in the third decimal because of rounding: the in-domain mean of PDM-Face in Table 2 (0.997) and the FF++ seed-1 gain in Table 13 (+0.001). Two detector values follow the submitted PDF (AIDE on FF++ 0.533, RINE on WildDeepfake 0.666).

The revision states its comparisons per benchmark and gives the setup of each new experiment in the appendix. The individual replies give the full tables.

We thank the AC and the reviewers again. The follow-up discussion asked for specific experiments, and the paper now contains them.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.