acceptodds
Under review as a conference paper at ICLR 2027

Can a world model predict a deepfake video? A Consistency Margin Prior for Deepfake Detection

Abstract

Deepfake detectors often perform well on the dataset they are trained on, but their performance drops when tested on videos from different datasets. In this paper, we study whether a predictive video model can provide a signal that generalizes better across datasets. Instead of relying only on generator-specific artifacts, as many existing detectors do, we use a model trained to predict natural video and measure the difference between what it predicts and what it observes in a video. Using V-JEPA 2.1, we keep the encoder frozen and adapt the predictor using rank-16 LoRA. We define the difference between the predictor output and the frozen encoder target as the prediction residual. We hypothesize that manipulated clips produce, on average, larger prediction residuals than authentic clips, and use this as our consistency-margin prior. We process these residuals using a lightweight Latent Consistency Head (LCH), which separates residual magnitude from direction for training and classification. To test whether the direction of the consistency margin affects cross-dataset generalization, we compare standard and reversed margin objectives while keeping the remaining training setup fixed. We match the two settings on FF++ validation performance before evaluating them on unseen data. The reversed-margin setting exhibits a larger cross dataset generalization gap on average, providing evidence that the direction of the consistency margin affects cross-dataset transfer. Our complete model adds only M trainable parameters, which is 2.47% of the total M parameters. Trained only on FF++, it achieves an AUC of on Celeb-DFv2 and performs well across several other cross-dataset benchmarks. Our ablations show that separating residual magnitude from direction, reducing dataset-specific variation, and using a low-rank projection improve cross-dataset generalization in our experiments.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Deepfake Detection, VJPEA2, Forensic Residual, Video Foundation Model

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_fe1WOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes ConsistencyNet, a deepfake video detection framework that leverages latent prediction residuals from a pretrained V-JEPA 2.1 model to improve cross-dataset generalization. The method introduces a consistency-margin prior, hypothesizing that manipulated videos exhibit larger prediction errors than authentic videos. A Latent Consistency Head (LCH) is designed to separate residual magnitude from direction and incorporates attention-conditioned centering, multi-horizon projections, and spatial anomaly pooling for classification. The authors further propose a Sign-Flip Identification Protocol (SFIP) to investigate the effect of margin direction on generalization. Experiments on multiple cross-dataset and cross-manipulation benchmarks demonstrate competitive performance, while ablation studies examine the effectiveness of the proposed components.

Critical strengths
  1. Interesting research perspective. Leveraging prediction residuals from a pretrained video world model provides a potentially useful direction for generalizable deepfake detection, which should be less explored before.
  2. Reasonable intuition for prediction residual. The core intuition of prediction residual is coherent with common sense that fake videos should exhibit larger discrepancies.
  3. The experimental comparisions are extensive.
Critical weaknesses
  1. The central hypothesis is not sufficiently validated. The paper assumes that manipulated videos naturally produce larger prediction residuals which is consistent with common sense, but there is still no direct theoritical or empirical proof of this hypothesis. Besides, I think whether the prediction residuals are related to the training data used by the world model, since it would affect the predicted data distributions towards real or fake.
  2. The effectiveness of SFIP is not convincingly established. The reported 1.8% average generalization-gap difference is relatively modest, and the paper lacks in-depth explanations or experiments to support the claimed effect of margin direction, such as why different direction leads to different performance.
  3. Insufficient analysis of the proposed architectural contributions. Although LCH incorporates several components, the ablations do not independently examine PASC, spatial anomaly pooling, or the multi-head projection design, which makes it harder to recognize which part contributes.
  4. Paper presentation. The presentation of some figures is recommended to further polish.
Questions for authors

Please see the weakness above. My major concerns focus on the hypothesis proof and the method design. Thus, I currently lean towards a negative ratings.

Reviewer_HEajOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a method for deepfake video detection based on the prediction ability of the world model V-JEPA, which is independent of the deepfake training dataset. To better model fake detection from prediction residuals, the paper designed the LCH module. The experiments show promising improvement compared to other video-based methods.

Critical strengths
  1. Innovatively introduces the world model into video detection based on abnormal detection like residual design.
  2. Latent Consistency Head (LCH) considers token amount, direction, and pooling design to provide explainable benefits to the task.
  3. Clear definition and motivation expression.
Critical weaknesses
  1. The ablation study does not cover the LCH module, which is a core design in the architecture. Maybe use a simpler structure for comparison.
  2. The consistency-margin prior is assumed but is not experimentally evaluated.
  3. Figure 2 is not carefully designed, with math equations uncompiled and text font disordered.
Questions for authors
  1. Is it possible to extend the residual-based method to frame-based detection? It seems that frame-based detection is better than video-based detection in Table 1.
  2. Is it possible to present an ablation study on the LCH module?
Reviewer_punPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a deepfake detector based on prediction residuals from V-JEPA 2.1, guided by the prior that manipulated videos exhibit greater predictive inconsistency than authentic ones. With a frozen encoder, LoRA-adapted predictor, and lightweight Latent Consistency Head, it achieves 90.5% AUC on Celeb-DFv2 when trained only on FF++, using 8.25M trainable parameters. Controlled experiments support the role of the consistency-margin direction in cross-dataset generalization.

Critical strengths
  1. Using latent video prediction errors as forensic cues is a promising extension of reconstruction-based detection to pretrained predictive video models.
  2. Only 8.25M parameters are trainable, allowing adaptation while keeping the large pretrained encoder frozen.
Critical weaknesses
  1. The methodological novelty is moderate, and ablations are incomplete. The approach combines established operations, including low-rank projection, normalization, gated pooling, and margin learning. Their integration may be useful, but Table 3 does not individually establish the contributions of PASC, the low-rank bridge, or spatial anomaly pooling.
  2. The empirical advantage and reliability remain limited. ConsistencyNet is competitive but trails several methods in Table 1 and achieves 95.5 average AUC on DF40, compared with 98.3 for the strongest listed baseline.
  3. In Table 1, the generalization ability of the proposed method is not good. It is worse than DiffusionFF and query-token forgery purification on two benchmarks. The advantage of the proposed method is still not clear.
Questions for authors

Can you provide individual ablations for PASC, residual direction, prediction-length-specific projections, and spatial anomaly pooling?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.