acceptodds
Withdrawn from ICLR 2027

Physics-Guided Cascaded Refocusing for Efficient Deepfake Video Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Advances in generative models have dramatically improved the realism of manipulated face videos, posing escalating threats to public security, information credibility, and digital forensics. Dominant detectors follow a fixed-sampling, one-shot paradigm that overlooks the non-uniform temporal distribution of forgery evidence, where traces concentrate within only a few frames, rendering results unstable and subject to considerable randomness. A second active sampling stage promises gains in accuracy and stability yet confronts an intrinsic trade-off: if locating complementary evidence is costly, enlarging the initial sample would be cheaper. We propose a physics-guided cascaded forensic perception framework resolving this trade-off. In the first stage, a Local–Global Resonance Mechanism aggregates cross-frame CLS tokens into global forgery evidence and re-injects it into local patch representations, while predictive uncertainty lets high-confidence samples terminate early and others trigger re-inspection. Candidate clips are then ranked by feature discrepancy and filtered by abnormal facial motion, a discriminative yet network-free signal, enabling targeted acquisition of complementary evidence at negligible cost. Extensive experiments on multiple benchmarks, unseen manipulations, and cross-dataset settings demonstrate consistent improvements in accuracy, stability, and generalization with reasonable efficiency, while qualitative analyses provide interpretable evidence by highlighting facial regions and abnormal motion patterns.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Deepfake Detection, Physics, Local-global Resonance

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_SqqGOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a cascaded framework for generalizable deepfake video detection, motivated by the uneven temporal distribution of forgery cues. An initial eight-frame clip is analyzed using a Local–Global Resonance Mechanism that exchanges information between cross-frame CLS tokens and spatial patch features, together with second-order temporal feature differences. Predictive entropy determines whether to terminate early or perform a second inspection. For uncertain predictions, the framework selects complementary clips by feature distance, then uses facial landmark motion residuals to identify a candidate for re-inspection and prediction fusion. Evaluation includes cross-dataset testing, generalization to five unseen manipulation methods, component ablations, qualitative visualizations, and accuracy–efficiency comparisons. The reported results suggest improved detection performance and reduced sensitivity to initial clip sampling, with lower computational cost than fixed multi-clip inference.

Critical strengths
  1. The paper addresses a meaningful limitation of fixed-sampling detection: a short initial clip may miss temporally sparse forgery cues. Its framework connects the decision of whether to acquire more evidence with the selection of where to look next, using feature complementarity and facial motion residuals. This provides a coherent approach to allocating additional computation selectively rather than processing multiple clips for every video.
  2. The method reports the highest AUC among the compared approaches on three of four cross-dataset benchmarks and four of five unseen manipulation methods. Component ablations support the complementary value of feature-based and motion-based selection, while repeated initial sampling yields low reported performance variation. These results provide encouraging evidence that the framework can improve both generalization and sampling stability.
Critical weaknesses
  1. The key assumption underlying re-inspection gating is insufficiently validated. Re-inspection is triggered only when the initial prediction has low confidence. However, missing informative forgery cues does not necessarily produce uncertainty: a spatially convincing and temporally coherent eight-frame clip may be confidently misclassified as authentic, causing early termination even when detectable artifacts exist elsewhere in the video. The paper does not report the error rate among early-exit samples, the proportion of initial errors routed to re-inspection, or how many confidently incorrect predictions could be corrected by forced re-inspection. Consequently, the aggregate AUC improvements do not establish whether the gate reliably identifies videos requiring additional evidence. This gap directly affects the central motivation of addressing missed evidence under limited sampling.
  2. The controls and cost analysis do not sufficiently establish the benefits of targeted evidence acquisition. Although the paper includes component ablations and multi-clip baselines, the supplementary sampling strategy used in the UCFA-only configuration is unclear. An explicit comparison against random supplementary sampling under the same gate, detector, and observation budget is needed to distinguish the benefit of targeted selection from that of simply acquiring more observations. Furthermore, candidate clips require encoding, yet the candidate pool size, landmark extraction procedure, and stage-wise runtime costs are insufficiently specified. In Table 4, the proposed method has lower FLOPs but higher latency than the compute-matched baseline (23.4 versus 19.7 ms). Since low-cost complementary evidence discovery is a central contribution, these gaps leave the claimed performance–efficiency trade-off insufficiently substantiated.
Questions for authors

The following questions concern clarification or validation of the current method and experiments, rather than suggestions for future work.

  1. Effectiveness of the re-inspection gate: On each cross-dataset benchmark, what is the error rate among early-exit samples, and what proportion of initially incorrect predictions triggers re-inspection? If the second stage is forcibly applied to confidently incorrect predictions, how many can be corrected? These results would clarify whether the gate excludes errors that additional evidence could resolve.

  2. Independent contribution of targeted selection: In the UCFA configuration without PGED in Table 3, how is the supplementary clip selected? With the confidence gate, detector, number of supplementary clips, and prediction fusion rule held fixed, how much does the proposed selection strategy improve over random supplementary sampling? Please also specify how the two predictions are fused.

  3. End-to-end cost and comparison fairness: Please specify the candidate pool size, temporal sampling stride, retained candidate count (K), landmark extraction method, and whether candidate features are precomputed or cached. Does Table 4 include the full cost of obtaining these features and landmarks? What are the stage-wise runtimes, and are timing and batching settings consistent across baselines? These details are important for assessing the low-cost search claim, particularly given the higher latency relative to the compute-matched baseline.

Reviewer_VL8sOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a physics-guided cascaded framework for generalizable deepfake video detection. It combines a Local–Global Resonance Mechanism (LGRM) for cross-frame feature interaction, uncertainty-based activation for selective re-inspection, and physics-guided evidence discovery to identify informative video segments. Experiments on multiple cross-dataset and unseen-manipulation benchmarks show competitive or state-of-the-art performance.

Critical strengths
  1. The cascaded design addresses the limitations of fixed video sampling by selectively inspecting additional clips when predictions are uncertain.
  2. The method achieves impressive cross-dataset generalization, particularly on DFDC and CDF-v2.
Critical weaknesses
  1. Sec. 3.4 requires encoding candidate clips to compute feature discrepancies, which may introduce nontrivial overhead. Although Table 4 reports FLOPs and latency, the computational cost of candidate search is not sufficiently broken down, making the claimed efficiency advantage difficult to assess.
  2. Large facial motion residuals can also arise from natural expressions, rapid movements, or inaccurate landmark detection. The paper lacks targeted experiments examining these factors, leaving the robustness of the physics-guided selection mechanism uncertain.
Questions for authors
  1. How many candidate clips are evaluated during re-inspection, and how much computation is spent on candidate encoding versus landmark-based motion analysis?
  2. How robust is the physics-guided selection strategy to rapid facial expressions, head movements, and landmark detection errors?
  3. Does the threshold selected on FF++ remain well calibrated across different datasets?
Reviewer_g5nzOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This work reformulates Deepfake video detection from fixed-sampling, one-shot prediction into an uncertainty-driven cascaded forensic process, where easy samples are classified directly while difficult cases trigger targeted re-inspection for complementary evidence. By combining local–global feature interaction, feature discrepancy, and physics-guided facial motion cues, the method improves cross-dataset generalization and sampling stability while maintaining reasonable efficiency and interpretability.

Critical strengths
  1. The paper presents a well-motivated adaptive forensic framework that goes beyond conventional fixed-sampling, one-shot deepfake detection. By combining uncertainty-guided re-inspection with targeted complementary evidence acquisition, the method directly addresses the instability caused by sparsely distributed forgery cues and achieves strong cross-dataset performance with low sampling variance.
  2. The proposed framework achieves a favorable balance between detection accuracy, efficiency, and interpretability. The combination of local–global feature interaction and physics-guided motion cues consistently improves generalization, while activating re-inspection for only a subset of samples and providing region-level evidence around suspicious facial areas.
Critical weaknesses
  1. Novelty: The novelty of the proposed adaptive re-inspection strategy is somewhat weakened by prior work such as ADA-FInfer[1],[2] and [3], which has already explored adaptive frame selection for deepfake detection. The paper should more clearly differentiate its uncertainty-triggered, physics-guided clip selection from existing adaptive sampling approaches and provide direct comparisons to demonstrate the added value of its design.

2.The ablation study does not sufficiently rule out simpler re-sampling strategies. In particular, it would be valuable to compare PGED against a straightforward baseline that directly re-inspects the temporally most dynamic or most changed clip, to verify whether the proposed feature-discrepancy and physics-guided selection provides benefits beyond simply selecting high-motion segments.

3.The paper lacks sufficient implementation details for the re-inspection stage, as the decision fusion strategy and the specific values of key hyperparameters such as KK, Δ\Delta, and δ\delta are not provided.

4.Although the paper demonstrates cross-dataset and cross-manipulation generalization, it lacks dedicated robustness evaluations under common video degradations or challenging facial conditions. This is particularly important for the proposed physics-guided module, whose reliance on facial landmarks may make it sensitive to compression, blur, occlusion, large poses, or landmark detection errors.

[1]ADA-FInfer: Inferring face representations from adaptive select frames for high-visual-quality deepfake detection. Juan Hu, et al. TDSC, 2024.

[2] Frame Selection Strategies for Video Deepfake Detection: Benchmarking Accuracy and Runtime Trade-Offs. Serackis A, et al. 2026.

[3] Deepfake video detection with feature interaction amongst key frames. Zhu Kaiman, et al. 2022

Questions for authors

Refer to the weaknesses. Could you provide additional ablation studies and robustness experiments?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.