acceptodds
Under review as a conference paper at ICLR 2027

DiffMotion: Differential Facial Motion Modeling for Robust Deepfake Detection

Abstract

Recent advances in deepfake generation have made forged facial videos increasingly realistic, challenging detectors that rely on appearance artifacts. We propose, a landmark-sequence framework that models facial motion as a complementary forgery cue. DiffMotion combines raw coordinates with bidirectional lag-1 and lag-2 differences, propagates them over a muscle-inspired sparse facial graph, and aggregates local temporal context using folded, overlapping windows. This design provides an interpretable account of which facial regions and motion relations contribute to a decision. Experiments on FaceForensics++ and Celeb-DF show competitive within-domain performance and strong resilience to controlled compression and landmark perturbations; they also reveal a substantial cross-dataset gap. Notably, DiffMotion maintains an accuracy of over 86.75% under severe noise and high frame-drop rates, significantly outperforming existing methods.We therefore position DiffMotion as an anatomically motivated and degradation-resilient detector, while treating domain generalization as an important limitation rather than a solved problem.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
deepfake detection, facial landmark dynamics, spatio-temporal modeling, robustness

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_e8nnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes DiffMotion, a deepfake detection method based on facial landmark movements. It combines raw landmark coordinates with forward and backward temporal differences, uses a sparse graph based on facial anatomy, and processes the resulting features with folded temporal windows and temporal convolutions. The method is evaluated on FaceForensics++ and Celeb-DF, including cross-manipulation, cross-dataset, and degradation robustness experiments.

Critical strengths

The method achieves some good cross-manipulation results on FaceForensics++, with the best average among the compared methods for three of the four training subsets. The degradation experiments are also interesting, and suggest that the model can tolerate quite substantial perturbations to the landmark sequences.

I also appreciate that the authors acknowledge the poor cross-dataset generalisation performance.

Critical weaknesses
  1. Limited novelty and insufficient ablations.

I am not convinced that the proposed method offers sufficient novelty. The related work section also seems quite sparse, particularly given the proposed contributions. For example, LRNet (Sun et al., CVPR 2021) already explores facial landmarks and temporal dynamics for deepfake detection. STUT (Cui and Hayama, 2023), although proposed for skeleton-based action recognition, also uses forward/backward temporal differences and temporal unfolding with structured spatial attention. The related work section would benefit from more relevant papers to better position the paper's contribution.

The ablations are also insufficient to establish the importance of the individual components. For example, folded temporal windows are presented as one of the main contributions, but there is no ablation removing them. The facial graph is only compared against a fully connected graph, rather than another sparse graph, so it is unclear whether the specific anatomical connections actually matter.

I also find the interpretation of the temporal differencing ablation questionable. The authors argue that removing temporal differences leaves static geometry that cannot capture temporal inconsistencies. However, the model still receives a sequence of landmark coordinates and contains temporal modelling layers. It is therefore not obvious why it should be unable to learn motion information. The very large drop from 99.65 to 70.58 AUC needs further explanation.

Overall, I do not think the paper sufficiently establishes what is new about the proposed architecture, or which of its design choices are actually responsible for the reported improvements.

  1. Weak experimental evaluation and limited evidence of generalisation.

The evaluation is restricted to FaceForensics++ and Celeb-DF, which are quite old benchmarks at this point. Given the rapid progress in deepfake generation, I would expect experiments on more recent and diverse datasets, especially for a paper that motivates its method by the increasing realism of generated videos.

The cross-dataset results are also quite poor. The model reaches only 66.70 and 59.62 AUC on Celeb-DF V1 and V2, substantially below the compared methods. I do not think the results establish a competitive deepfake detector overall, despite some good performance on the FF++ cross-manipulation experiments.

There are also some inconsistencies in the reported results. For example, Table 2 reports 97.9 AUC on FF++ C23, whereas Tables 3 and 4 report 99.65 for what appears to be the same model and dataset. I could not find an explanation for this.

Finally, the robustness experiments are not entirely convincing. Frame dropping and Gaussian noise are applied to already extracted landmarks, rather than to the original videos. This does not account for failures in face detection, landmark extraction, or tracking caused by degraded input videos. The comparison with UMCL is also difficult to interpret because the methods use different input representations and the perturbations are not equivalent. I therefore do not think these experiments establish the claimed robustness advantage over existing approaches.

Questions for authors

Why does the FF++ C23 result differ between Table 2 (97.9) and Tables 3/4 (99.65)? Are these different checkpoints or evaluation settings?

How does the proposed method differ from existing landmark-based approaches such as LRNet? I would also like to see an ablation of the folded temporal windows and a comparison with another sparse graph to better understand which components are actually important.

Can the authors clarify what remains in the model when temporal differencing is removed? Given that temporal convolutions are still present, why does removing explicit differences cause such a large performance drop?

Can the authors evaluate robustness by applying degradations to the input videos before landmark extraction, ideally under the same conditions as other landmark-based methods? It would also be useful to see results on a more recent deepfake dataset.

2 comments

Common response: verified records and new runs

We thank the reviewers for their reports. Since submission we audited the archived training logs and completed multiple training runs and fixed-set evaluations. Below we give the verified archived numbers, the new numbers, and the revisions they require. New results are video-level AUC (%), mean ±\pm sample standard deviation over seeds 1--3.

Cross-dataset and unseen generators.

Test collectionVideosFull DiffMotionRepeated lag-1LRNet, common protocol
FaceShifter (held-out source IDs)28069.02 ±\pm 2.3469.15 ±\pm 2.9356.81 ±\pm 1.23
DF40 BlendFace81770.94 ±\pm 1.6870.64 ±\pm 1.4367.38 ±\pm 2.22
DF40 DeepFaceLab14972.59 ±\pm 5.4770.20 ±\pm 7.6664.82 ±\pm 1.01
DF40 E4S1,33262.12 ±\pm 1.7565.89 ±\pm 3.8352.23 ±\pm 1.51
DF40 FaceSwap82372.45 ±\pm 1.9571.24 ±\pm 2.3968.40 ±\pm 1.45
DF40 FSGAN80979.09 ±\pm 1.8974.59 ±\pm 1.3068.38 ±\pm 0.69
DF40 HeyGen10016.29 ±\pm 11.2220.45 ±\pm 12.945.77 ±\pm 4.38
DF40 MobileSwap82277.49 ±\pm 0.8577.72 ±\pm 2.5864.59 ±\pm 1.47

Across the seven DF40 subsets, cross-dataset AUC ranges from 62.12 to 79.09. It is above the repeated-lag-1 control on four subsets and below it on E4S, HeyGen and MobileSwap; it is above the adapted LRNet on all seven. HeyGen is 16.29 ±\pm 11.22 against 5.77 ±\pm 4.38 for LRNet, both near or below chance with large variance.

Response to Reviewer e8nn

The two FF++ numbers (Q1). Table 2 reports the early checkpoint and Tables 3--4 the late checkpoint of the same 1,200-epoch C23/ALL run: 97.88 at epoch 23 and 99.65 at epoch 684. The archived pipeline had no validation partition and selected checkpoints on the configured test inputs with a single seed; every number in the revision uses a 700-video validation partition and seeds 1--3.

Novelty and related work (Q2). The revision adds LRNet (optical-flow/Kalman calibration, coordinate and difference GRU streams) and STUT, which uses forward/backward differences and temporal unfolding with structured spatial attention, and compares with LRNet under identical inputs: the full model is above LRNet on FaceShifter and on all seven DF40 subsets (table above). The contribution is stated as the facial-landmark instantiation and its evaluation; none of the operators is presented as new.

Evaluation breadth (Q4). The external evaluation now covers DF40 (seven subsets, 4,852 videos) and FaceShifter in addition to Celeb-DF: cross-dataset AUC ranges from 62.12 to 79.09 on DF40 and FaceShifter is 69.02 ±\pm 2.34. Celeb-DF V1 is 64.53 ±\pm 2.30 and V2 52.83 ±\pm 0.18, replacing 66.70 and 59.62. These results do not establish a competitive general detector on Celeb-DF, and the revision says so; the abstract and Section 5.4 no longer claim cross-dataset superiority.

Robustness (Q4). The two perturbations act at different pipeline stages. Frame dropping (30% retained frames) removes frames from the landmark sequences after extraction and calibration, so it isolates the sequence model. Gaussian noise is applied to the video frames before landmark extraction, so the perturbation passes through face detection, landmark extraction and calibration before reaching the model; this is the experiment that exercises the full pipeline under pixel-level corruption. The submission describes the noise as added to extracted landmarks; that description is incorrect, and Section 5.5 is corrected to state the stage of each perturbation, with the abstract's robustness claim rewritten. No comparison against other methods under identical pixel-level corruptions was run, and the UMCL comparison is removed because the input representations and perturbations differ.

Thank you for the additional experiments. The improvement over LRNet is encouraging, but the ablations show no clear benefit from e.g., lag-2 differences or folded windows, as noted in the response to another reviewer. I remain unonvinced that the paper demonstrates a sufficient advance over prior work in the field, and so maintain my rating.

Reviewer_BSPeOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors use facial landmarks and their motion trajectories for deepfake detection. Specifically, they extract 68 facial landmarks and compute differences between the current frame and frames 1 or 2 steps before and after it, aiming to distinguish real and fake videos through temporal asymmetries in landmark motion. The approach is clear, but its novelty over existing landmark-based temporal methods is insufficiently. Furthermore, the experiments lack comparisons with closely related methods, and cross-dataset performance remains weak.

Critical strengths
  1. The approach is clearly motivated and technically feasible.

  2. The trajectory autocorrelation and inter-landmark correlation analyses provide some empirical motivation.

Critical weaknesses
  1. Landmark-based temporal modeling has been widely explored, e.g., LRNet [1]. The paper neither clearly explains its advances over the related methods nor directly compares against them.

  2. When trained on FF++, the model achieves only 66.70% and 59.62% AUC on Celeb-DF V1 and V2, respectively, below the reported baselines. This raises concerns that the learned motion features depend heavily on dataset-specific factors. In addition, Table 2 reports 97.9% AUC on FF++ C23, whereas Tables 3 and 4 report 99.65% for the undegraded/full model. The authors are required to make this number conflict clear.

  3. There is no separate evaluation of lag-2 differences or the folded three-frame window. Moreover, backward differences are shifted, sign-reversed forward differences, so capacity-matched comparisons are needed to establish the benefit of the additional branches.

  4. The specific value of the muscle-inspired topology is not stated. Comparing it only with a fully connected graph cannot distinguish the benefit of the proposed facial connections from that of sparsity alone.

[1] Improving the Efficiency and Robustness of Deepfakes Detection Through Precise Geometric Features. CVPR, 2021.

Questions for authors

Please see weaknesses.

2 comments

Common response: verified records and new runs

We thank the reviewers for their reports. Since submission we audited the archived training logs and completed multiple training runs and fixed-set evaluations. Below we give the verified archived numbers, the new numbers, and the revisions they require. New results are video-level AUC (%), mean ±\pm sample standard deviation over seeds 1--3.

Cross-dataset and unseen generators.

Test collectionVideosFull DiffMotionRepeated lag-1LRNet, common protocol
FaceShifter (held-out source IDs)28069.02 ±\pm 2.3469.15 ±\pm 2.9356.81 ±\pm 1.23
DF40 BlendFace81770.94 ±\pm 1.6870.64 ±\pm 1.4367.38 ±\pm 2.22
DF40 DeepFaceLab14972.59 ±\pm 5.4770.20 ±\pm 7.6664.82 ±\pm 1.01
DF40 E4S1,33262.12 ±\pm 1.7565.89 ±\pm 3.8352.23 ±\pm 1.51
DF40 FaceSwap82372.45 ±\pm 1.9571.24 ±\pm 2.3968.40 ±\pm 1.45
DF40 FSGAN80979.09 ±\pm 1.8974.59 ±\pm 1.3068.38 ±\pm 0.69
DF40 HeyGen10016.29 ±\pm 11.2220.45 ±\pm 12.945.77 ±\pm 4.38
DF40 MobileSwap82277.49 ±\pm 0.8577.72 ±\pm 2.5864.59 ±\pm 1.47

Across the seven DF40 subsets, cross-dataset AUC ranges from 62.12 to 79.09. It is above the repeated-lag-1 control on four subsets and below it on E4S, HeyGen and MobileSwap; it is above the adapted LRNet on all seven. HeyGen is 16.29 ±\pm 11.22 against 5.77 ±\pm 4.38 for LRNet, both near or below chance with large variance.

Response to Reviewer BSPe

The two FF++ numbers (W2). Table 2 reports the early checkpoint and Tables 3--4 the late checkpoint of the same 1,200-epoch C23/ALL run: 97.88 at epoch 23 and 99.65 at epoch 684. The archived pipeline had no validation partition and selected checkpoints on the configured test inputs, with a single seed. Every number in the revision comes from a run that selects on a 700-video validation partition and repeats seeds 1--3, and each table states which checkpoint and selection rule it uses.

Cross-dataset results (W2). The external evaluation was re-run under the new protocol; the table above gives the seven DF40 subsets and FaceShifter, with the repeated-lag-1 control and the adapted LRNet on the same inputs. Celeb-DF V1 gives 64.53±2.3064.53 \pm 2.30 (241 videos) and V2 52.83±0.1852.83 \pm 0.18 (518 videos), replacing the submitted 66.70 and 59.62; the difference comes from the landmark collection and the validation-selected checkpoint, not from the model. V2 is at chance, the full model is below the repeated-lag-1 control on E4S, HeyGen and MobileSwap, and it is above the adapted LRNet on all seven DF40 subsets, so the motion features do not transfer uniformly; the abstract and Section 5.4 no longer claim cross-dataset superiority and state this as a limitation.

Ablation limitations (W3, W4). The capacity-matched ablations were run with three seeds and identical partitions, but they cover one within-domain protocol only, and capacity is matched by parameter count rather than by compute. Within that setting the result is negative for most branches: explicit differencing is the only element with a measured gain (about 2.3 AUC over the coordinate-only variant), neither the lag-2 branch nor the folded window shows a measurable gain, and a random sparse graph with the same edge count matches the facial graph, so the specific facial connections are not established as a source of gain. The revision removes the claims for these branches and the muscle-inspired wording, reports the ablations as such, and describes the graph as a sparsity prior. Backward differences are a shifted, sign-reversed forward difference, so they are not counted as an independent motion observation.

Thank you for the detailed response and the additional experiments. I appreciate the clarification of the checkpoint-selection issue and the new evaluation under a validation-based protocol. However, the additional results confirm that cross-dataset generalization remains limited. The capacity-matched ablations also do not show clear benefits. Given these findings and the remaining concern about the novelty of the contribution, I will maintain my original rating.

Reviewer_GKZbOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper proposes DiffMotion, a deepfake detector that operates on facial landmark trajectories. It combines raw coordinates with bidirectional lag-1/lag-2 temporal differences, propagates features over a hand-designed sparse facial graph, and aggregates local context with three-frame sliding windows followed by temporal convolution. It reports strong within-dataset and cross-manipulation results on FF++ and robustness to test-time frame dropping and landmark noise, while cross-dataset AUC on Celeb-DF remains low.

Critical strengths
  1. Using facial motion as a cue complementary to appearance is a reasonable direction. The paper is also candid that cross-dataset generalization remains unsolved.

  2. Working on landmark trajectories keeps the detector lightweight. Its performance degrades gracefully under heavier compression, test-time frame dropping and coordinate noise.

Critical weaknesses
  1. The novelty is limited, and the closest prior work is missing. LRNet already detects deepfakes from landmark sequences: it calibrates landmarks with optical flow and Kalman filtering and models temporal landmark differences in a dedicated stream, with robustness to compression and noise as its main claim. DiffMotion’s preprocessing closely follows this pipeline, yet LRNet is neither cited nor compared. (LRNet: Improving the Efficiency and Robustness of Deepfakes Detection through Precise Geometric Features )

  2. The paper even pads a constant third channel “for skeleton compatibility.” None of this literature is cited. As a result, all three claimed contributions are established components applied to facial landmarks.

  3. The results do not support the claimed significance. Cross-dataset AUC on Celeb-DF-v2 is 59.62, barely above chance and below the 2019 Xception baseline. This undercuts the motivation that motion cues help against increasingly realistic forgeries, and no modern forgeries are evaluated. Within-domain FF++ is saturated, and the reported numbers are inconsistent: 97.9 in Table 2 versus 99.65 in Tables 3–4 for the same setting. The robustness study perturbs landmarks directly and compares against a single method. All results come from a single seed, and the checkpoint-selection procedure is unspecified.

  4. The paper has correctness and presentation problems throughout. The motivation analysis contradicts its own numbers: Section 3.2 claims that differencing increases DTW separability, yet both the t-statistic and the KS p-value become weaker after differencing. Figures 1–2 show largely overlapping real and fake patterns rather than the clear differences described. Section 3.3 and Equation 7 give different cross-region edge sets and use different landmark indexing. The abstract reports AUC as accuracy, claims to significantly outperform existing methods on the basis of a comparison with one method, and claims an interpretability that is never evaluated.

Overall, the paper is not ready for publication. It would need repositioning against prior landmark-based and skeleton-based methods, a substantially stronger evaluation, and a careful rewrite.

Questions for authors
  1. How does DiffMotion differ from LRNet, which also uses calibrated landmark sequences with a temporal-difference stream, and how do the two compare under the same protocol?
  2. Which components are new relative to skeleton-based action recognition networks such as ST-GCN, 2s-AGCN and MS-G3D? Why does Table 2 report 97.9 AUC on FF++ C23 while Tables 3 and 4 report 99.65 for the same setting? How were checkpoints selected given training for up to 1,200 epochs, and are the results stable across seeds?
  3. Given near-chance AUC on Celeb-DF-v2, what evidence supports motion as a cue against recent, more realistic forgeries? How does DiffMotion perform on modern reenactment or talking-head forgeries?
  4. How does DiffMotion perform under pixel-level degradations that affect landmark detection itself, compared with appearance-based detectors and LRNet under identical corruptions?
2 comments

Common response: verified records and new runs

We thank the reviewers for their reports. Since submission we audited the archived training logs and completed multiple training runs and fixed-set evaluations. Below we give the verified archived numbers, the new numbers, and the revisions they require. New results are video-level AUC (%), mean ±\pm sample standard deviation over seeds 1--3.

Cross-dataset and unseen generators.

Test collectionVideosFull DiffMotionRepeated lag-1LRNet, common protocol
FaceShifter (held-out source IDs)28069.02 ±\pm 2.3469.15 ±\pm 2.9356.81 ±\pm 1.23
DF40 BlendFace81770.94 ±\pm 1.6870.64 ±\pm 1.4367.38 ±\pm 2.22
DF40 DeepFaceLab14972.59 ±\pm 5.4770.20 ±\pm 7.6664.82 ±\pm 1.01
DF40 E4S1,33262.12 ±\pm 1.7565.89 ±\pm 3.8352.23 ±\pm 1.51
DF40 FaceSwap82372.45 ±\pm 1.9571.24 ±\pm 2.3968.40 ±\pm 1.45
DF40 FSGAN80979.09 ±\pm 1.8974.59 ±\pm 1.3068.38 ±\pm 0.69
DF40 HeyGen10016.29 ±\pm 11.2220.45 ±\pm 12.945.77 ±\pm 4.38
DF40 MobileSwap82277.49 ±\pm 0.8577.72 ±\pm 2.5864.59 ±\pm 1.47

Across the seven DF40 subsets, cross-dataset AUC ranges from 62.12 to 79.09. It is above the repeated-lag-1 control on four subsets and below it on E4S, HeyGen and MobileSwap; it is above the adapted LRNet on all seven. HeyGen is 16.29 ±\pm 11.22 against 5.77 ±\pm 4.38 for LRNet, both near or below chance with large variance.

Response to Reviewer GKZb

Novelty and the LRNet relation (W1; Q1). LRNet is the closest prior work and was missing from the submission. The revision adds it to Related Work with its optical-flow/Kalman calibration and its coordinate and difference GRU streams, and compares the two methods under identical inputs: the authors' PyTorch LRNet, the same 128-frame clips and the same partitions. Mechanically, LRNet feeds coordinates and one temporal-difference stream through GRUs, while DiffMotion propagates bidirectional lag-1/lag-2 differences over a sparse facial graph and aggregates folded windows. The full model is above LRNet on FaceShifter (69.02 ± 2.34 vs 56.81 ± 1.23) and on all seven DF40 subsets (e.g. FSGAN 79.09 ± 1.89 vs 68.38 ± 0.69; DeepFaceLab 72.59 ± 5.47 vs 64.82 ± 1.01). LRNet's 30-frame block training was not reproduced, so the comparison runs under our input budget.

What is new (W1; Q2). None of the operators is new: differencing, graph propagation and temporal unfolding are established in skeleton action recognition (ST-GCN, 2s-AGCN, MS-G3D, STUT) and in LRNet. The constant third channel (0.1) only pads the tensor to a skeleton-compatible layout. The contribution is the facial-landmark instantiation and its evaluation, and the revision states it that way instead of claiming four components. The component ablation is capacity-matched and runs three seeds; explicit differencing is the only element with a measured gain, and the other elements are reported as ablations without a benefit claim.

The two FF++ numbers (W3; Q2). The 97.9/99.65 difference is the checkpoint stage of one 1,200-epoch C23/ALL run: 97.88 at epoch 23 and 99.65 at epoch 684. The archived pipeline has no validation partition and selected checkpoints on the configured test inputs, with a single seed. Every new run selects on a 700-video validation partition and repeats seeds 1–3; mean and standard deviation are reported for each number. The revision labels each table with its checkpoint and selection rule.

Modern forgeries (Q3). The external evaluation was re-run on DF40 under the new protocol: seven subsets, 4,852 videos. The full model reaches 70.94 ± 1.68 on BlendFace (817 videos), 72.59 ± 5.47 on DeepFaceLab (149), 62.12 ± 1.75 on E4S (1,332), 72.45 ± 1.95 on FaceSwap (823), 79.09 ± 1.89 on FSGAN (809), 16.29 ± 11.22 on HeyGen (100, talking-head) and 77.49 ± 0.85 on MobileSwap (822); FaceShifter gives 69.02 ± 2.34. Celeb-DF V2 gives 52.83 ± 0.18, at chance. These numbers do not support motion as a cue against recent synthesis; the abstract and Section 5.4 are rewritten, and the cross-dataset gap is stated as a limitation.

Degradation (Q4). The two perturbations act at different pipeline stages. Frame dropping (30% retained frames) removes frames from the landmark sequences after extraction and calibration, so it isolates the sequence model. Gaussian noise is added to the video frames before landmark extraction, so the perturbation passes through face detection, landmark extraction and calibration before reaching the model; this is the experiment that exercises the full pipeline under pixel-level corruption. The submission describes the noise as added to extracted landmarks; that description is incorrect, and Section 5.5 will be corrected to state the stage of each perturbation, with the abstract's robustness claim rewritten.

I thank the authors' effort in rebuttal. However, the additional DF40 results confirm rather than resolve my concern about significance. Under the standard cross-manipulation setting, DiffMotion reaches 70.9 on BlendFace, 77.5 on MobileSwap, 62.1 on E4S and 79.1 on FSGAN, an average of 72.4. Published detectors evaluated on the same four subsets are far ahead: Effort (ICML 2025) reports 87.3, 95.3, 98.3 and 95.7, and ForensicsAdapter (CVPR 2025) reports 88.8, 98.1, 97.1 and 97.7, i.e. averages of 94.2 and 95.4. The gap is 16 to 36 AUC points on every subset and more than 20 points on average, which is too large to attribute to differences in preprocessing or evaluation protocol. On HeyGen, the only talking-head forgery in the table, the AUC is 16.3, meaning the model systematically ranks fake videos as more real than genuine ones. With cross-manipulation performance this far below the current state of the art, I do not think the paper meets the bar for acceptance, and I maintain my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.