- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes DiffMotion, a deepfake detection method based on facial landmark movements. It combines raw landmark coordinates with forward and backward temporal differences, uses a sparse graph based on facial anatomy, and processes the resulting features with folded temporal windows and temporal convolutions. The method is evaluated on FaceForensics++ and Celeb-DF, including cross-manipulation, cross-dataset, and degradation robustness experiments.
Critical strengths
The method achieves some good cross-manipulation results on FaceForensics++, with the best average among the compared methods for three of the four training subsets. The degradation experiments are also interesting, and suggest that the model can tolerate quite substantial perturbations to the landmark sequences.
I also appreciate that the authors acknowledge the poor cross-dataset generalisation performance.
Critical weaknesses
- Limited novelty and insufficient ablations.
I am not convinced that the proposed method offers sufficient novelty. The related work section also seems quite sparse, particularly given the proposed contributions. For example, LRNet (Sun et al., CVPR 2021) already explores facial landmarks and temporal dynamics for deepfake detection. STUT (Cui and Hayama, 2023), although proposed for skeleton-based action recognition, also uses forward/backward temporal differences and temporal unfolding with structured spatial attention. The related work section would benefit from more relevant papers to better position the paper's contribution.
The ablations are also insufficient to establish the importance of the individual components. For example, folded temporal windows are presented as one of the main contributions, but there is no ablation removing them. The facial graph is only compared against a fully connected graph, rather than another sparse graph, so it is unclear whether the specific anatomical connections actually matter.
I also find the interpretation of the temporal differencing ablation questionable. The authors argue that removing temporal differences leaves static geometry that cannot capture temporal inconsistencies. However, the model still receives a sequence of landmark coordinates and contains temporal modelling layers. It is therefore not obvious why it should be unable to learn motion information. The very large drop from 99.65 to 70.58 AUC needs further explanation.
Overall, I do not think the paper sufficiently establishes what is new about the proposed architecture, or which of its design choices are actually responsible for the reported improvements.
- Weak experimental evaluation and limited evidence of generalisation.
The evaluation is restricted to FaceForensics++ and Celeb-DF, which are quite old benchmarks at this point. Given the rapid progress in deepfake generation, I would expect experiments on more recent and diverse datasets, especially for a paper that motivates its method by the increasing realism of generated videos.
The cross-dataset results are also quite poor. The model reaches only 66.70 and 59.62 AUC on Celeb-DF V1 and V2, substantially below the compared methods. I do not think the results establish a competitive deepfake detector overall, despite some good performance on the FF++ cross-manipulation experiments.
There are also some inconsistencies in the reported results. For example, Table 2 reports 97.9 AUC on FF++ C23, whereas Tables 3 and 4 report 99.65 for what appears to be the same model and dataset. I could not find an explanation for this.
Finally, the robustness experiments are not entirely convincing. Frame dropping and Gaussian noise are applied to already extracted landmarks, rather than to the original videos. This does not account for failures in face detection, landmark extraction, or tracking caused by degraded input videos. The comparison with UMCL is also difficult to interpret because the methods use different input representations and the perturbations are not equivalent. I therefore do not think these experiments establish the claimed robustness advantage over existing approaches.
Questions for authors
Why does the FF++ C23 result differ between Table 2 (97.9) and Tables 3/4 (99.65)? Are these different checkpoints or evaluation settings?
How does the proposed method differ from existing landmark-based approaches such as LRNet? I would also like to see an ablation of the folded temporal windows and a comparison with another sparse graph to better understand which components are actually important.
Can the authors clarify what remains in the model when temporal differencing is removed? Given that temporal convolutions are still present, why does removing explicit differences cause such a large performance drop?
Can the authors evaluate robustness by applying degradations to the input videos before landmark extraction, ideally under the same conditions as other landmark-based methods? It would also be useful to see results on a more recent deepfake dataset.
2 comments
Thank you for the additional experiments. The improvement over LRNet is encouraging, but the ablations show no clear benefit from e.g., lag-2 differences or folded windows, as noted in the response to another reviewer. I remain unonvinced that the paper demonstrates a sufficient advance over prior work in the field, and so maintain my rating.
Common response: verified records and new runs
We thank the reviewers for their reports. Since submission we audited the archived training logs and completed multiple training runs and fixed-set evaluations. Below we give the verified archived numbers, the new numbers, and the revisions they require. New results are video-level AUC (%), mean ± sample standard deviation over seeds 1--3.
Cross-dataset and unseen generators.
Across the seven DF40 subsets, cross-dataset AUC ranges from 62.12 to 79.09. It is above the repeated-lag-1 control on four subsets and below it on E4S, HeyGen and MobileSwap; it is above the adapted LRNet on all seven. HeyGen is 16.29 ± 11.22 against 5.77 ± 4.38 for LRNet, both near or below chance with large variance.
Response to Reviewer e8nn
The two FF++ numbers (Q1). Table 2 reports the early checkpoint and Tables 3--4 the late checkpoint of the same 1,200-epoch C23/ALL run: 97.88 at epoch 23 and 99.65 at epoch 684. The archived pipeline had no validation partition and selected checkpoints on the configured test inputs with a single seed; every number in the revision uses a 700-video validation partition and seeds 1--3.
Novelty and related work (Q2). The revision adds LRNet (optical-flow/Kalman calibration, coordinate and difference GRU streams) and STUT, which uses forward/backward differences and temporal unfolding with structured spatial attention, and compares with LRNet under identical inputs: the full model is above LRNet on FaceShifter and on all seven DF40 subsets (table above). The contribution is stated as the facial-landmark instantiation and its evaluation; none of the operators is presented as new.
Evaluation breadth (Q4). The external evaluation now covers DF40 (seven subsets, 4,852 videos) and FaceShifter in addition to Celeb-DF: cross-dataset AUC ranges from 62.12 to 79.09 on DF40 and FaceShifter is 69.02 ± 2.34. Celeb-DF V1 is 64.53 ± 2.30 and V2 52.83 ± 0.18, replacing 66.70 and 59.62. These results do not establish a competitive general detector on Celeb-DF, and the revision says so; the abstract and Section 5.4 no longer claim cross-dataset superiority.
Robustness (Q4). The two perturbations act at different pipeline stages. Frame dropping (30% retained frames) removes frames from the landmark sequences after extraction and calibration, so it isolates the sequence model. Gaussian noise is applied to the video frames before landmark extraction, so the perturbation passes through face detection, landmark extraction and calibration before reaching the model; this is the experiment that exercises the full pipeline under pixel-level corruption. The submission describes the noise as added to extracted landmarks; that description is incorrect, and Section 5.5 is corrected to state the stage of each perturbation, with the abstract's robustness claim rewritten. No comparison against other methods under identical pixel-level corruptions was run, and the UMCL comparison is removed because the input representations and perturbations differ.