- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces source-matched original, pipeline-only, self-swap, and cross-identity-swap controls to characterize deepfake detector responses. Evaluation of 29 checkpoints shows that strong conventional detection AUC can coexist with sensitivity to non-generative processing and weak cross-versus-self-swap discrimination. Controlled Xception training interventions and a generated-video case study demonstrate how supervision and processing settings affect these responses.
Critical strengths
1.The matched controls expose behavior that conventional real/fake AUC leaves unresolved. Explicitly distinguishing synthesis detection from identity discrimination makes the protocol useful and its interpretation careful.
2.Broad checkpoint coverage, native inference pipelines, compression checks, and tightly matched training interventions provide substantial evidence for the central diagnostic claims.
Critical weaknesses
1.Evidence for practical training improvements is limited to one Xception system. Gains transfer unevenly across generators, and the intervention models perform near chance on standard FaceForensics++ manipulations. This limits conclusions about improving broadly useful detectors.
2.Construction code, inference harnesses, and predictions are unavailable during review. Because the results depend on precise processing choices, this limits independent verification of the central empirical contribution.
Questions for authors
1.Does the self-swap relabeling gain persist when effective class balance and loss contributions are controlled? Relabeling also halves the positive fraction, leaving its contribution unresolved.