- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper highlights potential biases in existing AI-generated video detection datasets that may encourage detectors to learn shortcuts. The authors show experimentally that some detectors are highly sensitive to particular motion patterns in the evaluation dataset and perform poorly on a dataset without this motion bias. They also evaluate WaveRep, a frequency-domain baseline, and find that it substantially outperforms motion-based detectors.
Critical strengths
The paper addresses an important research question by directly examining shortcut learning in existing methods. Its focus on detector generalization has practical value for assessing the reliability of AI-generated content detection.
Critical weaknesses
1 Some details of the experimental design need improvement. In Table 6, AI-generated and real videos are filtered at different rates, which may introduce a distributional bias and complicate the comparison. Filtering both classes at the same rate would provide a better controlled evaluation.
2 The paper’s incremental contribution appears limited. Shortcut learning is already a recognized challenge in AI-generated content detection [1] [2]. Although the paper seeks to determine how much detection performance stems from dataset and preprocessing biases, its central contribution may amount to experimentally confirming a phenomenon that is already well known.
[1] Fang, Ziyuan, et al. "Real Data Lies: Unveiling and Closing the Quality Shortcut in Generalizable AI-Generated Video Detection." Forty-third International Conference on Machine Learning. 2026.
[2] Smeu, Stefan, et al. "Circumventing shortcuts in audio-visual deepfake detection datasets with unsupervised learning." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.
Questions for authors
1 Could the authors clarify the paper’s contribution, particularly how it differs from prior work that has identified shortcut learning in AI-generated video detection?
2 The authors find substantial sampling and preprocessing biases in three of the four evaluated detectors. Could the authors explain why similar biases were not observed for Over-Coherence?
Dear reviewers,
The discussion phase is nearing its end; could you please respond to the authors?
Thank you & kind regards,
Your AC