- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes Real Distribution Bias Correction (RDBC), a framework for generalisable deepfake detection built on Effort (CLIP ViT-L/14). A key finding is that frozen pretrained weights do not freeze their responses, allowing weakly constrained tail channels to work against the correct class on unknown domains. RDBC calibrates the channel space using statistics from real images via Real-Kurtosis-Guided Channel Alignment and Real-Covariance-Guided Channel Enhancement. After training on FF++, the model is tested on six datasets along with ablation studies, robustness tests, and a drift analysis.
Critical strengths
-
The paper makes a useful observation that frozen weights do not imply frozen output responses. This addresses parameter-preserving fine-tuning methods, which remains underexplored. The channel-wise analysis in Fig. 1 translates this into a testable hypothesis. This is the main contribution of the paper, since the perspective could be relevant beyond deepfake detection.
-
RDBC requires no target-domain data and adds little overhead, but still improves over Effort in all reported settings, in some settings by a large margin (e.g. 6.62% AUC on DFDC at video level). The ablation in Table 5 shows that both modules contribute individually, which supports the practical value of the method.
Critical weaknesses
-
The main results are not comparable to the state of the art. The central claim relies on a reimplementation of Effort that falls short compared to the original Effort paper, by almost 10 AUC points on DFDC at video level. RDBC stays below the published Effort results at nearly all points. It is therefore hard to say whether RDBC actually improves on the state of the art. Section 4 states that the same protocol as the backbone is used, but the reported setup differs, which also makes the values taken from other papers hard to compare. The origin of the numbers is sometimes unclear and is even mixed within rows. Finally, all results come from a single seed without variance, while some margins are below one point. This is critical, because state of the art cross-domain performance is the main claim of the paper.
-
It is not shown that the gains actually come from correcting tail-channel mismatch. The mismatch is never clearly defined, and the tail improvements in Table 7 are tiny, even smaller than in the head channels, and measured on one domain without error bars. The arguments for the modules are also weak. LayerNorm does not change kurtosis, so RKA only reacts to the frozen scaling parameters, and kurtosis is driven by extreme channels rather than the tail. The separability claim for RCE holds for any rescaling of the features. Without further evidence, what remains are two regularizers that help Effort, which is much less than the paper acually claims.
Questions for authors
Clarification:
- Why does your reimplementation of Effort fall below the published results, by almost 10 AUC points on DFDC at video level? Could you report RDBC on top of Effort under the original protocol?
- Please clarify for every entry in Table 2 and 3 whether it was reimplemented or copied from another paper.
- Could you report mean and standard deviation over multiple seeds for the main results and the ablation in Table 5?
- What are the pre- and post-LayerNorm kurtosis values for the unmodified pretrained CLIP? Since LayerNorm itself does not change kurtosis, what does RKA constrain, and how does it affect the tail channels specifically?
- How does RDBC compare to MixStyle and simple feature distillation toward frozen CLIP on the same backbone and protocol?
Suggestions for future work:
- Evaluation on general AI-generated image benchmarks such as GenImage or UniversalFakeDetect.