- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces CounterFake, an MLLM-based deepfake detector designed to generalize across unseen image domains and forgery methods on both real and fake images. It combines reference-guided forensic annotation (ReForensics), textual and visual counterfactual preference optimization, weight-space interpolation, and intermediate-layer linear probing. Extensive experiments demonstrate the effectiveness and superiority of the proposed method compared with existing methods.
Critical strengths
- Well-motivated research problem. The paper explicitly addresses distribution shifts affecting both real and fake images, which should be more severe for real image detection I believe. This is a critical issue in the deepfake detection area.
- Well-motivated optimization method. Reference-guided annotation and complementary textual/visual preference objectives offer a coherent approach to aligning forensic descriptions with visual evidence for both real and fake images.
- Extensive experiments with recent baselines demonstrate the effectiveness of the proposed method.
Critical weaknesses
- Methodological novelty. Although I acknowledge the effectiveness and contribution of the designed modules, such as ReForensics, counterfactual preference losses, weight-space interpolation, and intermediate representation probing, they make contributions like combinations of established techniques. The authors should justify their contributions and explain how each module contributes to their contributions.
- Potential imbalance during optimization. I acknowledge that the ReForensics and counterfactual preference losses should benefit the detection for both real and fake images, but I doubt there may be potential imbalance introduced during optimization, since by constructing counterfactual pairs, the negative pairs can be constructed from many different combinations, such as many unpaired fake clues, which means the negative space should be larger and have more directions in latent space compared to positive pairs. How did the authors address this issue during optimization?
- Post hoc design. I believe the introduction of weight-space interpolation and intermediate representation probing can improve detection performance since there are already many support references. However, I cannot understand why the authors specifically introduce these modules after fine-tuning. It feels like 'extra parts' outside the main claim of this paper. More explanations are needed.
- Insufficient attribution of performance gains. As stated above, there are many technical modules introduced. In Table 3, preference optimization improves macro-F1 over SFT by 3.74%, whereas removing interpolation or probing reduces performance by 7.03% and 8.86%, respectively. Does this mean the post hoc designs are more important than the main claim 'counterfake'? More justifications are needed.
Questions for authors
Please see the weaknesses above. My major concerns focus on the method novelty, technical design, and performance attribution. Thus, I currently lean towards a negative rating. Addressing these issues should make this paper clearer and more convincing. I hope the authors address these issues during rebuttal.