- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes COVE, a framework for improving deepfake detection by combining visual evidence with multimodal language model reasoning. It introduces continuous visual states supervised by three pretrained visual models to capture subtle image details. A counterfactual reward encourages the model to use these visual states when making decisions, while contrastive training aims to keep its reasoning grounded in the image.
Critical strengths
-
Using different models to supervise continuous visual states is a reasonable way to combine different types of visual information, including appearance, semantics, and boundaries.
-
The proposed counterfactual reward is interesting because it goes beyond learning visual features and explicitly encourages the model to use them when making authenticity decisions.
-
The experiments cover several types of distribution shifts, and the ablation studies provide useful evidence that the visual experts, counterfactual reward, and reasoning refinement contribute to the improvements.
Critical weaknesses
-
The counterfactual reward replaces forensic token embeddings with an average vocabulary embedding, which may introduce unnatural inputs rather than isolate the contribution of meaningful visual evidence.
-
The contrastive training pairs are created using generated images, but it is unclear whether they preserve enough visual details to ensure that authenticity is the main difference between the two images.
-
The paper mainly evaluates reasoning quality using other language models as judges, without directly verifying whether the visual evidence described in the explanations is actually present in the images.
Questions for authors
-
How can the authors verify that the counterfactual reward measures the actual contribution of forensic evidence, rather than the effects of replacing tokens with unnatural embeddings?
-
Can the authors provide a more direct evaluation of whether the generated explanations are supported by real visual evidence, rather than simply being plausible explanations associated with correct predictions?
8 comments
W3 / Q2: Evaluation of visual evidence in model explanations
We thank the reviewer for the suggestion. For a fair comparison, we follow the evaluation protocols of Veritas[1] and AIGI-Holmes[2], both of which emphasize interpretability and the grounding of visual evidence in the input image. The judge model receives the image, the ground-truth label, and the complete response, and is explicitly instructed to consider whether the evidence corresponds to the image. Thus, the existing evaluation includes visual grounding in its scoring criteria. The prompt template is provided in Appendix D of the revised manuscript.
To further assess the factual correctness of the explanations, we add a comparison using an additional external judge and a newly designed prompt template, also provided in Appendix D. This prompt assigns separate scores from 1 to 5 for Visual Grounding, Evidence Specificity, and Reasoning Soundness. We average the three scores to obtain the final score for each sample, placing greater emphasis on whether the model’s reasoning contains actual visual evidence.
Table 2. Evaluation of reasoning quality based on visual evidence.
| Model | Score Evaluation (GPT-6-Sol) |
|---|---|
| MiMo-VL-7B | 2.901 |
| GPT-4o | 2.665 |
| Gemini-2.5-Pro | 3.134 |
| Veritas | 4.015 |
| COVE (cold-start) | 3.489 |
| COVE (base stage) | 4.041 |
| COVE (ours) | 4.151 |
As shown in Table 2, COVE continues to achieve a high score with the new prompt template and the more advanced external judge, GPT-6-Sol, indicating that its reasoning is supported by actual evidence from the corresponding image for most samples. Appendix D of the revised manuscript provides an example of the external judge’s output.
[1] Veritas: Generalizable Deepfake Detection via Pattern-Aware. ICLR 2026.
[2] AIGI-Holmes: Towards Explainable and Generalizable AI-Generated Image Detection via Multimodal Large Language Models. ICCV 2025.
Dear Reviewer, The authors have provided their responses to the raised concerns and revisions. Please review their responses and join the discussion before the end of the author-review discussion phase (Nov 18, AoE). Thanks.
Best, AC
Response to Reviewer Bk6H
Dear Reviewer Bk6H,
We sincerely thank you for the time and effort you have devoted to reviewing our paper. Your constructive feedback is invaluable in helping us improve the clarity and rigor of our work. Below, we provide point-by-point responses to your comments. We have also incorporated all corresponding improvements into the revised manuscript, with all changes highlighted in blue for ease of reference. Please feel free to let us know if you have any further concerns or questions.
W1 / Q1: CTUR with alternative replacement strategies
We thank the reviewer for pointing out that the average vocabulary embedding may differ in distribution from the original forensic tokens. To examine whether the training gains from CTUR depend on this particular replacement strategy, we add two counterfactual interventions: sampling from the embedding distribution of each expert’s tokens, and replacing forensic states with those from trajectories generated for other images. These experiments examine the sensitivity of CTUR to the source of the replacement vectors. We use each replacement strategy to compute CTUR in a separate GRPO training run, and evaluate the resulting models on normal inputs without intervention.
Table 1. CTUR training results with different counterfactual interventions (%). Exp1: sampling from expert-specific token embedding distributions; Exp2: substituting states from another image; Exp3: average vocabulary embedding (Ours).
As shown in Table 1, the three interventions achieve similar detection performance, with overall accuracy ranging from 93.0% to 93.1%. This represents an improvement of 1.8%–1.9% over the 91.2% achieved without CTUR. These results show that the training gains from CTUR are observed across different replacement strategies and are not limited to the average vocabulary embedding.
Given their similar performance, we use the average vocabulary embedding as the default implementation because it can be computed in advance and reused directly. It requires neither additional distribution sampling nor trajectory generation for other images, making the replacement procedure simpler. We have included these comparisons in the Appendix H.
W2: Semantic consistency of paired images
We thank the reviewer for raising this concern. For the contrasting image conditions, we use pairs of real source images and their corresponding fake images from the HydraFake training set. We pair each fake image with its real source image using the existing source records. This provides a traceable basis for their content correspondence while reducing unrelated visual differences. The refinement stage uses only the existing HydraFake training data and does not generate additional data. The content correspondence between the paired training samples follows from their generation or manipulation relationship, which helps reduce the influence of unrelated differences in subjects and scenes on the objective. We have corrected the descriptions of generating new images in Section 3.4 and Appendix A.3.