acceptodds
Under review as a conference paper at ICLR 2027

Let Detectors Talk: Detector-Guided Data Selection for Deepfake Detection

Abstract

Deepfake detectors often struggle to generalize to unseen forgery patterns. We investigate whether adapting the use of available training data to a detector’s evolving weaknesses can improve generalization. To this end, we alternate detector training with the selection of samples for subsequent training stages. The difficulty is deciding what to select: although labeled samples reveal where the current detector struggles, similar prediction scores may conceal different failure patterns and offer limited guidance on how to address them. We use the samples the detector finds difficult as anchors for identifying its current weaknesses. To characterize these anchors beyond the detector’s own scores, we select auxiliary detectors based on their detection performance and response redundancy. Their responses provide multiple views of both the anchors and the samples available in the training pool. We then select additional training samples that exhibit similar response patterns to the anchors, retrain the target detector, and repeat the process with updated anchors. Experiments on six cross-domain test sets demonstrate the effectiveness of our method and show competitive performance.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Deepfake Detection, Data Selection

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_mFZTOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes an iterative data selection procedure for improving deepfake detectors. Misclassified images in a labeled diagnosis set serve as hard anchors. Auxiliary detectors are selected for their accuracy and response diversity, and each image is represented by normalized differences between expert and target-detector scores. Same-label candidates nearest to these anchors are added to the training set before the classifier is retrained. Experiments use frozen image encoders, several target backbones, and six external test sets, with a default budget of 250,000 additional training images.

Critical strengths
  1. Using complementary detectors to characterize difficult examples is a reasonable idea. The procedure can use existing labeled data and pretrained experts, while requiring only the target detector at inference.

  2. The matched comparisons provide evidence that expert responses are useful for selection.

Critical weaknesses
  1. The methodological contribution appears incremental, and the distinctive response-difference representation has limited support. The framework combines error-based anchor identification, expert screening, and nearest-neighbor retrieval. Table 3 shows that directly using expert scores already achieves 90.3 external average AUC, while the proposed differences reach 90.7. Most of the improvement therefore comes from introducing expert information, with a relatively small additional benefit from the central representation choice. Moreover, proximity in a low-dimensional score space does not establish that two samples share an underlying failure mode or that training on one will correct the other. The reported gains are not accompanied by analysis connecting selected samples to specific weaknesses being repaired. Since all target encoders remain frozen, the demonstrated scope is also restricted to classifier retraining on fixed representations.

  2. The baseline coverage is insufficient to establish the value of the selection procedure. "Rand. / score" randomly chooses anchors and then retrieves by target-score distance; it is not random sampling of training candidates. The reported comparisons omit direct class-balanced random sampling, source-stratified sampling, and retrieval using the already available frozen image features. Consequently, outperforming the tested one-dimensional score-retrieval variants leaves the advantage over straightforward alternatives unresolved. In addition, expert responses may encode dataset or generator preferences, so selection could improve performance largely by changing the source composition of the training set. No analysis or control matches that composition while randomizing selection within each source. Such a comparison would distinguish source rebalancing from the proposed sample-level matching benefit. A full-pool training reference would further contextualize the utility of repeated selection, given the available labels and cached DINO features.

Questions for authors
  1. How does the method compare with direct class-balanced random sampling and frozen-feature nearest-neighbor retrieval under the same addition budget and retraining schedule?

  2. Does the improvement persist when compared against random selection matched to the proposed method's per-source and per-class sample counts?

  3. What exactly is held fixed in Appendix Table 10's "Dynamic - fixed" comparison, and are both variants matched in added samples and retraining schedule?

Reviewer_aqDEOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how to select additional training examples from existing labeled data to improve a deepfake detector. Misclassified diagnosis examples serve as anchors. Auxiliary detectors and the current detector provide response-difference representations for retrieving candidates with the same label. The authors alternate sample selection and classifier retraining while keeping the image backbone frozen. Across six external test sets, the full method reaches an average AUC of 90.7%, compared with 86.9% for retrieval using the target detector's scores.

Critical strengths

Improving an existing detector by selecting from available training data is a useful practical problem. The gains over target-score retrieval appear across multiple external test sets and target backbones. These results provide empirical support for using complementary detector responses to guide data selection.

Critical weaknesses

1.The contribution of response differences remains unclear. Under the same anchor-selection rule, Table 3 reports an external average AUC of 90.3% for expert-score retrieval and 90.7% for response-difference retrieval, a gain of only 0.4 percentage points. Most of the improvement is already present with expert responses, and the consistency of the additional gain needs verification.

2.The selection baselines are incomplete. “Rand./score” randomizes anchors and still retrieves candidates by detector score, leaving out uniform candidate sampling under the same budget. The uncertainty and high-loss variants defined in Appendix A.4 also lack reported results. The method's advantage over these baselines remains insufficiently established.

3.Regarding presentation, I recommend manually redrawing conceptual diagrams such as Figs. 1 and 3 as vector graphics. Although the authors disclose using AI for writing, figures, and discussing results, the paper's presentation still needs improvement, and I recommend that they personally review and revise the full manuscript. The main text should also include more important content, as it does not fully use the 9-page allowance; the AI use statement and similar sections are excluded from this limit.

Questions for authors

1.Can you provide paired results for response-difference retrieval and direct expert-score retrieval using the same random seeds? How consistent is the additional gain, and how do these results support the contribution of the response-difference design?

2.Can you report class-balanced uniform sampling of candidate images under the same sample budget and training protocol, along with the uncertainty and high-loss variants defined in the appendix?

Reviewer_8ff9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

In this paper, the authors propose a hard-sample-inspired data selection strategy to improve the generalization of deepfake detectors. Specifically, they use samples misclassified by the current detector as anchors and retrieve additional training samples from an external labeled data pool that exhibit similar detector-response patterns. The selected samples are added to the training set, and the detector is retrained iteratively. Experiments report performance improvements after adding 250K training images.

Critical strengths
  1. The paper explores a useful direction for improving generalization by selecting training data according to a detector’s current weaknesses.

  2. Experiments across multiple test sets and target backbones provide evidence of the method’s effectiveness.

Critical weaknesses
  1. The method requires 50 rounds of data selection and classifier retraining, alongside auxiliary-detector inference and response comparisons. Although the backbone is frozen and image features are reused in the DINOv3 experiments, the cost of this iterative procedure may be inpractical. Appendix A.5 reports aggregate computational costs but does not isolate the overhead of the proposed selection method relative to simpler alternatives.

  2. The comparisons with existing detectors in Table 1 do not isolate the contribution of the proposed data selection strategy. These detectors differ in backbone, training data, and training protocol, while the proposed approach uses a large pretrained backbone and a different training pool. Evaluating FF++-trained detectors such as Effort_FF on general AI-generated imagery also introduces a substantial domain mismatch. Furthermore, the initial model already outperforms all listed existing detectors in average AUC, so the final performance advantage cannot be attributed to data selection alone. Although the paper includes experiments with different target backbones, it does not establish whether the proposed strategy outperforms competing training or augmentation methods under the same backbone, data access, and training budget.

  3. A direct random-data-addition baseline is missing. The “Rand. / score” baseline randomly selects anchors and then retrieves candidates according to prediction-score distance. Since the training set grows from 29,356 to 279,356 images, a class-balanced random-addition baseline with the same addition schedule and retraining protocol is essential.

  4. The evidence supporting the specific response-difference formulation is limited. Table 3 shows that matching candidates using expert scores alone already achieves an external average AUC of 90.3%, compared with 90.7% for the full method and 86.9% for current-detector score matching. Therefore, most of the observed gain comes from introducing expert responses, while subtracting the current detector’s response provides a comparatively small additional improvement. Moreover, similarity in response patterns does not by itself establish that the selected samples address the same underlying failure mode. Further analysis is needed to justify the response-difference representation and substantiate the interpretation that it targets the detector’s evolving weaknesses.

Questions for authors

Please see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.