- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an iterative data selection procedure for improving deepfake detectors. Misclassified images in a labeled diagnosis set serve as hard anchors. Auxiliary detectors are selected for their accuracy and response diversity, and each image is represented by normalized differences between expert and target-detector scores. Same-label candidates nearest to these anchors are added to the training set before the classifier is retrained. Experiments use frozen image encoders, several target backbones, and six external test sets, with a default budget of 250,000 additional training images.
Critical strengths
-
Using complementary detectors to characterize difficult examples is a reasonable idea. The procedure can use existing labeled data and pretrained experts, while requiring only the target detector at inference.
-
The matched comparisons provide evidence that expert responses are useful for selection.
Critical weaknesses
-
The methodological contribution appears incremental, and the distinctive response-difference representation has limited support. The framework combines error-based anchor identification, expert screening, and nearest-neighbor retrieval. Table 3 shows that directly using expert scores already achieves 90.3 external average AUC, while the proposed differences reach 90.7. Most of the improvement therefore comes from introducing expert information, with a relatively small additional benefit from the central representation choice. Moreover, proximity in a low-dimensional score space does not establish that two samples share an underlying failure mode or that training on one will correct the other. The reported gains are not accompanied by analysis connecting selected samples to specific weaknesses being repaired. Since all target encoders remain frozen, the demonstrated scope is also restricted to classifier retraining on fixed representations.
-
The baseline coverage is insufficient to establish the value of the selection procedure. "Rand. / score" randomly chooses anchors and then retrieves by target-score distance; it is not random sampling of training candidates. The reported comparisons omit direct class-balanced random sampling, source-stratified sampling, and retrieval using the already available frozen image features. Consequently, outperforming the tested one-dimensional score-retrieval variants leaves the advantage over straightforward alternatives unresolved. In addition, expert responses may encode dataset or generator preferences, so selection could improve performance largely by changing the source composition of the training set. No analysis or control matches that composition while randomizing selection within each source. Such a comparison would distinguish source rebalancing from the proposed sample-level matching benefit. A full-pool training reference would further contextualize the utility of repeated selection, given the available labels and cached DINO features.
Questions for authors
-
How does the method compare with direct class-balanced random sampling and frozen-feature nearest-neighbor retrieval under the same addition budget and retraining schedule?
-
Does the improvement persist when compared against random selection matched to the proposed method's per-source and per-class sample counts?
-
What exactly is held fixed in Appendix Table 10's "Dynamic - fixed" comparison, and are both variants matched in added samples and retraining schedule?