- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper proposes a method to improve the detection of blackbox generators with a limited number of synthetic images called Forensic Proxy Generation (FPG). It adapts a pretrained student generator using latent and frequency alignment with the teacher blackbox generator, as a way to generate proxy images using the student generator to increase data quantity. These proxy images are then used to train the detector. The method is evaluated using six target generators, with additional experiments using eleven detectors. It shows that the proxy datasets can improve detection over direct training with a few target images.
Critical strengths
- Some of the experiments are interesting. Using only 10 or 25 target images to produce larger datasets that improve detection suggests that the method could be useful when collecting synthetic images is difficult.
- The empirical evaluation covers multiple generators and detector architectures. The improvements across these settings show that the proxy images can be useful for adapting existing detectors.
Critical weaknesses
- The paper does not show why this specific method is actually useful. Training a generator to mimic a target generator seems like a superfluous step instead of simply adapting detector training. The paper shows that expending the small dataset with the proxy generators can help, but it does not compare against a non-adapted student (does simply adding training data help the detector?) nor compare against a fine-tuned detector trained with the same number of target image (what is the degradation?). One hypothesis could be that the gains come from the student's pre-trained knowledge and the additional variation it provides. It is also unclear how the student can correctly mimic the target generator as we known from prior work that the model’s architecture plays a large role in the unseen artifacts a detector can pick up. These missing controls make it unclear whether training the student provides a meaningful benefit.
- The evaluation is also restricted to face images. This is a useful setting, but it gives little evidence that the method would work on full images with different content, multiple faces, or partial edits, which I would argue are the much more common types of deepfakes. Since the method explicitly matches frequency characteristics, which is often much different on different image types, it is important to establish whether the improvements depend on the single-face portraits used in these experiments.
Questions for authors
- Can you compare fine-tuning on 1,000 Nano Banana images with fine-tuning on 1,000 proxies, using the same detector initialization, augmentation, real-image data, and number of updates?
- Can you report results using the pretrained student without teacher-guided adaptation, with the same prompts and image counts, both with and without Stage 2?
- What augmentation and number of updates were used for direct training on the 10 or 25 target images in Figure 4? Were these matched to the proxy-training conditions?
- Can you compare with simply reconstructing images through the student's VAE? This would help understand whether decoder artifacts explain some of the improvements.