- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
Building upon BAGEL and UniGenDet, this paper trains a model to simultaneously perform AI-generated image detection and image generation. In Stage 1, the generator's single-step flow (velocity) error on the input image is fed into the detection head. In Stage 2, the detector is frozen, and generator features are aligned with detector features using time-varying weights. Detection performance is evaluated on FakeClue, DMImage, and ARForensics, while generation quality is assessed using GenEval and FID.
Critical strengths
- Using the model's own flow path as a detection cue is a simple yet sound idea. The comparison with UniGenDet was conducted under controlled conditions, with identical initialization, data, and computational budgets.
- On the ARForensics benchmark, the worst-case accuracy is significantly better than that of UniGenDet (97.5 vs. 89.4 on LlamaGen).
Critical weaknesses
-
The gains in detection performance are modest, and the evaluation scope is limited. Compared to UniGenDet, the improvement is +1.0 on FakeClue and +0.1 on ARForensics; notably, UniGenDet outperformed the proposed method on 5 out of the 7 generators in the ARForensics suite. A simple sweep of the detection threshold () could yield similar fluctuations in accuracy. No random seeds were reported. The only evidence supporting the "flow-error" feature claim is the comparison with UniGenDet, yet it is unclear whether the UniGenDet figures in Tables 2–3 stem from retraining or the use of original weights (though the Table 2 caption specifies "original model weights"). Key benchmarks such as GenImage, UniversalFakeDetect, and Chameleon were omitted, as were tests involving JPEG compression and resizing—factors critical for methods relying on reconstruction error cues.
-
Table 2 is difficult to interpret. By reverse-engineering the real/fake ratios from the "Real," "Fake," and "Overall" accuracy figures, I estimate that real images constitute approximately 78% of the UniGenDet set but only about 25% of the SynerGenDet set. Since SynerGenDet’s accuracy on real images is actually lower, the reported overall gain is heavily influenced by this class distribution.
-
The benefits on the generation side were not rigorously controlled. Most of the FID improvement (22.5 → 18.0) occurred during Stage 1; however, Stage 1 involved fine-tuning on 80K LAION images, and the FID was subsequently evaluated using LAION prompts. GenEval scores remained virtually unchanged (an average of 0.877 across six categories versus BAGEL’s 0.875). I would like to see the results of simply fine-tuning BAGEL on the same dataset, as well as the outcome of aligning with an encoder that is not a detector.
Questions for authors
- What are the respective numbers of real and fake images in Table 2? Were all baselines evaluated using the same data split?
- What is the FID score when BAGEL is fine-tuned solely on the LAION dataset?
- How much does detection performance drop if the component is removed from the detection head of the same checkpoint? Do Tables 1–3 utilize the Stage 1 or Stage 2 checkpoint? (Note: Stage 2 involves continued training of the generation expert responsible for producing .)