- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work proposes Forensic Obfuscation of geneRator-specific cues via Gradient-guided Embedding perturbation (FORGE), a token-aware adversarial training framework for open-world AIGV detection. FORGE identifies tokens that are strongly associated with generator- specific shortcuts yet less critical to real/fake discrimination, and selectively perturbs them during training to suppress generator-dependent patterns while preserving useful forensic evidence. Extensive experiments validate the effectiveness of the proposed method.
Critical strengths
1.The basic idea is good. The paper targets a practically important problem—generator-specific shortcut reliance in open-world AIGV detection—and addresses it with a coherent token-aware adversarial training framework. The selective perturbation criterion is well motivated, and the intervention analysis provides direct empirical support for the core premise.
2.The performance compared with SOTA is good. FORGE achieves 92.62 average F1 on 29 held-out entries, outperforming eight representative baselines by 5.31 points, with consistent gains across backbones, informative ablations, robustness under post-processing and white-box attacks, and a family-disjoint subset analysis that strengthens the empirical claims.
Critical weaknesses
-
The "generator-sensitive but detection-safe" dichotomy lacks theoretical guarantee and may not hold universally. The core assumption of FORGE is that there exists a class of tokens strongly encoding generator identity yet contributing little to real/fake discrimination, so perturbing them removes shortcuts without harming detection. This is supported only empirically, with no theoretical justification. If a new generator's artifacts are entangled with genuine forensic cues—e.g., its compression fingerprints are also key to real/fake discrimination—the assumption breaks down, and selective perturbation could damage detection performance. A discussion of boundary conditions, or a stress test where generator-specific and detection-relevant cues are deliberately correlated, would strengthen the claim.
-
The token importance measure relies on a classic gradient-norm heuristic. While computationally efficient, it is not compared against more recent, faithful ViT attribution methods. As a result, it remains unclear whether the observed gains stem from the proposed selective perturbation strategy or simply from the choice of a coarse importance proxy, and whether a more faithful measure would further improve OOD generalization.
-
The "open-world" claim is not fully supported by the experimental setup. Although the paper evaluates on 29 held-out entries, all videos are uniformly sampled to 8 frames and resized to 224×224, and are drawn from only three public benchmarks. This setup primarily tests cross-generator generalization rather than true open-world conditions, which would involve longer videos, higher resolutions, multimodal inputs, and repeated editing or compression. The paper acknowledges in its limitations that longer videos and broader real-world domains remain uncovered, but the title and abstract still frame the contribution as "open-world" generalization. This may be a cross-generator OOD study rather than an open-world one. A more precise framing or additional evidence beyond the current benchmark would make the claim more defensible.
Questions for authors
- When generator-specific artifacts are strongly correlated with genuine forensic cues, can the proposed method still select the right tokens? Is there a stress test for this entangled case?
- The perturbation is guided solely by the generator classifier's KL gradient. If the generator classifier is biased, could the perturbation direction be misled? Is there a control using the real/fake gradient to guide perturbation?
- How do you delineate "open-world" from "cross-generator OOD"? Do the conclusions hold on longer videos or realistically compressed content?