- Rating
- 4
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates why AI-generated image detectors based on pretrained vision foundation models often struggle to generalize to unseen domains. The authors identify a phenomenon termed semantic fallback, in which semantic priors inherited from pretraining remain influential after forensic fine-tuning and interfere with learning transferable forensic representations. To address this issue, they propose Geometric Semantic Decoupling (GSD), which uses a frozen CLIP encoder and singular value decomposition (SVD) to identify and suppress dominant pretrained feature components, while adaptively balancing prior suppression and forensic feature preservation. The paper further introduces an efficient mini-batch SVD approximation to reduce computational overhead and develops multiple inference protocols for practical deployment. Experiments on face forgery detection and synthetic image detection demonstrate the effectiveness of the proposed approach in improving cross-domain generalization and computational efficiency.
Critical strengths
The motivation is clear and interesting. Looking at how inherited pretrained representations can interfere with forensic learning is a useful perspective. The identity similarity analysis, geometry occupancy measurements, and suppression experiments provide complementary evidence for the proposed explanation. Soft suppression makes sense here, since useful forensic cues may overlap with the pretrained subspace. And the empirical results are strong across several settings.
Critical weaknesses
1. Insufficiently controlled ablation of adaptive suppression. Figure 3 compares the proposed adaptive coefficient with fixed constants, but this comparison does not isolate the contributions of learnability, layer specificity, and sample dependence. A more controlled ablation comparing a single learned coefficient shared across layers, layer-specific learned constants, and the proposed occupancy-based coefficient would better establish the benefit of sample-adaptive suppression.
2. Inconsistent averaging protocols and limited statistical evidence. Some baselines do not report results on every dataset, making the reported averages not directly comparable. For example, the video-level AUC averages in Table 4 cover five datasets for GSD but only four for χ²-DFD. Comparisons over shared datasets are needed. In addition, reporting means and standard deviations across independent training runs for GSD, a matched CLIP baseline, and key ablations would help establish the reliability of the improvements.
3. Insufficient validation of inference-subspace consistency. Appendix B assumes that the projectors estimated by different inference protocols approximate a shared latent subspace. Although Tables 7 and 8 show similar aggregate detection performance and Figure 7 provides qualitative visualizations, they do not directly quantify subspace agreement. Principal angles or projector distances at the selected layers would provide stronger support. Furthermore, Proposition B.1 assumes a fixed suppression coefficient, whereas the proposed coefficient depends on the estimated projector.
4. Unclear prediction stability under different batch compositions. Batch Inference estimates a shared subspace from the current test batch, making individual predictions potentially dependent on other samples. Although Table 10 examines different batch sizes, it does not establish whether the same image receives consistent predictions when the batch composition changes while its size remains fixed.
5. Incomplete validation of the semantic-suppression mechanism and reference-set construction. The dominant directions extracted by uncentered SVD may reflect feature means or other dominant feature structures rather than exclusively harmful semantic priors. Centered PCA and same-rank random-subspace controls would help distinguish these possibilities. Moreover, the target-risk formulation in Eq. (1) is based on the local surrogate assumed in Assumption A.3, and its correspondence to the actual detection risk remains empirically unverified. Finally, the construction of the CDv2 reference set, including its sampling procedure and potential overlap with evaluation images or source videos, requires clearer documentation.
Questions for authors
-
Could the authors provide controlled ablations comparing a single learned suppression coefficient shared across layers, separate learned constants for each layer, and the proposed occupancy-based adaptive coefficient, using otherwise identical training settings?
-
Could the authors report average improvements over the datasets shared by each pair of compared methods, along with means and standard deviations from at least three independent training runs for GSD and a matched CLIP baseline?
-
How does GSD compare with centered PCA and same-rank random-subspace suppression? Is there direct quantitative evidence that the suppressed directions encode domain-sensitive semantic information rather than feature means or general dominant feature directions?
-
Could the authors quantify subspace agreement across inference protocols using principal angles or projector distances, and clarify whether Proposition B.1 can be extended to the adaptive suppression coefficient? Additionally, how stable are prediction scores and labels for the same image across test batches with different compositions but identical sizes?
-
For reference-based inference, how does performance vary across semantically matched, partially matched, and mismatched reference sets? How were the CDv2 reference images selected, and how was overlap with evaluation samples excluded? Could the authors also clarify the classification readout and, if feasible, compare CLS-token classification with mean pooling over non-CLS patch tokens?
4 comments
Author Response to Reviewer 69L7(2/4)
Table: ACC (%) reported on different subtraction strategies over four runs.
| Setting | Result | CDv2 | DFDCP | InSwap |
|---|---|---|---|---|
| Reported | 81.20 | 80.90 | 84.00 | |
| Repeated | ||||
| Reported | 82.30 | 82.00 | 84.50 | |
| Repeated | ||||
| Reported | 82.80 | 82.30 | 84.60 | |
| Repeated | ||||
| Adaptive | Reported | 83.10 | 82.30 | 85.10 |
| Adaptive | Repeated |
Table: Frame-level AUC (%) using different reference sets and batch SVD.
| Setting | CDv2 | DFDC | DFD | e4s | FaceDancer |
|---|---|---|---|---|---|
| Ref.FF++ | |||||
| Ref.CDv2 | |||||
| Batch SVD |
W3 and Q4: Inference-subspace consistency.
Response: As you suggested, we have measured subspace consistency using the mean principal angle (MPA), projected-feature consistenc (cosine similarity) and symmetric relative distance. All metrics are averaged over layers 21--24. Their MPAs and relative distances are substantially lower than those of random subspaces, while their cosine similarities are much higher. These results suggest that the estimated subspaces and projected features remain relatively consistent across inference protocols, with a clear separation from the random-subspace baseline.
| Dataset | Comparison | MPA (°) ↓ | Cosine similarity ↑ | Relative ↓ |
|---|---|---|---|---|
| CDv2 | Different batches | 38.58 | 0.8705 | 0.2449 |
| CDv2 | Batch vs. FF++ reference | 45.89 | 0.8271 | 0.2839 |
| CDv2 | Batch vs. CDv2 reference | 41.06 | 0.8612 | 0.2539 |
| CDv2 | Single vs. batch | 53.99 | 0.7874 | 0.3106 |
| CDv2 | Batch vs. random subspace | 77.59 | 0.1382 | 0.7118 |
| DFDC | Different batches | 37.23 | 0.8799 | 0.2360 |
| DFDC | Batch vs. FF++ reference | 49.21 | 0.8242 | 0.2866 |
| DFDC | Batch vs. CDv2 reference | 50.46 | 0.8177 | 0.2924 |
| DFDC | Single vs. batch | 51.95 | 0.7969 | 0.3044 |
| DFDC | Batch vs. random subspace | 77.56 | 0.1345 | 0.7171 |
| SimSwap | Different batches | 36.47 | 0.8885 | 0.2270 |
| SimSwap | Batch vs. FF++ reference | 40.16 | 0.8673 | 0.2473 |
| SimSwap | Batch vs. CDv2 reference | 46.67 | 0.8425 | 0.2705 |
| SimSwap | Single vs. batch | 52.62 | 0.7906 | 0.3081 |
| SimSwap | Batch vs. random subspace | 77.59 | 0.1343 | 0.7187 |
W3 and Q4: Can Proposition B.1 be extended to the adaptive suppression coefficient?
Response: We have carefully revised Proposition B.1 to accommodate adaptive suppression coefficients. Let , where . For orthogonal projectors, the same decomposition yields
Thus, the adaptive case introduces an additional term measuring coefficient variation. More specifically, suppose , where is -Lipschitz and for . Since and are orthogonal projectors, we have . Under Assumption B.1, this gives
The corresponding prediction-score bound follows by multiplying the right-hand side by the Lipschitz constant of the downstream function .
W4 and Q4: Prediction stability under different batch compositions.
Response: As per your comments, we have evaluated each image in five different randomly sampled batches. Across the tested datasets, more than 98% of images received the same predicted label in all five runs. For your reference, we have added discussions of Inference sensitivity to batch composition in Section 4.4 of the main text. The results show that the prediction probability curves (Figure6 in main paper) closely overlap across the five runs. Accuracy varies by only 0.10 and 0.16 percentage points on CDv2 and SimSwap, respectively. These results suggest that predictions remain largely stable under random changes in batch composition.
Author Response to Reviewer 69L7(3/4)
| Dataset | Run 1 | Run 2 | Run 3 | Run 4 | Run 5 |
|---|---|---|---|---|---|
| CDv2 | 83.12 | 83.15 | 83.05 | 83.11 | 83.08 |
| SimSwap | 83.94 | 84.10 | 83.94 | 83.95 | 83.98 |
W5 and Q3: The dominant directions extracted by uncentered SVD may reflect feature means or other dominant feature structures. How does GSD compare with centered PCA and same-rank random-subspace suppression?
Response: We would like to clarify that our implementation centered the features before applying SVD to extract the principal directions. We have added this clarification to Section~3.1 of the main paper. We additionally compared a random subspace of the same rank, with the results reported below. The results show that GSD outperforms same-rank random-subspace suppression.
| Strategy | CDv2 | CDv1 | DFDC | DFDCP |
|---|---|---|---|---|
| Naively fine-tuned CLIP | 78.5 | 78.9 | 73.1 | 78.2 |
| Bottom () | 81.8 | 82.0 | 76.1 | 78.4 |
| Random () | 81.3 | 81.7 | 76.5 | 79.8 |
| GSD () | 83.1 | 82.8 | 77.0 | 82.3 |
W5 and Q3: Does the suppressed directions encode domain-sensitive semantic information rather than feature means or general dominant feature directions?
Response: We sincerely appreciate your detailed feedback! We centered pretrained features before applying SVD, so the extracted directions reflect variation around the mean feature rather than the mean itself. We agree that the subspace identified by SVD of pretrained features may include general visual structure, but suppressing it still helps mitigate semantic fallback. We further compared of GSD with same-rank random and bottom-direction suppression. For your reference, the table below shows that GSD produces larger reductions in identity-related semantic structure than either control across all three datasets. This supports the view that the dominant directions capture identity-related semantic priors, with better detection performance.
| Strategy | FF++ | CDv2 | DFDC |
|---|---|---|---|
| Naively fine-tuned CLIP | 0.412 | 0.431 | 0.394 |
| Random () | 0.258 | 0.259 | 0.296 |
| Bottom () | 0.270 | 0.241 | 0.275 |
| GSD () | 0.201 | 0.145 | 0.182 |
W5: Empirical validation of the target-risk surrogate in Assumption A.3.
Response: The target-risk formulation in Eq. (1) provides an analytical interpretation of the balance between suppressing semantic priors and preserving useful forensic cues. GSD seeks to learn this balance, qualitatively consistent with the compensatory adaptation illustrated in Figure 9. However, we acknowledge that these observations are insufficient to quantitatively validate the surrogate or its predicted optimal . Fully disentangling semantic priors from forensic cues is challenging, making the individual risk terms difficult to estimate. We have revised Section 2.4 to frame the surrogate as a qualitative interpretation of the trade-off observed in subsequent experiments.
W5 and Q5: The construction of the CDv2 reference set.
Response: We have provided specific details (Section 4.3 in the paper) on the construction of the fixed CDv2 reference pool. We select 512 images from the designated CDv2 training split, excluding all video directories represented in the full test set to prevent source-video overlap. We sample 256 real and 256 fake images in rounds across shuffled video groups, with frames shuffled within each video to reduce the selection of adjacent frames.
Q5: How does performance vary across semantically matched, partially matched, and mismatched reference sets?
Response: Following your suggestion, we have compared semantically matched, partially matched, and mismatched reference sets. The matched set contains only CDv2 images, the mixed set contains equal numbers of CDv2 and ImageNet images, and the mismatched set contains only ImageNet images. Matched references consistently yield the best performance. We also find that mismatched references remain effective, with ACC decreasing by only 0.6–1.4%. To investigate this robustness, we measured mean principal angles (MPA), projected-feature cosine similarity, and symmetric relative distance. Semantic mismatch increases subspace differences, yet projected features retain cosine similarities of 0.739–0.781. We think that this robustness reflects general visual structur shared across different images. The subspace identified by SVD of pretrained features may include general visual structure, but suppressing it still helps mitigate semantic fallback.
Author Response to Reviewer 69L7(4/4)
| Dataset | Reference alignment | ACC (%) ↑ | MPA (°) ↓ | Projection cosine ↑ | Relative ↓ |
|---|---|---|---|---|---|
| CDv2 | Matched | 82.7 | 42.08 | 0.856 | 0.261 |
| CDv2 | Partially matched | 82.6 | 52.76 | 0.820 | 0.290 |
| CDv2 | Mismatched | 82.1 | 63.53 | 0.771 | 0.331 |
| DFDCP | Matched | 81.9 | 53.33 | 0.791 | 0.312 |
| DFDCP | Partially matched | 81.5 | 56.99 | 0.772 | 0.331 |
| DFDCP | Mismatched | 81.2 | 63.37 | 0.739 | 0.356 |
| SimSwap | Matched | 83.2 | 46.22 | 0.845 | 0.267 |
| SimSwap | Partially matched | 82.9 | 53.08 | 0.816 | 0.285 |
| SimSwap | Mismatched | 82.3 | 63.37 | 0.781 | 0.325 |
| InSwap | Matched | 84.1 | 46.76 | 0.846 | 0.266 |
| InSwap | Partially matched | 83.4 | 53.49 | 0.823 | 0.287 |
| InSwap | Mismatched | 82.7 | 63.60 | 0.777 | 0.327 |
Q5: Classification readout: CLS token versus patch-token mean pooling.
Response: We have compared CLS-token classification with mean pooling over non-CLS patch tokens under the same training setting. For your reference, the table below shows that patch-token mean pooling achieves higher video-level AUC on test datasets.
| Method | Metric | CDv2 | CDv1 | DFDCP | DFD | DFDC |
|---|---|---|---|---|---|---|
| Mean pooling | Video-AUC | 96.4 | 97.0 | 93.2 | 97.7 | 88.3 |
| CLS token | Video-AUC | 94.6 | 95.3 | 91.7 | 97.5 | 88.1 |
Thank you again for your valuable time and effort in reviewing our work. We have revised the paper following your constructive and insightful suggestions. Thanks to your comments, the paper is now significantly improved. Thanks again for your valuable time and efforts!
Author Response to Reviewer 69L7(1/4)
We sincerely appreciate your positive assessment of our work and the valuable comments, which helps us significantly improve the quality and clarity of the manuscript.
Response: Following your suggestion, we compare learnable layer-wise constants and a learnable constant shared across layers. For your reference, the empirical results shown in the table indicate that occupancy-based adaptive modulation achieves better performance than both alternatives.
Response: Thank you for your valuable suggestion. We have recalculated results in Table 4 over datasets shared by GSD and the corresponding baselines. For video-level AUC, GSD outperforms Effort across all five datasets (94.52% vs. 91.82%) and X$^2$-DFD on the four excluding CDv1 (93.90% vs. 91.93%). For frame-level AUC, GSD outperforms the strongest corresponding baselines across all five datasets (90.44% vs. 84.02%, CLIP†), the four excluding CDv1 (90.23% vs. 84.83%, UDD), the four excluding DFDC (91.63% vs. 85.18%, FSgD), and the three shared datasets (90.27% vs. 84.57%, UDD).
Response: Kindly, we have repeated the main experiments for two detection tasks using four different random seeds. For face forgery detection, the mean video-level AUCs averaged over the cross-domain and cross-manipulation benchmarks are 94.6% ± 0.3% and 97.6% ± 0.2%, respectively, close to the originally reported 94.5% and 97.8%. For synthetic image detection, the average accuracy is 97.1% ± 0.2%, compared with 97.3% in the paper. We also report results on three ablation studies across four random seeds, covering vision foundation models, subtraction strategies and inference protocols.
Table: Cross-domain Face Forgery Detection
Table: Cross-manipulation Face Forgery Detection
Table: Synthetic Image Detection
Table: CLIP ablation results (video-level AUC): originally reported results versus four runs.