acceptodds
Under review as a conference paper at ICLR 2027

Less Prior, More Forensics: Reducing Pretrained Semantic Priors for Generalizable AI-Generated Image Detection

10 views

Abstract

The growing realism of generative models has blurred the boundary between real and synthetic content, posing significant challenges to reliable AI-generated image detection. In this paper, we identify a failure pattern, termed semantic fallback, where semantic priors inherited from pretraining remain dominant in the representation space after forensic fine-tuning, thereby limiting the generalization of VFM-based detectors. Building on this insight, we propose a Geometric Semantic Decoupling (GSD) framework, which explicitly suppresses dominant pretrained geometry, thereby promoting invariant forensic representations. Specifically, GSD leverages a frozen CLIP encoder to estimate the dominant pretrained subspace via Singular Value Decomposition (SVD). It then suppresses the prior-aligned components through a geometry-constrained formulation with the suppression strength adaptively modulated across samples and layers. We further introduce a mini-batch SVD approximation strategy that amortizes subspace estimation, achieving over a 15× reduction in computational overhead while preserving effectiveness. Empirical results show that our method achieves state-of-the-art performance on various datasets, with average video-level AUCs of 94.5% (+2.6%) and 97.8% (+3.0%) on two face-forgery protocols, and average accuracies of 97.3% (+2.1%) on the synthetic-image benchmark. Our code is available at https://anonymous.4open.science/r/Null-Space_CLIP.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI-generated Images, Vision Foundation Models, Geometric Semantic Decoupling, Generalizable Detection.

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_69L7OpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates why AI-generated image detectors based on pretrained vision foundation models often struggle to generalize to unseen domains. The authors identify a phenomenon termed semantic fallback, in which semantic priors inherited from pretraining remain influential after forensic fine-tuning and interfere with learning transferable forensic representations. To address this issue, they propose Geometric Semantic Decoupling (GSD), which uses a frozen CLIP encoder and singular value decomposition (SVD) to identify and suppress dominant pretrained feature components, while adaptively balancing prior suppression and forensic feature preservation. The paper further introduces an efficient mini-batch SVD approximation to reduce computational overhead and develops multiple inference protocols for practical deployment. Experiments on face forgery detection and synthetic image detection demonstrate the effectiveness of the proposed approach in improving cross-domain generalization and computational efficiency.

Critical strengths

The motivation is clear and interesting. Looking at how inherited pretrained representations can interfere with forensic learning is a useful perspective. The identity similarity analysis, geometry occupancy measurements, and suppression experiments provide complementary evidence for the proposed explanation. Soft suppression makes sense here, since useful forensic cues may overlap with the pretrained subspace. And the empirical results are strong across several settings.

Critical weaknesses

1. Insufficiently controlled ablation of adaptive suppression. Figure 3 compares the proposed adaptive coefficient with fixed constants, but this comparison does not isolate the contributions of learnability, layer specificity, and sample dependence. A more controlled ablation comparing a single learned coefficient shared across layers, layer-specific learned constants, and the proposed occupancy-based coefficient would better establish the benefit of sample-adaptive suppression.

2. Inconsistent averaging protocols and limited statistical evidence. Some baselines do not report results on every dataset, making the reported averages not directly comparable. For example, the video-level AUC averages in Table 4 cover five datasets for GSD but only four for χ²-DFD. Comparisons over shared datasets are needed. In addition, reporting means and standard deviations across independent training runs for GSD, a matched CLIP baseline, and key ablations would help establish the reliability of the improvements.

3. Insufficient validation of inference-subspace consistency. Appendix B assumes that the projectors estimated by different inference protocols approximate a shared latent subspace. Although Tables 7 and 8 show similar aggregate detection performance and Figure 7 provides qualitative visualizations, they do not directly quantify subspace agreement. Principal angles or projector distances at the selected layers would provide stronger support. Furthermore, Proposition B.1 assumes a fixed suppression coefficient, whereas the proposed coefficient depends on the estimated projector.

4. Unclear prediction stability under different batch compositions. Batch Inference estimates a shared subspace from the current test batch, making individual predictions potentially dependent on other samples. Although Table 10 examines different batch sizes, it does not establish whether the same image receives consistent predictions when the batch composition changes while its size remains fixed.

5. Incomplete validation of the semantic-suppression mechanism and reference-set construction. The dominant directions extracted by uncentered SVD may reflect feature means or other dominant feature structures rather than exclusively harmful semantic priors. Centered PCA and same-rank random-subspace controls would help distinguish these possibilities. Moreover, the target-risk formulation in Eq. (1) is based on the local surrogate assumed in Assumption A.3, and its correspondence to the actual detection risk remains empirically unverified. Finally, the construction of the CDv2 reference set, including its sampling procedure and potential overlap with evaluation images or source videos, requires clearer documentation.

Questions for authors
  1. Could the authors provide controlled ablations comparing a single learned suppression coefficient shared across layers, separate learned constants for each layer, and the proposed occupancy-based adaptive coefficient, using otherwise identical training settings?

  2. Could the authors report average improvements over the datasets shared by each pair of compared methods, along with means and standard deviations from at least three independent training runs for GSD and a matched CLIP baseline?

  3. How does GSD compare with centered PCA and same-rank random-subspace suppression? Is there direct quantitative evidence that the suppressed directions encode domain-sensitive semantic information rather than feature means or general dominant feature directions?

  4. Could the authors quantify subspace agreement across inference protocols using principal angles or projector distances, and clarify whether Proposition B.1 can be extended to the adaptive suppression coefficient? Additionally, how stable are prediction scores and labels for the same image across test batches with different compositions but identical sizes?

  5. For reference-based inference, how does performance vary across semantically matched, partially matched, and mismatched reference sets? How were the CDv2 reference images selected, and how was overlap with evaluation samples excluded? Could the authors also clarify the classification readout and, if feasible, compare CLS-token classification with mean pooling over non-CLS patch tokens?

4 comments
AuthorsOpenReview ↗

Author Response to Reviewer 69L7(1/4)

We sincerely appreciate your positive assessment of our work and the valuable comments, which helps us significantly improve the quality and clarity of the manuscript.

W1 and Q1: Comparison with learnable suppression constants.

Response: Following your suggestion, we compare learnable layer-wise constants and a learnable constant shared across layers. For your reference, the empirical results shown in the table indicate that occupancy-based adaptive modulation achieves better performance than both alternatives.

SettingCDv2DFDCe4sInSwap
Layer-specific82.876.792.684.8
Shared across layers82.676.592.284.5
GSD83.177.093.585.1

W2 and Q2: Inconsistent averaging protocols.

Response: Thank you for your valuable suggestion. We have recalculated results in Table 4 over datasets shared by GSD and the corresponding baselines. For video-level AUC, GSD outperforms Effort across all five datasets (94.52% vs. 91.82%) and X$^2$-DFD on the four excluding CDv1 (93.90% vs. 91.93%). For frame-level AUC, GSD outperforms the strongest corresponding baselines across all five datasets (90.44% vs. 84.02%, CLIP†), the four excluding CDv1 (90.23% vs. 84.83%, UDD), the four excluding DFDC (91.63% vs. 85.18%, FSgD), and the three shared datasets (90.27% vs. 84.57%, UDD).

W2 and Q2: Reporting means and standard deviations.

Response: Kindly, we have repeated the main experiments for two detection tasks using four different random seeds. For face forgery detection, the mean video-level AUCs averaged over the cross-domain and cross-manipulation benchmarks are 94.6% ± 0.3% and 97.6% ± 0.2%, respectively, close to the originally reported 94.5% and 97.8%. For synthetic image detection, the average accuracy is 97.1% ± 0.2%, compared with 97.3% in the paper. We also report results on three ablation studies across four random seeds, covering vision foundation models, subtraction strategies and inference protocols.

Table: Cross-domain Face Forgery Detection

MethodMetricCDv2CDv1DFDCDFDCPDFDAvg.
GSD-CLIPVideo-level AUC96.3±0.196.3 \pm 0.196.7±0.396.7 \pm 0.388.6±0.588.6 \pm 0.593.5±0.593.5 \pm 0.598.0±0.398.0 \pm 0.394.6±0.394.6 \pm 0.3

Table: Cross-manipulation Face Forgery Detection

MethodMetricUniFacee4sFaceDancerFSGANInSwapSimSwapAvg.
GSD-CLIPVideo-level AUC98.1±0.498.1 \pm 0.499.2±0.399.2 \pm 0.397.3±0.297.3 \pm 0.298.6±0.398.6 \pm 0.396.3±0.296.3 \pm 0.295.8±0.295.8 \pm 0.297.6±0.297.6 \pm 0.2

Table: Synthetic Image Detection

MethodProGANCycleGANBigGANStyleGANGauGANStarGANDeepfakesSITDSANCRNIMLE
GSD-CLIP100.0±0.0100.0 \pm 0.099.4±0.299.4 \pm 0.299.6±0.199.6 \pm 0.196.4±2.196.4 \pm 2.199.8±0.099.8 \pm 0.0100.0±0.0100.0 \pm 0.090.5±1.990.5 \pm 1.990.9±2.590.9 \pm 2.591.8±0.791.8 \pm 0.797.9±0.497.9 \pm 0.498.7±0.598.7 \pm 0.5
MethodGuidedLDM 200 stepsLDM 200 w/ cfgLDM 100 stepsGlide 100/27Glide 50/27Glide 100/10DALL-EAvg.
GSD-CLIP92.6±3.092.6 \pm 3.099.4±0.299.4 \pm 0.298.3±0.198.3 \pm 0.199.3±0.399.3 \pm 0.397.0±1.597.0 \pm 1.597.5±1.497.5 \pm 1.497.1±0.797.1 \pm 0.798.8±0.698.8 \pm 0.697.1±0.297.1 \pm 0.2

Table: CLIP ablation results (video-level AUC): originally reported results versus four runs.

MethodSettingCDv2DFDCPDFDCInSwapFaceDancerAvg.
CLIPReported90.788.483.190.391.588.8
CLIPRepeated91.0±0.291.0 \pm 0.288.4±0.488.4 \pm 0.483.3±0.483.3 \pm 0.490.8±0.590.8 \pm 0.591.8±0.391.8 \pm 0.389.1±0.389.1 \pm 0.3
GSD-CLIPReported96.493.288.396.597.594.4
GSD-CLIPRepeated96.3±0.196.3 \pm 0.193.5±0.593.5 \pm 0.588.6±0.588.6 \pm 0.596.3±0.296.3 \pm 0.297.3±0.297.3 \pm 0.294.4±0.294.4 \pm 0.2
AuthorsOpenReview ↗

Author Response to Reviewer 69L7(2/4)

Table: ACC (%) reported on different subtraction strategies over four runs.

SettingResultCDv2DFDCPInSwap
λ=0.05\lambda = 0.05Reported81.2080.9084.00
λ=0.05\lambda = 0.05Repeated81.23±0.2181.23 \pm 0.2181.10±0.2881.10 \pm 0.2883.98±0.1383.98 \pm 0.13
λ=0.4\lambda = 0.4Reported82.3082.0084.50
λ=0.4\lambda = 0.4Repeated82.30±0.1682.30 \pm 0.1682.20±0.2482.20 \pm 0.2484.45±0.1784.45 \pm 0.17
λ=0.8\lambda = 0.8Reported82.8082.3084.60
λ=0.8\lambda = 0.8Repeated82.78±0.1282.78 \pm 0.1282.35±0.2582.35 \pm 0.2584.53±0.2584.53 \pm 0.25
AdaptiveReported83.1082.3085.10
AdaptiveRepeated82.97±0.1482.97 \pm 0.1482.40±0.1882.40 \pm 0.1884.93±0.1784.93 \pm 0.17

Table: Frame-level AUC (%) using different reference sets and batch SVD.

SettingCDv2DFDCDFDe4sFaceDancer
Ref.FF++90.31±0.0290.31 \pm 0.0285.52±0.0085.52 \pm 0.0094.30±0.0194.30 \pm 0.0197.91±0.0097.91 \pm 0.0093.75±0.0293.75 \pm 0.02
Ref.CDv290.42±0.0090.42 \pm 0.0085.50±0.0285.50 \pm 0.0294.30±0.0194.30 \pm 0.0197.87±0.0197.87 \pm 0.0193.74±0.0293.74 \pm 0.02
Batch SVD90.65±0.0290.65 \pm 0.0285.76±0.0085.76 \pm 0.0094.53±0.0194.53 \pm 0.0197.98±0.0197.98 \pm 0.0193.82±0.0093.82 \pm 0.00

W3 and Q4: Inference-subspace consistency.

Response: As you suggested, we have measured subspace consistency using the mean principal angle (MPA), projected-feature consistenc (cosine similarity) and symmetric relative L2L_2 distance. All metrics are averaged over layers 21--24. Their MPAs and relative L2L_2 distances are substantially lower than those of random subspaces, while their cosine similarities are much higher. These results suggest that the estimated subspaces and projected features remain relatively consistent across inference protocols, with a clear separation from the random-subspace baseline.

DatasetComparisonMPA (°) ↓Cosine similarity ↑Relative L2L_2 ↓
CDv2Different batches38.580.87050.2449
CDv2Batch vs. FF++ reference45.890.82710.2839
CDv2Batch vs. CDv2 reference41.060.86120.2539
CDv2Single vs. batch53.990.78740.3106
CDv2Batch vs. random subspace77.590.13820.7118
DFDCDifferent batches37.230.87990.2360
DFDCBatch vs. FF++ reference49.210.82420.2866
DFDCBatch vs. CDv2 reference50.460.81770.2924
DFDCSingle vs. batch51.950.79690.3044
DFDCBatch vs. random subspace77.560.13450.7171
SimSwapDifferent batches36.470.88850.2270
SimSwapBatch vs. FF++ reference40.160.86730.2473
SimSwapBatch vs. CDv2 reference46.670.84250.2705
SimSwapSingle vs. batch52.620.79060.3081
SimSwapBatch vs. random subspace77.590.13430.7187

W3 and Q4: Can Proposition B.1 be extended to the adaptive suppression coefficient?

Response: We have carefully revised Proposition B.1 to accommodate adaptive suppression coefficients. Let T(α)(f)=(I−λα(f)Pα)fT^{(\alpha)}(f)=(I-\lambda_\alpha(f)P_\alpha)f, where 0≤λα(f)≤λmax⁡0\leq\lambda_\alpha(f)\leq\lambda_{\max}. For orthogonal projectors, the same decomposition yields

∥T(α)(f)−T(β)(f)∥≤[λmax⁡∥Pα−Pβ∥+∣λα(f)−λβ(f)∣]∥f∥.\|T^{(\alpha)}(f)-T^{(\beta)}(f)\|\leq\left[\lambda_{\max}\|P_\alpha-P_\beta\|+|\lambda_\alpha(f)-\lambda_\beta(f)|\right]\|f\|.

Thus, the adaptive case introduces an additional term measuring coefficient variation. More specifically, suppose λα(f)=g(rα(f))\lambda_\alpha(f)=g(r_\alpha(f)), where gg is LgL_g-Lipschitz and rα(f)=∥Pαf∥2/∥f∥2r_\alpha(f)=\|P_\alpha f\|^2/\|f\|^2 for f≠0f\neq 0. Since PαP_\alpha and PβP_\beta are orthogonal projectors, we have ∣rα(f)−rβ(f)∣=∣f⊤(Pα−Pβ)f∣/∥f∥2≤∥Pα−Pβ∥|r_\alpha(f)-r_\beta(f)|=|f^\top(P_\alpha-P_\beta)f|/\|f\|^2\leq\|P_\alpha-P_\beta\|. Under Assumption B.1, this gives

∥T(α)(f)−T(β)(f)∥≤(λmax⁡+Lg)(δα+δβ)∥f∥.\|T^{(\alpha)}(f)-T^{(\beta)}(f)\|\leq(\lambda_{\max}+L_g)(\delta_\alpha+\delta_\beta)\|f\|.

The corresponding prediction-score bound follows by multiplying the right-hand side by the Lipschitz constant LL of the downstream function hh.

W4 and Q4: Prediction stability under different batch compositions.

Response: As per your comments, we have evaluated each image in five different randomly sampled batches. Across the tested datasets, more than 98% of images received the same predicted label in all five runs. For your reference, we have added discussions of Inference sensitivity to batch composition in Section 4.4 of the main text. The results show that the prediction probability curves (Figure6 in main paper) closely overlap across the five runs. Accuracy varies by only 0.10 and 0.16 percentage points on CDv2 and SimSwap, respectively. These results suggest that predictions remain largely stable under random changes in batch composition.

AuthorsOpenReview ↗

Author Response to Reviewer 69L7(3/4)

DatasetRun 1Run 2Run 3Run 4Run 5
CDv283.1283.1583.0583.1183.08
SimSwap83.9484.1083.9483.9583.98

W5 and Q3: The dominant directions extracted by uncentered SVD may reflect feature means or other dominant feature structures. How does GSD compare with centered PCA and same-rank random-subspace suppression?

Response: We would like to clarify that our implementation centered the features before applying SVD to extract the principal directions. We have added this clarification to Section~3.1 of the main paper. We additionally compared a random subspace of the same rank, with the results reported below. The results show that GSD outperforms same-rank random-subspace suppression.

StrategyCDv2CDv1DFDCDFDCP
Naively fine-tuned CLIP78.578.973.178.2
Bottom (K=64K = 64)81.882.076.178.4
Random (K=64K = 64)81.381.776.579.8
GSD (K=64K = 64)83.182.877.082.3

W5 and Q3: Does the suppressed directions encode domain-sensitive semantic information rather than feature means or general dominant feature directions?

Response: We sincerely appreciate your detailed feedback! We centered pretrained features before applying SVD, so the extracted directions reflect variation around the mean feature rather than the mean itself. We agree that the subspace identified by SVD of pretrained features may include general visual structure, but suppressing it still helps mitigate semantic fallback. We further compared ΔS\Delta S of GSD with same-rank random and bottom-direction suppression. For your reference, the table below shows that GSD produces larger reductions in identity-related semantic structure than either control across all three datasets. This supports the view that the dominant directions capture identity-related semantic priors, with better detection performance.

StrategyFF++CDv2DFDC
Naively fine-tuned CLIP0.4120.4310.394
Random (K=64K = 64)0.2580.2590.296
Bottom (K=64K = 64)0.2700.2410.275
GSD (K=64K = 64)0.2010.1450.182

W5: Empirical validation of the target-risk surrogate in Assumption A.3.

Response: The target-risk formulation in Eq. (1) provides an analytical interpretation of the balance between suppressing semantic priors and preserving useful forensic cues. GSD seeks to learn this balance, qualitatively consistent with the compensatory adaptation illustrated in Figure 9. However, we acknowledge that these observations are insufficient to quantitatively validate the surrogate or its predicted optimal λ\lambda. Fully disentangling semantic priors from forensic cues is challenging, making the individual risk terms difficult to estimate. We have revised Section 2.4 to frame the surrogate as a qualitative interpretation of the trade-off observed in subsequent experiments.

W5 and Q5: The construction of the CDv2 reference set.

Response: We have provided specific details (Section 4.3 in the paper) on the construction of the fixed CDv2 reference pool. We select 512 images from the designated CDv2 training split, excluding all video directories represented in the full test set to prevent source-video overlap. We sample 256 real and 256 fake images in rounds across shuffled video groups, with frames shuffled within each video to reduce the selection of adjacent frames.

Q5: How does performance vary across semantically matched, partially matched, and mismatched reference sets?

Response: Following your suggestion, we have compared semantically matched, partially matched, and mismatched reference sets. The matched set contains only CDv2 images, the mixed set contains equal numbers of CDv2 and ImageNet images, and the mismatched set contains only ImageNet images. Matched references consistently yield the best performance. We also find that mismatched references remain effective, with ACC decreasing by only 0.6–1.4%. To investigate this robustness, we measured mean principal angles (MPA), projected-feature cosine similarity, and symmetric relative L2L_2 distance. Semantic mismatch increases subspace differences, yet projected features retain cosine similarities of 0.739–0.781. We think that this robustness reflects general visual structur shared across different images. The subspace identified by SVD of pretrained features may include general visual structure, but suppressing it still helps mitigate semantic fallback.

AuthorsOpenReview ↗

Author Response to Reviewer 69L7(4/4)

DatasetReference alignmentACC (%) ↑MPA (°) ↓Projection cosine ↑Relative L2L_2 ↓
CDv2Matched82.742.080.8560.261
CDv2Partially matched82.652.760.8200.290
CDv2Mismatched82.163.530.7710.331
DFDCPMatched81.953.330.7910.312
DFDCPPartially matched81.556.990.7720.331
DFDCPMismatched81.263.370.7390.356
SimSwapMatched83.246.220.8450.267
SimSwapPartially matched82.953.080.8160.285
SimSwapMismatched82.363.370.7810.325
InSwapMatched84.146.760.8460.266
InSwapPartially matched83.453.490.8230.287
InSwapMismatched82.763.600.7770.327

Q5: Classification readout: CLS token versus patch-token mean pooling.

Response: We have compared CLS-token classification with mean pooling over non-CLS patch tokens under the same training setting. For your reference, the table below shows that patch-token mean pooling achieves higher video-level AUC on test datasets.

MethodMetricCDv2CDv1DFDCPDFDDFDC
Mean poolingVideo-AUC96.497.093.297.788.3
CLS tokenVideo-AUC94.695.391.797.588.1

Thank you again for your valuable time and effort in reviewing our work. We have revised the paper following your constructive and insightful suggestions. Thanks to your comments, the paper is now significantly improved. Thanks again for your valuable time and efforts!

Reviewer_APCCOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies why vision foundation models may struggle to generalize in deepfake detection. The authors argue that pretrained semantic information can remain dominant after fine-tuning and interfere with forensic feature learning. To address this issue, they propose Geometric Semantic Decoupling (GSD), which uses a frozen CLIP branch to identify dominant feature subspaces and suppresses the corresponding components in the trainable branch. They also introduce Batch-SVD to reduce computational cost. Experiments on cross-dataset deepfake detection and AI-generated image detection show promising results. However, I have concerns about the novelty of the method and whether the experiments sufficiently support the proposed explanation.

Critical strengths

Improving the generalization of vision foundation models in deepfake detection is a meaningful research direction, especially when models are evaluated on unseen datasets and manipulation methods.

GSD does not require additional forgery-region annotations and could potentially be integrated into existing VFM-based detectors. Batch-SVD also makes the method more computationally efficient.

Critical weaknesses
  1. The paper argues that pretrained semantic priors cause cross-domain detection failures, but the current evidence mainly shows a correlation between feature-subspace alignment and prediction errors. It remains unclear whether the problem is actually caused by semantic priors or by other factors, such as image quality, distribution shifts, or generic feature redundancy.

  2. Batch-SVD estimates a shared subspace from the images in the same batch. As a result, the prediction for a given image may depend on which other images are processed alongside it. The current experiments do not adequately establish the stability of predictions under different batch compositions.

  3. The paper does not consistently report results across multiple random seeds or provide uncertainty estimates for key comparisons. More controlled comparisons under the same training and evaluation settings are needed to establish the reliability of the reported improvements.

  4. What evidence shows that the removed feature directions primarily encode harmful semantic information rather than useful forensic cues or general visual structure? Can the authors provide experiments that separately measure semantic information and forensic information before and after suppression?

Questions for authors
  1. How sensitive is Batch-SVD to batch composition? If the same image is evaluated alongside different sets of images, do its feature representation and prediction score change significantly? Please provide quantitative results across different batch compositions and orderings.

  2. Can the authors report results across multiple random seeds, including means and standard deviations, for the main comparisons and key ablation studies?

5 comments
AuthorsOpenReview ↗

Author Response to Reviewer APCC(1/3)

We thank the reviewer for your constructive comments and insightful suggestions. Below we share some factual points with respect to your concerns, for reassessing the contributions of our work and also clarifying with missed details.

W1: Whether the problem is actually caused by semantic priors or by other factors?

Thanks for your valuable comments! We respectfully clarify that cross-domain generalization under image-quality variation and distribution shifts is precisely the problem we study. Kindly, GSD is designed to reduce the continued dominance of pretrained semantic priors after naive fine-tuning. Empirical results in Tables 4-6 of the main paper show that mitigating this semantic fallback substantially improves cross-domain performance. Following your comments, we have added the following experiments to substantiate the link between mitigating semantic fallback and improving cross-domain generalization.

1) We have compared ΔS\Delta S and ACC under random-direction suppression (K=64K=64), bottom-direction suppression (K=64K=64, corresponding to the smallest singular values), and top-direction suppression (K=1K=1). Random, bottom, and top-11 suppression also slightly reduce semantic priors, consistent with their limited performance gains. These empirical results suggest that stronger suppression of identity-related semantic priors tend to yield greater gains from GSD.

Table: Identity similarity gap ΔS\Delta S

StrategyFF++CDv2DFDC
Naively fine-tuned CLIP0.4120.4310.394
Random (K=64K = 64)0.2580.2590.296
Bottom (K=64K = 64)0.2700.2410.275
Top (K=1K = 1)0.2900.2630.260
GSD (K=64K = 64)0.2010.1450.182

Table: Detection Accuracy

Strategy (ACC)CDv2CDv1DFDCDFDCP
Naively fine-tuned CLIP78.578.973.178.2
Random (K=64K = 64)81.381.776.579.8
Bottom (K=64K = 64)81.882.076.178.4
Top (K=1K = 1)81.381.777.081.3
GSD (K=64K = 64)83.182.877.082.3

2) We have grouped samples misclassified by fine-tuned CLIP into quintiles according to their occupancy r. Within these fixed quintiles, we evaluated GSD’s correction rate and each quintile’s share that remained uncorrected, averaging both metrics equally across real and fake classes. Correction rates generally increased with r, whereas the highest-occupancy quintile accounted for the smallest share of remaining errors. These findings show that the proposed method improves detection by suppressing semantic fallback, and preferentially corrects errors in high-occupancy samples.

DatasetOccupancy rr rank (%)Correction rate (%) ↑Remaining-error share (%)
CDv20–2029.6029.52
CDv220–4036.9820.53
CDv240–6036.4718.08
CDv260–8042.0616.15
CDv280–10046.4215.71
FF++0–2010.4423.14
FF++20–4017.7620.84
FF++40–6023.4620.40
FF++60–8039.0019.71
FF++80–10046.1615.91

W2 and Q1: The stability of predictions under different batch compositions.

Response: We appreciate your detailed feedback! We have evaluated each image in five different randomly sampled batches. Across the tested datasets, more than 98% of images received the same predicted label in all five runs. For your reference, we have added discussions of Inference sensitivity to batch composition in Section 4.4 of the main text. The results show that the prediction probability curves (Figure 6 in main paper) closely overlap across the five runs. Accuracy varies by only 0.10 and 0.16 percentage points on CDv2 and SimSwap, respectively. These results suggest that predictions remain largely stable under random changes in batch composition.

W3 and Q2: Main results across multiple random seeds.

Response: Following your suggestion, we have repeated the main experiments for two detection tasks using four different random seeds. For face forgery detection, the mean video-level AUCs averaged over the cross-domain and cross-manipulation benchmarks are 94.6% ± 0.3% and 97.6% ± 0.2%, respectively, close to the reported 94.5% and 97.8% in the paper. For synthetic image detection, the average accuracy is 97.1% ± 0.2%, compared with 97.3% in the paper.

AuthorsOpenReview ↗

Author Response to Reviewer APCC(2/3)

W3: More controlled comparisons under the same training and evaluation settings.

Response: Thanks to your insightful comments! Kindly, GSD follows the DeepfakeBench and UniversalFakeDetect protocols for face forgery and synthetic image detection (including data augmentation), respectively. These protocols are also used by the corresponding strong baselines: Effort, VbSaT, FSgD, and X²-DFD for face forgery detection, and C2P-CLIP and MPFT for synthetic image detection. We additionally reproduced DeepShield, Effort, and C2P-CLIP using exactly the same training settings as GSD. For your information, the empirical results below show that GSD achieves higher average performance than the reproduced baselines.

Table: Cross-domain Face Forgery Detection

MethodMetricCDv2CDv1DFDCDFDCPDFDAvg.
DeepShield (Reported)Video-level AUC92.2–82.893.296.191.1
DeepShield (Reproduced)Video-level AUC91.891.281.893.995.690.9
Effort (Reported)Video-level AUC95.691.884.390.996.591.8
Effort (Reproduced)Video-level AUC93.492.785.191.697.092.0
GSD-CLIPVideo-level AUC96.3±0.1\mathbf{96.3} \pm 0.196.7±0.3\mathbf{96.7} \pm 0.388.6±0.5\mathbf{88.6} \pm 0.593.5±0.5\mathbf{93.5} \pm 0.598.0±0.3\mathbf{98.0} \pm 0.394.6±0.3\mathbf{94.6} \pm 0.3

Table: Cross-manipulation Face Forgery Detection

MethodMetricUniFacee4sFaceDancerFSGANInSwapSimSwapAvg.
Effort (Reported)Video-level AUC96.298.392.695.793.692.694.8
Effort (Reproduced)Video-level AUC96.298.493.896.394.192.995.3
GSD-CLIPVideo-level AUC98.1±0.4\mathbf{98.1} \pm 0.499.2±0.3\mathbf{99.2} \pm 0.397.3±0.2\mathbf{97.3} \pm 0.298.6±0.3\mathbf{98.6} \pm 0.396.3±0.2\mathbf{96.3} \pm 0.295.8±0.2\mathbf{95.8} \pm 0.297.6±0.2\mathbf{97.6} \pm 0.2

Table: Synthetic Image Detection

MethodProGANCycleGANBigGANStyleGANGauGANStarGANDeepfakesSITDSANCRNIMLE
C2P-CLIP (Reported)100.097.399.196.499.299.693.895.664.493.393.3
C2P-CLIP (Reproduced)100.099.197.692.996.8100.086.885.879.772.572.5
GSD-CLIP100.0±0.0\mathbf{100.0} \pm 0.099.4±0.2\mathbf{99.4} \pm 0.299.6±0.1\mathbf{99.6} \pm 0.196.4±2.1\mathbf{96.4} \pm 2.199.8±0.0\mathbf{99.8} \pm 0.0100.0±0.0\mathbf{100.0} \pm 0.090.5±1.990.5 \pm 1.990.9±2.590.9 \pm 2.591.8±0.7\mathbf{91.8} \pm 0.797.9±0.4\mathbf{97.9} \pm 0.498.7±0.5\mathbf{98.7} \pm 0.5
MethodGuidedLDM 200 stepsLDM 200 w/ cfgLDM 100 stepsGlide 100/27Glide 50/27Glide 100/10DALL-EAvg.
C2P-CLIP (Reported)69.199.397.399.395.395.396.198.693.8
C2P-CLIP (Reproduced)81.899.596.399.294.595.895.998.891.9
GSD-CLIP92.6±3.0\mathbf{92.6} \pm 3.099.4±0.299.4 \pm 0.298.3±0.1\mathbf{98.3} \pm 0.199.3±0.3\mathbf{99.3} \pm 0.397.0±1.5\mathbf{97.0} \pm 1.597.5±1.4\mathbf{97.5} \pm 1.497.1±0.7\mathbf{97.1} \pm 0.798.8±0.6\mathbf{98.8} \pm 0.697.1±0.2\mathbf{97.1} \pm 0.2

W3 and Q2: Ablation studies across multiple random seeds.

Response: As per your comments, we have repeated three sets of key ablation experiments with different random seeds. Below, we share the empirical results of four repeated runs.

Table: CLIP ablation results (video-level AUC): originally reported results versus four runs.

MethodSettingCDv2DFDCPDFDCInSwapFaceDancerAvg.
CLIPReported90.788.483.190.391.588.8
CLIPRepeated91.0±0.291.0 \pm 0.288.4±0.488.4 \pm 0.483.3±0.483.3 \pm 0.490.8±0.590.8 \pm 0.591.8±0.391.8 \pm 0.389.1±0.389.1 \pm 0.3
GSD-CLIPReported96.493.288.396.597.594.4
GSD-CLIPRepeated96.3±0.196.3 \pm 0.193.5±0.5\mathbf{93.5} \pm 0.588.6±0.5\mathbf{88.6} \pm 0.596.3±0.296.3 \pm 0.297.3±0.297.3 \pm 0.294.4±0.2\mathbf{94.4} \pm 0.2
AuthorsOpenReview ↗

Author Response to Reviewer APCC(3/3)

Table: ACC (%) reported on different subtraction strategies over four runs.

SettingResultCDv2DFDCPInSwap
λ=0.05\lambda = 0.05Reported81.2080.9084.00
λ=0.05\lambda = 0.05Repeated81.23±0.2181.23 \pm 0.2181.10±0.2881.10 \pm 0.2883.98±0.1383.98 \pm 0.13
λ=0.4\lambda = 0.4Reported82.3082.0084.50
λ=0.4\lambda = 0.4Repeated82.30±0.1682.30 \pm 0.1682.20±0.2482.20 \pm 0.2484.45±0.1784.45 \pm 0.17
λ=0.8\lambda = 0.8Reported82.8082.3084.60
λ=0.8\lambda = 0.8Repeated82.78±0.1282.78 \pm 0.1282.35±0.2582.35 \pm 0.2584.53±0.2584.53 \pm 0.25
AdaptiveReported83.1082.3085.10
AdaptiveRepeated82.97±0.1482.97 \pm 0.1482.40±0.1882.40 \pm 0.1884.93±0.1784.93 \pm 0.17

Table: Frame-level AUC (%) using different reference sets and batch SVD.

SettingCDv2DFDCDFDe4sFaceDancer
Ref.FF++90.31±0.0290.31 \pm 0.0285.52±0.0085.52 \pm 0.0094.30±0.0194.30 \pm 0.0197.91±0.0097.91 \pm 0.0093.75±0.0293.75 \pm 0.02
Ref.CDv290.42±0.0090.42 \pm 0.0085.50±0.0285.50 \pm 0.0294.30±0.0194.30 \pm 0.0197.87±0.0197.87 \pm 0.0193.74±0.0293.74 \pm 0.02
Batch SVD90.65±0.0290.65 \pm 0.0285.76±0.0085.76 \pm 0.0094.53±0.0194.53 \pm 0.0197.98±0.0197.98 \pm 0.0193.82±0.0093.82 \pm 0.00

W4: Semantic and forensic information before and after suppression.

Response: We would like to clarify that GSD focuses on reducing the dominance of pretrained semantic priors that persists after naive fine-tuning. This does not require explicitly identifying which semantic directions are harmful. Instead, the key is to balance semantic suppression with the preservation of useful forensic cues. We agree that the subspace identified by SVD of pretrained features may include general visual structure, but suppressing it still helps mitigate semantic fallback. Supporting evidence is provided in the response to W1: GSD substantially reduces ΔS\Delta S across all three datasets while improving cross-domain detection accuracy.

We have additionally trained deepfake linear probes on both decision features and their suppressed projections of GSD, using pretrained CLIP as a reference. The projected features exhibit weaker forensic discriminability than the decision features, with performance closer to that of pretrained CLIP. These findings indicate that GSD reduces reliance on identity-related semantic priors while retaining useful forensic information.

RepresentationSimSwapCDv2DFDC
Pretrained63.166.164.0
Fine-tuned projection74.172.466.4
Fine-tuned75.675.371.9
GSD projection69.569.461.7
GSD77.078.475.9

Thank you again for your valuable time and efforts in reviewing this paper. We have revised the paper following your constructive comments. Your valuable feedback has helped us present our work more clearly and comprehensively. If any questions remain unresolved, we appreciate the opportunity for further discussion and clarification.

Reviewer_Tko7OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the persistence of pretrained semantic priors in vision foundation models after forensic fine-tuning. To mitigate its potential impact on cross-domain generalization, the authors propose Geometric Semantic Decoupling (GSD), which estimates the dominant pretrained feature subspace using a frozen vision encoder and suppresses the corresponding components in the trainable forensic representations. The suppression strength is adaptively adjusted based on pretrained geometry occupancy. The paper further introduces Batch-SVD to reduce subspace estimation overhead and evaluates the framework on face forgery and synthetic-image detection benchmarks.

Critical strengths

1.The proposed perspective complements existing approaches that primarily focus on biases acquired during downstream training.

2.The proposed perspective complements existing approaches that primarily focus on biases acquired during downstream training.

3.The paper provides extensive analyses of suppression strength, insertion depth, subspace dimensionality, inference protocols, and computational efficiency, which help establish the practical effectiveness of the proposed approach.

Critical weaknesses

1.The proposed framework uses the top singular vectors of frozen pretrained features as a proxy for semantic priors. However, SVD identifies directions of dominant feature variation, which do not necessarily correspond to semantic information or domain-sensitive shortcuts. Table 11 shows that suppressing only the first principal direction already achieves performance close to the full configuration. It is unclear whether the improvements arise from removing harmful semantic priors or from more general effects of dominant-component suppression, such as reducing representation anisotropy.

2.The paper shows a correlation between pretrained geometry occupancy and detection errors, but this does not establish that semantic fallback is responsible for these errors. The observed relationship may also reflect differences in image content or manipulation difficulty.

3.The occupancy-guided suppression is not sufficiently justified. Although the authors compare against fixed suppression strengths, they do not consider learnable layer-wise constants, making it difficult to tell whether the gains come from occupancy-based adaptation or simply more flexible scaling. Moreover, high occupancy does not necessarily indicate harmful semantic information, so stronger suppression may also remove useful forensic features.

Questions for authors

1.Can the authors provide direct evidence that the top singular directions predominantly encode semantic priors rather than general high-variance feature components? How does GSD compare with random, low-variance, or explicitly semantic subspace suppression?

2.Since Batch-SVD estimates a shared subspace from the current inference batch, how sensitive are individual prediction scores to changes in batch composition? Could the authors also report end-to-end latency rather than SVD-only acceleration?

3 comments
AuthorsOpenReview ↗

Author Response to Reviewer Tko7(1/2)

We sincerely appreciate your positive comments and insightful suggestions! Below, we provide our point-by-point responses to each of your comments.

W1: It is unclear whether the improvements arise from removing harmful semantic priors or from more general effects of dominant-component suppression, such as reducing representation anisotropy.

Q1: Can the authors provide direct evidence that the top singular directions predominantly encode semantic priors rather than general high-variance feature components? How does GSD compare with random, low-variance, or explicitly semantic subspace suppression?

Response: Thank you for your keen observations. We would like to explain that GSD selectively reduces identity-related semantic priors while preserving useful forensic information.

1) Semantic specificity of the top singular directions. We have compared the identity similarity gap ΔS\Delta S under top- and middle-direction suppression of pretrained CLIP without additional training. The key observations suggest that identity-related priors are concentrated in the leading singular directions: suppressing these directions substantially reduces ΔS\Delta S, whereas suppressing the same number of middle directions produces much smaller changes.

Model / SettingFF++CDv2DFDC
Pretrained CLIP0.4280.4250.372
Top (K=8K = 8)0.3220.2860.292
Top (K=32K = 32)0.2110.1740.172
Top (K=64K = 64)0.1480.1170.114
Middle (K=8K = 8)0.4290.4200.368
Middle (K=32K = 32)0.4140.4060.356
Middle (K=64K = 64)0.3970.3900.341

2) Comparison with random and low-variance. Following your suggestion, we have compared different training strategies: random-direction suppression (K=64K=64), bottom-direction suppression (K=64K=64, corresponding to the smallest singular values), and top-direction suppression (K=1K=1). Random, bottom, and top-11 suppression also slightly reduce semantic priors, consistent with their limited performance gains. These empirical results suggest that stronger suppression of identity-related semantic priors tend to yield greater gains from GSD.We additionally note that, although Table~13 (revised paper) shows relatively small differences in frame-level AUC between K=1K=1 and K=64K=64, the ACC results reveal a clearer performance gap.

Table: Identity similarity gap ΔS\Delta S

Model / SettingFF++CDv2DFDC
Naively fine-tuned CLIP0.4120.4310.394
Random (K=64K = 64)0.2580.2590.296
GSD Bottom (K=64K = 64)0.2700.2410.275
GSD Top (K=1K = 1)0.2900.2630.260
GSD (K=64K = 64)0.2010.1450.182

Table: Detection Accuracy

StrategyCDv2CDv1DFDCDFDCP
Random (K=64K = 64)81.381.776.579.8
GSD Bottom (K=64K = 64)81.882.076.178.4
GSD Top (K=1K = 1)81.381.777.081.3
GSD (K=64K = 64)83.182.877.082.3

3) Preservation of forensic information. As a reference, we have additionally trained deepfake linear probes on both decision features and their suppressed projections of GSD, using pretrained CLIP as a reference. The projected features exhibit weaker forensic discriminability than the decision features, with performance closer to that of pretrained CLIP. These findings indicate that GSD retains more useful forensic information.

Representation (ACC)SimSwapCDv2DFDC
Pretrained63.166.164.0
Fine-tuned projection74.172.466.4
Fine-tuned75.675.371.9
GSD projection69.569.461.7
GSD77.078.475.9

W2: Whether semantic fallback explains the relationship between pretrained geometry occupancy and detection errors?

Response: Motivated by your comment, we have grouped samples misclassified by fine-tuned CLIP into quintiles according to their occupancy r. Within these fixed quintiles, we evaluated GSD’s correction rate and each quintile’s share that remained uncorrected, averaging both metrics equally across real and fake classes. Correction rates generally increased with r, whereas the highest-occupancy quintile accounted for the smallest share of remaining errors. These findings show that the proposed method improves detection by suppressing semantic fallback, and preferentially corrects errors in high-occupancy samples.

AuthorsOpenReview ↗

Author Response to Reviewer Tko7(2/2)

DatasetOccupancy rr rank (%)Correction rate (%) ↑Remaining-error share (%)
CDv20–2029.6029.52
CDv220–4036.9820.53
CDv240–6036.4718.08
CDv260–8042.0616.15
CDv280–10046.4215.71
FF++0–2010.4423.14
FF++20–4017.7620.84
FF++40–6023.4620.40
FF++60–8039.0019.71
FF++80–10046.1615.91

W3: The occupancy-guided suppression is not sufficiently justified.

Response: Thank you very much for the valuable suggestion! We have compared learnable layer-wise constants and a learnable constant shared across layers. For your reference, the empirical results shown in the table indicate that occupancy-based adaptive modulation achieves better performance than both alternatives.

SettingCDv2DFDCe4sInSwap
Layer-specific82.876.792.684.8
Shared across layers82.676.592.284.5
GSD83.177.093.585.1

Q2: The stability of predictions under different batch compositions.

Response: As per your comments, we have evaluated each image in five different randomly sampled batches. Across the tested datasets, more than 98% of images received the same predicted label in all five runs. For your reference, we have added discussions of Inference sensitivity to batch composition in Section 4.4 of the main text. The results show that the prediction probability curves (Figure6 in main paper) closely overlap across the five runs. Accuracy varies by only 0.10 and 0.16 percentage points on CDv2 and SimSwap, respectively. These results suggest that predictions remain largely stable under random changes in batch composition.

DatasetRun 1Run 2Run 3Run 4Run 5
CDv283.1283.1583.0583.1183.08
SimSwap83.9484.1083.9483.9583.98

Q2: Could the authors also report end-to-end latency?

Response: To measure inference latency, we have computed the inference time per image of GSD-CLIP and naively fine-tuned CLIP over five runs. The results show per-sample inference incurs a high computational cost, while batch inference improves efficiency by amortizing subspace estimation through Batch-SVD. Reference-based inference uses a precomputed basis from the pretrained model, bringing the inference latency close to that of naively fine-tuned CLIP.

Method (ms/per image)Batch size = 1Batch size = 128
Naively fine-tuned CLIP7.564±0.0707.564 \pm 0.0701.207±0.0011.207 \pm 0.001
GSD (Per-sample Inference)65.158±1.26265.158 \pm 1.262–
GSD (Batch Inference)–5.217±0.0035.217 \pm 0.003
GSD (Reference-based Inference)8.546±0.1938.546 \pm 0.1931.254±0.0011.254 \pm 0.001

We sincerely appreciate the time and effort you have devoted to reviewing our paper. We have revised the paper following your insightful comments. We believe our paper has benefited a lot from your comments, and its presentation is now clearer and more complete. Thanks again!

Thank you for the additional experiments and clarifications, which address most of my concerns.

Upon further examination of the experimental comparisons, I noticed that several recent relevant works [1–4] have not been included. I encourage the authors to consider these works, as some address overlapping tasks and share similar evaluation benchmarks. The authors should also clarify the differences between these approaches in the final version.

Overall, I maintain my original score.

[1] Beyond [CLS] Token: Query-Driven Token-Level Forgery Purification for Generalizable Deepfake Detection, CVPR 2026

[2] DFD-HR: Generalizable Deepfake Detection via Hierarchical Routing Learning, CVPR 2026

[3] Unleashing Vision-Language Semantics for Deepfake Video Detection, CVPR 2026

[4] Forensics Adapter: Adapting CLIP for Generalizable Face Forgery Detection, CVPR 2025

Comments on the paper

1 comment

Dear reviewers,

As the discussion period approached the end, please take a moment to read the authors’ responses and share your thoughts on any remaining points of discussion. If the discussion has changed your assessment, please update your review accordingly.

Thank you for your time and contributions!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.