acceptodds
Withdrawn from ICLR 2027

Periodic Gradient-Variance Suppression on SVD-Adapted Residuals for Cross-Generator Deepfake Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

As deepfake generation methods diversify rapidly, detectors trained on a fixed set of manipulation techniques increasingly fail to generalize to unseen generation paradigms, undermining their real-world reliability. Parameter-efficient adaptation of frozen vision backbones, such as SVD-decomposed low-rank residuals, enables efficient deepfake detection but offers no mechanism to distinguish generalizable forgery signal from manipulation-specific shortcuts within the few directions it is allowed to adapt. We study whether environment-conditioned gradient disagreement across known manipulation methods can identify which trainable spectral coefficients in a frozen CLIP residual are reliable, and whether this signal differs meaningfully from temporal gradient instability, the criterion used by concurrent work. Through a controlled, same-seed comparison, we find that suppressing high-disagreement directions fails when applied once, but periodically refreshing the suppression mask matches a Fishr-style cross-environment baseline across three paired seeds. At matched intervention strength, this criterion also outperforms random and magnitude-based suppression controls, supporting an effect beyond simple capacity reduction. A matched FMSD-style temporal control performs consistently below the cross-manipulation approaches in the evaluated seeds, suggesting the environment signal contributes information beyond gradient instability alone. Leave-one-manipulation-out (LOMO) evaluation reveals a substantial transfer asymmetry that is not explained by simple gradient similarity between methods, with one manipulation method's detection collapsing to near-chance performance when held out entirely. On four unseen generators spanning diffusion and GAN-based synthesis, our approach improves over baselines concentrated on the diffusion-based DDIM generator, while the diffusion-transformer case remains difficult for all methods tested.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
deepfake detection, generalization, gradient variance, domain generalization, parameter-efficient adaptation, SVD

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_392qOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates cross-generator deepfake detection by periodically suppressing SVD-adapted residual coefficients based on gradient disagreement across manipulation methods. Building on a frozen CLIP backbone and Effort-style adaptation, the proposed approach periodically recalibrates coefficient-level suppression masks to reduce manipulation-specific shortcuts. Experiments on FaceForensics++ and DF40 show that periodic suppression performs comparably to continuous Fishr-style regularization and outperforms static, random, magnitude-based, and temporal-gradient suppression controls in the reported settings. Additional analyses examine gradient stability across seeds, weight-space interpolation, and transfer asymmetry across manipulation methods.

Critical strengths
  1. The paper provides controlled comparisons among static, periodic, and continuous gradient-variance interventions. The paired-seed evaluations and additional suppression controls help identify the importance of periodically updating gradient-based masks rather than applying a fixed intervention.

2.The study investigates gradient-statistic reproducibility and leave-one-manipulation-out transfer, revealing substantial asymmetry across manipulation methods. These findings provide useful insights into the limitations of gradient-based invariance criteria for deepfake detection.

Critical weaknesses
  1. The proposed method largely combines established components, including Effort's SVD-based adaptation, Fishr-style gradient variance, and periodic masking. More importantly, its performance is nearly identical to continuous Fishr-style regularization across the reported seeds, without offering computational advantages. Consequently, the incremental methodological contribution appears insufficiently substantial for ICLR.

  2. The evaluation is restricted to a single frozen backbone and adaptation architecture, with relatively few random seeds. Performance improvements are uneven across unseen generators, and the proposed approach does not resolve poor generalization to diffusion-transformer-generated images. Furthermore, the substantial failure on held-out NeuralTextures raises concerns about whether the gradient-disagreement criterion reliably identifies transferable forgery representations.

The paper presents a carefully designed empirical investigation of periodic gradient-variance suppression for cross-generator deepfake detection. The controlled comparisons and analysis of transfer asymmetry are informative. However, the proposed method offers limited methodological novelty, achieves performance comparable to existing continuous regularization without computational savings, and demonstrates inconsistent generalization across unseen generators. These limitations substantially weaken its significance as a methodological contribution.

Questions for authors

Given that periodic recalibration achieves nearly identical performance and computational cost to continuous Fishr-style regularization, what is its substantive advantage beyond producing an inspectable suppression mask?

Does the proposed method consistently outperform existing approaches when evaluated with additional backbones, adaptation strategies, and unseen generation methods, particularly diffusion-based generators?

How do the authors explain the near-chance performance on held-out NeuralTextures despite relatively high gradient similarity to the other manipulation methods? Does this suggest a fundamental limitation of gradient disagreement as an indicator of transferable forgery features?

Reviewer_2bqUOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes periodic recalibration, a training-time method that suppresses trainable coefficients whose gradients disagree across known manipulation methods. It builds on Effort, but trains only the smallest singular value of each attention projection in a frozen CLIP backbone. The authors score each coefficient by its gradient disagreement across FF++ manipulation methods and convert the score into a soft suppression mask. They report that computing the mask once gives no gain over the unregularized baseline, while computing the mask periodically performs close to a Fishr-style continuous penalty on unseen DF40 generators. The paper also includes temporal, random and magnitude controls, a weight-space interpolation, a cross-seed reproducibility analysis and a leave-one-manipulation-out (LOMO) evaluation. The authors claim that periodic recalibration is equivalent to continuous regularization at the same cost, and present the fixed, inspectable mask as an interpretability advantage.

Critical strengths
  • The paper distinguishes its cross-environment, per-coefficient signal from temporal, layer-level masking, and comparisons share the same architecture and evaluation set.
  • The authors report negative results candidly. The failure of static suppression, the withdrawn efficiency claim, and the unexplained LOMO asymmetry are all reported.
  • The controls are well chosen in principle. Random and magnitude suppression test capacity reduction, and the granularity-isolation control separates the criterion from mask resolution.
  • The authors report the reproducibility of the instability scores at two levels, the overall ranking across seeds and the identity of the top-flagged scalars, preventing over-interpretation of individual entries.
Critical weaknesses
  • It is unclear whether the gain over the unregularized baseline holds. Plain Effort is reported for a single seed, and the seed variation of periodic recalibration is about as large as its gain.
  • The mechanistic claims rest on limited and unmatched controls. Most controls have only one or two seeds. The random and magnitude controls suppress a fixed subset once, whereas periodic recalibration applies a graded mask that changes every cycle. The temporal control is the authors' reconstruction, and its scoring is not described.
  • The equivalence claim is not supported. A sign that flips across three seeds does not distinguish equivalence from a small effect, and no confidence interval or equivalence margin is reported. The flat interpolation in Figure 3 is weak evidence for models that share their initialization and differ only in the residual scalars and a linear head, and no reference path is given.
  • Some reported results are inconsistent or incomplete. The Fishr-style seed-1 per-generator values average to more than its reported overall AUC, whereas every other complete row matches its average (Table 8). Per-generator results are also missing for most rows, so the claim that gains concentrate on DDIM cannot be verified.
  • The contribution is limited. Periodic recalibration performs close to a Fishr-style penalty at the same cost, so its main advantage is the inspectable mask, which is not shown to be informative. All main results also come from a short pilot setting, and the full-scale runs score below the pilot baseline without matched comparisons.
Questions for authors
  • How does Plain Effort perform at seeds 2 and 3? Does it see the same number of training images as the environment-conditioned methods within the same number of steps?
  • How does a control perform if the mask values are randomly permuted across scalars at each recalibration?
  • The note under Table 9 states that the identical seed-2 random and magnitude results are genuine and refers to Section 5.2, which discusses these controls only at seed 1. Which scalars did each control suppress at seed 2?
  • Can you report a confidence interval for the paired differences in Table 2?
  • Are the endpoints of Figure 3 the same seed-1 checkpoints as in Table 1, and is the suppression mask applied during interpolation? The plotted values appear slightly lower than those in Table 1.
  • How is the overall AUC in Table 8 computed? In the Fishr-style seed-1 row, the per-generator values average to more than the reported overall AUC.
  • How much does the mask change between consecutive cycles and what does inspecting it reveal about which scalars carry generalizable signal?
Reviewer_sYkAOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies periodic gradient-variance suppression for cross-generator deepfake detection. Using a scalar-only Effort variant on CLIP ViT-L/14, it periodically refreshes suppression masks over 96 trainable SVD residual coefficients based on gradient disagreement across forgery methods. Experiments use FaceForensics++ for training and four unseen generators from DF40 for evaluation, with comparisons against static suppression, Fishr-style regularization, and targeted masking controls.

Critical strengths
  • The paper explores using cross-environment gradient variance across forgery methods to guide periodic suppression. The idea is intuitive, and the motivation for refreshing the suppression mask as training progresses is reasonable.

  • The temporal-gradient, random, magnitude-based, and layer-level controls examine which aspects of the intervention contribute to its performance.

Critical weaknesses
  • Limited added benefit and uncertain robustness across seeds. The gain is marginal, while its AUC varies noticeably across seeds (0.9068–0.9191).

  • I am unsure whether such a fine-grained analysis of the 96 scalars in a specific Effort variant, evaluated only on CLIP ViT-L/14, provides meaningful or broadly applicable insights. The current version is closer to a detailed case study of a specific architecture and is not yet sufficient to support more general conclusions.

  • The writing feels somewhat fragmented, especially in Secs. 5.1–5.3, where frequent short paragraphs make the discussion harder to follow.

Questions for authors
  • Why are Plain Effort and Static suppression reported for only one seed in Table 1?
Reviewer_hPmbOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies periodic suppression of SVD residual coefficients in a frozen CLIP backbone for cross-generator deepfake detection. It constructs manipulation-conditioned gradient statistics, converts them into multiplicative masks, and refreshes these masks during training. Experiments compare static and periodic suppression with continuous regularization, temporal masking, and simpler controls. The paper also examines weight interpolation, cross-seed diagnostic stability, and leave-one-manipulation-out transfer. It reports that periodic suppression achieves similar aggregate AUC to the continuous baseline, with comparable training cost.

Critical strengths
  1. The paper investigates a relevant question: whether cross-manipulation gradient disagreement can guide parameter-efficient adaptation for detecting unseen generators.
  2. The comparison design includes shared model parameterization and paired seeds for the principal interventions, alongside random, magnitude-based, and granularity controls.
  3. The authors explicitly report negative findings, including unsuccessful static suppression, difficult held-out transfer, and the absence of a training-cost advantage.
Critical weaknesses
  1. The “Plain Effort” baseline freezes both singular-vector factors and trains only 96 residual scalars plus a classifier, substantially changing the original method [R1]. These comparisons therefore establish behavior only within this restricted variant. Please rename the baseline, include a faithful implementation of the original method, and add a frozen-backbone linear probe to isolate the contribution of the residual scalars.

    [R1] Orthogonal Subspace Decomposition for Generalizable AI-Generated Image Detection. ICML, 2025.

  2. Fishr matches within-environment variances of per-sample gradients [R2], whereas Section 3.2 measures variance across environment-mean gradients. The “Fishr-style” baseline lacks an explicit objective, penalty coefficient, and description of its detached-gradient approximation. Please specify the statistic being penalized and the resulting parameter update, including which quantities are detached and how the penalty contributes nonzero gradients. This clarification is essential to interpreting the central comparison.

    [R2] Fishr: Invariant Gradient Variances for Out-of-Distribution Generalization. ICML, 2022.

  3. Static suppression uses a 300-step calibration period followed by 500 training steps, whereas periodic suppression first intervenes after 150 steps. Random and magnitude controls also use fixed masks and strengths. These comparisons therefore confound refresh frequency with intervention timing and dynamics. Please match the initial calibration period, intervention duration, and suppression-strength distribution, and include random masks refreshed on the same schedule.

  4. The sign changes in Table 2 do not establish statistical equivalence. Please justify a practically meaningful equivalence margin and report paired confidence intervals or an appropriate equivalence test. Repeated paired runs are also needed for the plain and static baselines, whose single reported seed leaves training variability unquantified.

  5. Figure 3 evaluates AUC along one interpolation path on one test set. Similar AUC values do not establish equivalent predictions or a shared low-loss region. Please report loss and prediction disagreement along the path, include additional seed pairs and a plain-baseline interpolation control, and clarify whether the final masks were folded into effective weights before interpolation and whether the classification heads were interpolated.

  6. In Table 8, the continuous-regularization baseline’s generator AUCs average to 0.9255, versus the reported overall 0.9192. With equal generator counts, a common real-image set, and identical scoring, these quantities should agree. Figure 3’s endpoints also appear inconsistent with Table 1. Please reconcile the evaluation subsets, checkpoints, and plotted values, and provide the underlying scores or result files.

  7. Section 3.4 draws batches from all four environments, but the reported batch size does not establish equal sample exposure per update across methods. Please specify batch composition and gradient aggregation, and compare against unregularized training with matched sampling and updates. Also reconcile “3,000 per class” with “3,000 per environment,” and document validation data, checkpoint selection, and hyperparameter selection.

Questions for authors
  1. Could the authors clarify how the minimum-signal gate distinguishes weak gradients from strong gradients that cancel across environments? For environment means (1,−1,1,−1)(1,-1,1,-1), μˉi=0\bar{\mu}_i=0 yields qi=0q_i=0 and mi=1m_i=1, despite substantial disagreement. Is this behavior intentional because the net gradient vanishes, and what evidence supports this choice over a gate based on gradient energy?

  2. Could the authors clarify whether the residual scalars are constrained and how mask changes interact with optimizer states? With fixed positive masks and unconstrained scalars, sieff=misis_i^{\mathrm{eff}}=m_i s_i preserves the representable model class while changing optimization. Why do the reported growth ratios rule out compensation without comparing the same coefficients under matched interventions?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.