acceptodds
Withdrawn from ICLR 2027

CIFT: Donor-Grounded Relational Gap Learning for Source-Free Face Forgery Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Face-forgery detectors often generalize poorly across synthesis pipelines because visual artifacts change with the renderer. We study whether source information available during training can provide a more transferable supervisory signal without being required at deployment. We introduce CIFT (Consistency-Invariant Forensic Training), a privileged-learning framework that uses a donor reference only during training. For swaps, the reference is the identity donor; for reenactment, it is the driving/source reference and defines a weaker relational discrepancy rather than identity replacement. XID-Mamba separately encodes the observed and donor streams with bidirectional Mamba modeling and symmetric cross-attention, while Type-Conditioned IGS structures their relation through complementary directional and magnitude constraints. Auxiliary diffusion regularizes the stream representations. At inference, all donor-dependent relational and diffusion components are discarded, leaving a single-image source-free detector. Trained only on FaceForensics++ (c23), CIFT generalizes strongly across five held-out benchmarks, remains effective under an encoder- and diffusion-prior-aligned comparison with DiffusionFake, and degrades under donor-correspondence interventions that replace the true donor with non-corresponding alternatives.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
face forgery detection, deepfake detection, privileged learning, representation learning, source-free inference, donor-grounded learning, cross-dataset generalization, Mamba

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_3UPtOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CIFT, a face-forgery detection framework based on learning using privileged information. During training, it uses donor references as privileged relational inputs. A dual-stream Mamba module (XID-Mamba) with symmetric cross-attention models donor–target relationships, while type-conditioned Identity Gap Supervision (IGS) constrains their direction and magnitude. These training objectives transfer donor information into target representations while preserving a target-only classification path. A frozen Stable Diffusion branch provides auxiliary denoising regularization. Cross-dataset experiments demonstrate promising generalization to unseen forgery methods and synthesis pipelines, without requiring donor references or diffusion computation at inference.

Critical strengths
  • The motivation of the paper is clear: existing methods mostly rely on artifact clues such as spatial, frequency, blending, or reconstruction. However, with the development of manipulation techniques and the continuous changes in synthesis methods, current methods face the core challenge of how to ensure generalization when the synthesis pipeline changes. Motivated by this, the paper designs a method based on Learning Using Privileged Information (LUPI), which uses donor information as privileged information available only during training to improve the generalization ability of the method.

  • During training, the method uses the actual donor reference as privileged relational supervision to shape the target representation, while entirely discarding the donor at inference and using only the single observed image as input. This not only enhances the capability of feature extraction but also ensures the fairness of comparison with other baseline methods under the single-image inference setting.

  • Extensive comparisons and component-level ablations provide substantial empirical support for the method’s generalization performance and help establish the contributions of its main components.

Critical weaknesses
  • The paper attempts to ensure the consistency of extracted features between the training and inference stages by employing an identical classification path in both stages, rather than relying on explicit alignment mechanisms for constraint. That is, the paper does not seem to provide statistical evidence demonstrating that the feature distribution under relational supervision during training is consistent with the feature distribution without such supervision during inference.

  • The paper defines "source-free inference" as "requiring no donor data at inference." However, in existing literature, "source-free" has an established meaning, generally referring to source-free domain adaptation, where source domain data is available during training but unavailable during adaptation, and "source" refers to the source domain, which is unrelated to the donor reference. Such phrasing is likely to cause confusion for readers. It is recommended to modify the terminology or add a footnote at its first occurrence to aid reader comprehension.

  • The paper employs a diffusion-based auxiliary denoising regularization branch but does not provide a detailed explanation of the rationale behind this choice. My question is: why must it be diffusion regularization specifically? It is recommended to compare it with other simpler regularization methods.

  • The paper only compares the inference parameters and computational cost of different relational heads in Table 13. However, this is an internal variant comparison within the model (and the inference parameter counts are identical), lacking a comparison of inference resources with other methods such as DiffusionFake, SBI, and UCF. The efficiency of the proposed method remains to be verified. It is recommended to supplement experiments on the inference parameters, per-frame FLOPs, and latency on the same hardware across the respective methods.

  • Some figures in the paper are confusing. For example, Section 3.1 (Line 159) states: "For face swapping, x_s is the identity donor associated with I_s, while x retains target-conditioned context associated with I_t." This is intended to specify that x_s serves as the identity provider. However, in Figure 1, x_s is depicted as an original image before face-swapping, rather than an identity provider. This contradicts the definition in the main text and makes it difficult for readers to understand. In addition, some small text in this figure is unclear.

  • The paper compares numerous methods to demonstrate the effectiveness of the proposed approach. However, most of the compared baselines were published before 2025 (although it includes 2025 works like SELFI, the results are incomplete). Adding more methods from 2025 and beyond to the comparative experiments would be more convincing.

Questions for authors

see weakness

Reviewer_FFs9OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes CIFT, a privileged-learning framework for generalizable face forgery detection that leverages donor reference images during training while maintaining single-image inference. The proposed XID-Mamba module models the relationship between observed and donor faces through bidirectional Mamba and cross-attention. Type-Conditioned Identity Gap Supervision (IGS) further constrains the learned relational representations using cosine and Euclidean distance objectives, while an auxiliary diffusion pathway provides additional regularization.

The method is trained on FaceForensics++ and evaluated across multiple held-out benchmarks, including Celeb-DF, WildDeepfake, DFDC-Preview, DFD, and DiffSwap. The authors report strong cross-dataset generalization and provide extensive ablations covering different model components, backbones, and donor-reference assignments.

Critical strengths
  1. A practically relevant privileged-learning formulation.

Using donor information during training while preserving source-free inference is a meaningful direction for face forgery detection. The proposed framework attempts to exploit information about the manipulation process without introducing additional reference requirements during deployment. This is practically appealing and provides an interesting alternative to directly learning manipulation artifacts from observed images alone.

  1. Strong empirical performance with substantial experimental analysis.

CIFT achieves promising cross-dataset detection results across several benchmarks. The paper provides component-wise ablations, backbone comparisons, donor-correspondence interventions, and additional robustness evaluations. In particular, the donor-assignment experiments provide useful evidence that the selection of the training-time reference influences the learned detector, rather than the improvement being solely attributable to the presence of an additional image.

Critical weaknesses
  1. The proposed identity-gap supervision lacks a sufficiently convincing forensic justification.

The central IGS objective encourages genuine donor–target pairs to remain close while pushing forged pairs apart in the learned relational space. However, the relationship between identity discrepancy and image manipulation is not straightforward.

For example, in face swapping, the manipulated face is generally intended to preserve the identity of the donor. More importantly, identity-preserving manipulations such as self-blending (SBI), face reenactment, and same-identity editing can introduce genuine forgery artifacts without necessarily producing an identity discrepancy. Although the authors adopt a smaller margin for reenactment, the objective still explicitly encourages separation between forged images and their corresponding references.

This raises an important question about what the proposed relational gap actually represents. Since the embeddings are learned through label-dependent supervision, a large gap for forged pairs may reflect the imposed training objective rather than an independently meaningful identity discrepancy. The reported separation in the learned relational space therefore does not, by itself, validate the proposed interpretation.

The method may still be effective as an auxiliary metric-learning strategy, but stronger evidence is needed to establish that the learned relation captures transferable forensic information rather than label-conditioned separation.

  1. The methodological complexity is substantial, while the necessity of the proposed relational modeling remains insufficiently isolated.

CIFT combines multiple components, including a ConvNeXt-V2 backbone, global and spatial classification heads, Mamba-based relational modeling, symmetric cross-attention, auxiliary diffusion regularization, and several geometric constraints.

Although the authors provide extensive ablations, the results show that a substantial portion of the performance improvement comes from the deployed classification architecture and donor-side diffusion supervision. The additional gain from XID-Mamba before introducing IGS is comparatively modest. Furthermore, the strongest overall results are obtained with a different backbone and classification architecture from several existing methods, making direct attribution of the performance advantage challenging.

While the encoder-aligned comparison with DiffusionFake partially addresses this concern, comparisons with simpler donor-based supervision strategies under the same backbone, training configuration, and inference architecture would provide stronger evidence for the necessity of XID-Mamba and the proposed geometric objectives.

Questions for authors

Identity-preserving forgeries and the interpretation of IGS: How does the proposed identity-gap supervision apply to identity-preserving manipulations such as SBI, face reenactment, or same-identity face editing? If the manipulated image preserves the donor identity, why should the relational objective explicitly push the two representations apart? Since SBI augmentation is also used during training, could the authors clarify whether SBI-generated samples participate in IGS and how their donor references and labels are defined?

Validation of the learned relational gap: Could the authors compare the learned donor–target gap with identity similarity measured by an independently pretrained face recognition model? This would help clarify whether the proposed gap reflects identity discrepancy, manipulation-related artifacts, or a label-dependent representation learned specifically for classification.

Necessity of the proposed relational architecture: Could the authors compare XID-Mamba and IGS against simpler donor-based feature alignment or contrastive learning objectives while keeping the backbone, classification heads, and diffusion supervision unchanged? Such comparisons would help establish whether the architectural complexity is necessary for the reported generalization improvements.

Reviewer_1HTTOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies how source-target relations inherent in deepfake datasets can assist the learning of a generalizable deepfake detector. It proposes a training framework that uses an auxiliary branch to model these relations. Particularly, training-only modules pull forged pairs, which matches fake images with their sources, away from each other. Experiments demonstrate strong results on commonly used benchmarks. Ablation studies show clear improvements with the introduction of each module, though some improvements raise questions.

Critical strengths
  • It is a reasonable choice to explicitly leverage source-target pairing from deepfake dataset for training. Although not entirely new, it is relatively less explored in the field.
  • Modeling their relationship with a training-time auxiliary branch ensures practicality.
  • Performances on cross-dataset evaluation achieves the state-of-the-art. Consistent improvement over bare backbones.
  • Varying the choices of donor beyond the correct pairing helps with understanding the contribution of relational gap modeling.
Critical weaknesses
  • The overall method is unnecessarily complicated. It includes a number of contributions that are not immediately related to the donor-centered motivation, including multiple heads, diffusion reconstruction, Mamba, etc. I'd be interested in how much the losses introduced in Section 3.3 work directly on a bare backbone, which would significantly strengthen the core claim of the paper.
  • Global Head and SMC appears to be heads added on top of a shared feature extractor. The main text does not specify the concrete design choices or motivations behind them, yet they collectively account for a +13.77 improvement. This is somewhat counter-intuitive given my experience with the task.
  • The paper doesn't provide much intuition behind donor selection, which matches swapped images with the identity sources, and at the same time matches reenactment images with driving sources. One could argue that both sources of a fake image could be used for training.
  • The method introduces many intuitively set hyperparameters. Section 3.3 alone introduces five margins that are set without any justifications.
  • Severe readability issue. Abbreviated terminologies and math symbols are constantly mentioned without any formal introduction. The full name of XID-Mamba is not even once mentioned in the main paper.
  • The method suggests that the identity-supplying image works as the donor in face swap, i.e. the resulting manipulation should have the same identity as the donor. The main figure suggests otherwise.
Questions for authors
  • Please provide the ablation studies mentioned in "weaknesses".
  • Please provide some intuitions or sensitivity analysis for the hyperparameters introduced in Section 3.3

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.