acceptodds
Desk rejected at ICLR 2027

HyperDetect: Hierarchy-Aware Representation Learning for Generalizable Deepfake Detection

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Generalizable deepfake detection requires representations that transfer across both manipulation techniques and visual domains. Existing detectors often learn generator-specific appearance cues while overlooking the hierarchical relationships among forgery families and the style changes induced by domain shifts. We introduce HyperDetect, a vision-language framework that represents forgery hierarchies in Lorentz hyperbolic space. Its learning objective combines hyperbolic angular contrastive learning with partial-order constraints, hierarchical entailment loss, and domain dispersion regularization. Together, these components exploit negative curvature to organize related forgery types, learn discriminative subtype representations, and address domain variation and class imbalance. Evaluations on multiple deepfake benchmarks demonstrate improved generalization to unseen manipulations compared with existing methods. The results highlight the value of explicitly modeling forgery hierarchies for transferable multimedia forensics. Code and data will be released publicly.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
deepfake detection, hyperbolic representation learning, vision-language models, domain generalization

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_saMkOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes HyperDetect, a hierarchy-aware deepfake detection framework that models real images, generic fake concepts, and specific forgery types in hyperbolic Lorentz space. It introduces hyperbolic angular contrastive learning to separate different forgery families, a hierarchical entailment loss to preserve parent-child relationships, and a domain dispersion regularizer to prevent different forgery domains from collapsing together. During inference, the model first performs real/fake detection and then further attributes fake samples to specific manipulation types.

Critical strengths

This paper provides a structured way to model relationships among different forgery types instead of treating them as independent flat classes. The combination of hyperbolic representation and hierarchical constraints makes the learned feature space more interpretable and explicitly preserves both shared forgery semantics and differences among specific manipulation types.

Critical weaknesses
  1. Compared with conventional deepfake attribution that already distinguishes different manipulation types, what additional benefit is brought by explicitly imposing the proposed hierarchical structure, especially for unseen forgery generalization?

  2. There are obvious formatting problems in the current manuscript. Line 304 even begins with “.”, and such an error should not appear in an academic paper. The font sizes of the three tables on Page 7 could also be further unified. The numbering before the descriptions corresponding to Equations (11) and (12) is also not standardized. The obvious misalignment in Tables 5 and 6 should not exist either. I also suggest unifying the reference format. Overall, I strongly suggest that the authors carefully revise it.

  3. The paper lacks a Related Work section.

  4. The main motivation, “moving beyond flat classification by exploiting hierarchical structures for better deepfake generalization,” has already been widely studied. Prior works have explored coarse-to-fine supervision, hierarchical labeling, hyperbolic representations, and hierarchical routing. Therefore, the current paper mainly repackages an established idea with a specific hyperbolic formulation, and its methodological novelty appears limited.

Questions for authors

The main motivation of this work has already been extensively explored in previous studies, including coarse-to-fine supervision, hierarchical labeling, hierarchy-aware representation learning, and hyperbolic modeling. Therefore, the motivation itself is not novel. Although this paper introduces a specific combination of angular contrastive learning, hierarchical entailment, and domain dispersion in hyperbolic space, it is currently unclear what fundamentally new capability or insight this formulation provides over existing hierarchical deepfake detection approaches.

3 comments
AuthorsOpenReview ↗

Response to Reviewer saMk (Part 1 of 2: Q1 & Novelty)

We thank Reviewer saMk for the candid feedback and for identifying specific formatting and structural issues in our manuscript. Due to OpenReview's character limit, we divide our response into Part 1 (Methodological Novelty) and Part 2 (Related Work & Formatting Revisions).

1. Additional Benefit Over Conventional Deepfake Attribution (Q1)

Conventional deepfake attribution treats manipulation types as flat, mutually exclusive categories (e.g., KK-way cross-entropy over Real,DF,F2F,FS,NT\\{ \text{Real}, \text{DF}, \text{F2F}, \text{FS}, \text{NT} \\}). This flat formulation suffers from a fundamental limitation on unseen manipulations: when a novel generator (e.g., SimSwap or e4s) appears at test time, it does not belong to any seen leaf class, causing flat attribution models to either misclassify it as Real or suffer severe confidence drop.

  • Quantitative Proof (Table 9): When we train our exact hyperbolic architecture as a flat classifier (Ours-Flat) versus our hierarchical formulation (Ours-Full), Tree Induced Error (TIE) drops from 0.42 to 0.08, and cross-dataset AUC improves from 90.2% to 92.9% (Table 8).
  • Mechanism: By explicitly modeling the partial order Specific Type≺Generic-Fake\text{Specific Type} \prec \text{Generic-Fake}, the Level-1 GENERIC-FAKE cone acts as a semantic safety net. Even when an unseen manipulation escapes all Level-2 leaf cones, it still shares high-level ancestral forgery cues and remains captured inside the Level-1 GENERIC-FAKE parent cone. We further verified this via an Out-of-Taxonomy experiment: removing all face-swapping prompts from the taxonomy only reduces CDF-v2 AUC slightly from 96.3% to 93.8%, whereas flat attribution models drop sharply.
AuthorsOpenReview ↗

Response to Reviewer saMk (Part 2 of 2: Prior Hyperbolic Comparison, Related Work & Formatting)

Response to Reviewer saMk (Part 2 of 2)

2. Fundamental Methodological Novelty Beyond Prior Hierarchical/Hyperbolic Works

While coarse-to-fine labels and hyperbolic geometry have been explored in general classification and physical Face Anti-Spoofing (FAS, e.g., Hu et al., 2024; Han et al., 2023), directly transferring existing hyperbolic formulations to digital deepfake detection fails both theoretically and empirically, resulting in a 7.1% to 13.3% AUC drop (Appendix Table 14). HyperDetect introduces distinct theoretical solutions and forensic capabilities not present in prior work:

  1. Solving the Geometric Tension Between Inclusion and Separation (Propositions 1 & 2): Prior hyperbolic methods minimize geodesic distance, pulling diverse forgery subtypes toward the origin (Radial Collapse, Proposition 2) where hyperbolic space degenerates into Euclidean geometry and loses angular resolution. Conversely, enforcing rigid parent cones while separating diverse sub-domains forces child clusters outside the parent boundary (Multi-Domain Cone Escape, Proposition 1). HyperDetect decouples angular similarity (LhFC\mathcal{L}_{hFC}) from radial depth and combines it with soft-cone Hierarchical Entailment (LhFE\mathcal{L}_{hFE}) and Riemannian Domain Dispersion (LhDD\mathcal{L}_{hDD}), utilizing the exponentially expansive boundary of Lorentz space (r∈[2.3,4.1]r \in [2.3, 4.1]).
  2. Controlled Isolation of Hyperbolic Gain: Equipping an Euclidean model with the exact same hierarchical prompt taxonomy yields only 89.4% average AUC, whereas HyperDetect's Lorentz geometry achieves 92.9% (+3.5% net gain).
  3. New Forensic Capabilities: Beyond binary detection, our radial-angular geometry enables multi-level forensic explanation retrieval that outperforms SIDA-13B with a 69%–91% human preference win rate (Fig. 8).

3. Addition of a Dedicated Related Work Section & Complete Formatting Revision

We agree that a dedicated Related Work section and careful formatting polish significantly improve the manuscript. In our revised PDF, we have completed all of the following updates:

  • Added Section 2 (Related Work): Synthesizing (i) Generalizable Deepfake Detection, (ii) Hierarchical & Language-Guided Forensics, and (iii) Hyperbolic Representation Learning, explicitly contrasting prior distance-based objectives with our angular-radial decoupling.
  • Line 304 Typo: Removed the stray leading period "." and revised the sentence transition.
  • Table & Equation Consistency: Unified font sizes across Tables 1–3 on Page 7, fixed column alignment in Tables 5 and 6, and standardized equation descriptions/punctuation following Equations (11) and (12).
  • Unified Bibliography: Removed duplicate entries (e.g., Liu et al. 2024a/b), completed missing venue citations, and standardized all BibTeX entries to ICLR format.

I thank the authors for their response. After considering it, I have decided to maintain my original score.

Reviewer_ignGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the cross domain / cross manipulation generalization problem for deepfake detection. The authors point out that existing flat classification pipelines treat different forgery types as independent labels and ignore their coarse to fine hierarchical genealogy, leading to over fitting towards dataset specific superficial artifacts. The authors propose HyperDetect, a vision language framework built upon Lorentz hyperbolic space. Three complementary loss components are designed:

  1. Hyperbolic Angular Contrastive Learning (L_hFC): performs contrastive learning based on angular similarity instead of geodesic distance, to avoid radial collapse while separating sibling forgery families.
  2. Hierarchical Entailment Loss (L_hFE): enforces parent child semantic inclusion, keeping fine grained forgery subtypes inside the semantic cone of their high level parent concept.
  3. Domain Dispersion Regularizer (L_hDD): pushes domain centroids apart in hyperbolic space to mitigate domain imbalance and dominant domain monopoly of embedding space.
Critical strengths
  1. Novel geometric perspective for deepfake detection: Most deepfake detectors work in Euclidean space and focus on low level artifacts. This paper brings hyperbolic geometry together with vision language semantic anchors, explicitly modeling the hierarchical genealogy among forgery families. It provides fresh insights for generalization beyond texture level artifacts.
  2. Solid theoretical treatment: Two core propositions (radial collapse, cone escape) are formally stated with complete proofs in appendix. The authors identify why prior hyperbolic anti spoofing methods cannot directly transfer to deepfake detection, analysing geometric, topological and semantic failure modes. Theoretical claims are further backed by dedicated ablation tables.
  3. Well designed multi component framework: The three losses are conceptually complementary: angular contrast handles sibling separation, hierarchical entailment enforces parent child inclusion, domain dispersion regularizes inter domain centroid spacing. The vision language prompt driven prototype initialization is well motivated.
  4. Comprehensive empirical evaluation: Extensive benchmarks covering cross dataset, cross manipulation, common corruption robustness, GAN / diffusion generated images, multiple backbones. The paper includes hierarchical classification metrics, embedding visualization, attention heatmaps, and a human user study for generated forensic explanations. Ablations cover loss components, geometry choice, hierarchy depth, hard vs soft cone constraints.
  5. Clear limitation discussion: The authors honestly list key limitations: dependency on frozen CLIP feature space, requirement of predefined forgery taxonomy, performance bottleneck from upstream face detector.
Critical weaknesses

W1: Limited ablation on hyper parameter sensitivity and interaction Key hyper parameters include curvature k, cone initial aperture ω_0, slack coefficients ξ_1,ξ_2, , margin δ for domain dispersion. Only λ_1,λ_2 are analysed in Figure 5(b). The paper does not show how performance varies with k, ω_0, slack terms and delta. It remains unclear whether good performance relies on careful per dataset grid search. If multiple hyper parameters need dataset specific tuning, practical deployment burden increases. Additionally, interactions between hyper parameters are not studied.

W2: Theoretical assumptions are idealized; gap between theory and real training dynamics Proposition 1 and Proposition 2 are derived under ideal mathematical conditions. In real training, the combination of three losses may create competing gradients. The paper does not empirically measure whether real model embeddings satisfy theoretical properties: e.g. whether real samples cluster near origin, whether child forgery subtypes strictly stay inside parent cones, whether radial variance behaves as Proposition 2 predicts. While t SNE visualization is provided, it is qualitative only, lacking quantitative geometric statistics (radial norm distribution, angular separation statistics across classes).

W3: Limited adversarial robustness evaluation Deepfake detection systems deployed online face adversarial image perturbations designed to fool detectors. However, the paper only evaluates natural corruptions (blur, noise, pixelation). There is no adversarial attack evaluation (white box PGD / Auto PGD adversarial perturbations targeted to evade real fake prediction). Without adversarial robustness test, we cannot know if the hyperbolic hierarchical representation remains robust under intentional adversarial manipulations. This is a major missing piece for safety critical forgery detection applications.

W4: Comparison with other vision language hierarchical representation methods is insufficient The paper mainly compares against conventional deepfake detectors. It only briefly cites HyCoCLIP, but lacks systematic comparison against other vision language hierarchical / hyperbolic representation works. For example: existing hyperbolic vision language papers for general hierarchical classification, other CLIP based hierarchical forgery detection approaches. It is unclear how much gain comes from hyperbolic geometry itself versus the hierarchical prompt augmented CLIP paradigm. A controlled ablation comparing Euclidean space version equipped with exactly the same hierarchical prompt pipeline is desired.

W5: Inference complexity and computational overhead are partially under discussed The supplementary appendix reports Params / FLOPs / FPS for the full pipeline. However, the analysis does not separate overhead introduced by hyperbolic manifold operations (exponential / logarithmic map, Lorentz inner product, iterative Riemannian centroid update). We cannot tell how much extra compute is brought by moving from Euclidean to Lorentz space. Moreover, the inference pipeline includes multi prompt ensemble, Riemannian centroid computation during training; the paper does not discuss latency when scaling to large scale streaming video inference.

W6: Limitation of predefined taxonomy is not systematically quantified One key limitation acknowledged by authors: the framework relies on manually predefined forgery taxonomy and text prompt gallery. The paper does not conduct controlled experiments to quantify performance degradation when test time forgery types fall outside the manually designed hierarchy. For real world scenarios, new forgery families keep emerging; it would be valuable to simulate out of taxonomy setting and report detection AUC drop when removing certain hierarchy branches during prompt construction.

W7: User study for explanation generation has constrained comparison scope The user study only compares against SIDA 13B. It does not include other explainable deepfake detection baselines. Besides, HyperDetect’s explanations are retrieval template based rather than native generative outputs, while SIDA is end to end generative. The two systems have different model paradigms, creating slight unfairness in human evaluation. More discussion about this paradigm mismatch should be added in main text.

Minor Weaknesses 1.Several key theoretical intuitions and proofs are pushed to appendix; some high level geometric interpretation could be condensed into main text for better readability. 2.The paper focuses heavily on face centric deepfake datasets; generalization to non face AIGC image forgery could receive more discussion. 3.Evaluation metric FPR 95 is less reported; for safety critical detection tasks, FPR 95 is complementary to AUC and should be supplemented in key tables.

Questions for authors

1.Could you provide additional ablation showing performance curves sweeping over hyper parameters including curvature k, cone aperture ω_0, slack coefficients ξ_1,ξ_2 and domain margin δ? Are these hyper parameters transferable across different datasets? 2.Can you provide quantitative geometric statistics (radial norm distributions, angular separation statistics) on real trained embeddings, to verify whether practical model satisfies the theoretical manifold assumptions in Proposition 1 2? 3.Could you add adversarial robustness evaluation: apply white box PGD / Auto PGD adversarial perturbations targeting to fool real/fake binary detection, and report AUC degradation? 4.Please run a controlled baseline: an Euclidean space model with exactly identical hierarchical prompt driven pipeline (only replacing Lorentz hyperbolic module). How much performance gain can be purely attributed to hyperbolic geometry? 5.Could you isolate and report the extra computational overhead purely caused by hyperbolic manifold operations (exponential / log map, Riemannian centroid update), compared to its Euclidean counterpart? 6.Can you simulate "out of taxonomy" setting: remove certain forgery branches from the prompt taxonomy, then test on forgery types belonging to removed branches, to quantify performance drop? 7.In your user study, HyperDetect uses template retrieval while SIDA-13B is native generative model. Could you add more discussion on this paradigm difference, and consider adding another retrieval based explainable baseline for fairer comparison? 8.Could you supplement FPR 95 metric in your main result tables for safety oriented deepfake detection?

3 comments
AuthorsOpenReview ↗

Response to Reviewer ignG (Part 1 of 3: Q1 & Q2)

We sincerely thank Reviewer ignG for the encouraging assessment, recognition of our theoretical formulation, and insightful suggestions. Due to OpenReview's character limit, we divide our response into three parts: Part 1 (Q1–Q2), Part 2 (Q3–Q5), and Part 3 (Q6–Q8).

Q1: Hyperparameter Sensitivity Curves and Cross-Dataset Transferability

All main experiments use a single fixed hyperparameter configuration (κ=1.0,ω0=π/4,β=0.1,ξ1=0.05,ξ2=0.10,δ=1.5,τ=0.07\kappa=1.0, \omega_0=\pi/4, \beta=0.1, \xi_1=0.05, \xi_2=0.10, \delta=1.5, \tau=0.07) selected on the FF++ validation split without per-dataset tuning (achieving 96.3±0.296.3 \pm 0.2\\% on CDF-v2 and 84.7±0.384.7 \pm 0.3\\% on DFDC across 3 random seeds). Sweeping each hyperparameter individually confirms strong cross-dataset transferability:

HyperparameterTested ValuesCDF-v2 AUC (%)DFDC AUC (%)HyperparameterTested ValuesCDF-v2 AUC (%)DFDC AUC (%)
Curvature κ\kappa0.1 / 0.5 / 1.0 / 2.0 / 5.094.8 / 95.9 / 96.3 / 95.7 / 93.482.9 / 84.1 / 84.7 / 83.8 / 81.2Domain Margin δ\delta0.5 / 1.0 / 1.5 / 2.5 / 3.595.6 / 96.1 / 96.3 / 96.0 / 95.583.9 / 84.5 / 84.7 / 84.3 / 83.7
Aperture ω0\omega_0π/6\pi/6 / π/4\pi/4 / π/3\pi/3 / 5π/125\pi/1294.9 / 96.3 / 95.8 / 94.283.1 / 84.7 / 84.2 / 82.4Slack (ξ1,ξ2)(\xi_1, \xi_2)(0,0) / (0.05,0.10) / (0.1,0.2) / (0.15,0.3)94.1 / 96.3 / 95.7 / 94.581.8 / 84.7 / 84.0 / 82.6

Notice that setting (ξ1,ξ2)=(0,0)(\xi_1, \xi_2)=(0,0) (hard cone boundary without slack) causes a noticeable −2.9-2.9\\% drop on DFDC, directly supporting Proposition 1.


Q2: Quantitative Geometric Statistics of Real Trained Embeddings (Verifying Propositions 1 & 2)

We measured the Lorentz geodesic radius r=dL(0,z)r = d_{\mathcal{L}}(\mathbf{0}, z), cone inclusion rates, and pairwise tangent angles on the FF++ test set after convergence:

  1. Radial Norm Distribution (Proposition 2):
    • Geodesic Distance Baseline: Real radius =0.68±0.21= 0.68 \pm 0.21, Fake radius =0.84±0.26= 0.84 \pm 0.26 (radial collapse near the origin).
    • HyperDetect (Ours): Real samples remain near the origin at 1.42±0.311.42 \pm 0.31, the GENERIC-FAKE prototype sits at 2.352.35, and fine-grained forgery subtypes expand outward to 3.68±0.443.68 \pm 0.44 (DeepFakes: 3.713.71, Face2Face: 3.593.59, FaceSwap: 3.823.82, NeuralTextures: 3.613.61), confirming utilization of the high-capacity boundary.
  2. Cone Inclusion & Angular Separation (Proposition 1):
    • 94.6% of test fake samples satisfy α(z,cGF)≤ω(cGF)+ξ2\alpha(z, c_{\text{GF}}) \le \omega(c_{\text{GF}}) + \xi_2 (≈41.7∘\approx 41.7^\circ).
    • Child domain centroids maintain an average angle of 22.4∘±3.9∘22.4^\circ \pm 3.9^\circ to the GENERIC-FAKE parent while keeping a 53.1∘±4.8∘53.1^\circ \pm 4.8^\circ average pairwise separation among sibling domains.
AuthorsOpenReview ↗

Response to Reviewer ignG (Part 2 of 3: Q3, Q4 & Q5)

Response to Reviewer ignG (Part 2 of 3: Q3–Q5)

Q3: White-Box Adversarial Robustness Evaluation (PGD-10 / Auto-PGD)

We evaluated white-box untargeted L∞L_\infty adversarial attacks (ϵ=2/255\epsilon = 2/255, step size 0.5/2550.5/255) targeting the Real/Fake boundary on FF++ →\rightarrow CDF-v2 (without adversarial training):

MethodClean AUC (%)PGD-10 AUC (%)Auto-PGD AUC (%)
Fine-tuned CLIP (ViT-B/16)90.024.819.3
ForensicsAdapter (CVPR'25)90.039.534.1
HyperDetect (Ours)96.354.248.9

HyperDetect exhibits substantially stronger resistance (+14.7+14.7\\% over ForensicsAdapter under PGD-10) because fooling the detector requires rotating the feature vector across a curved tangent-space cone boundary anchored by multi-template semantic prototypes rather than crossing a linear hyperplane.


Q4: Controlled Euclidean Baseline with Identical Hierarchical Prompt Pipeline

To isolate the gain of hyperbolic geometry from hierarchical text prompts, we trained an Euclidean counterpart with the exact same ViT-B/16 backbone, identical 3-level prompt gallery, hyperspherical cosine contrastive loss, and hierarchical margin losses:

ConfigurationCDF-v1CDF-v2DFDCDFDCPDFDAvg. AUC (%)
Euclidean Flat Baseline (Table 4)88.591.279.587.094.188.1
Euclidean + Identical Hierarchical Prompts90.192.480.888.595.289.4 (+1.3)
HyperDetect (Lorentz Hyperbolic Geometry)93.796.384.792.197.892.9 (+4.8)

Hierarchical prompting alone contributes +1.3+1.3\\% in Euclidean space, whereas Lorentz hyperbolic modeling adds +3.5+3.5\\% net AUC gain, proving that negative curvature and radius-dependent entailment cones are the primary drivers of generalization.


Q5: Isolating Pure Hyperbolic Manifold Overhead

Benchmarking single-frame inference (224×224224 \times 224, batch size 1, NVIDIA RTX 4090) and 32-frame video batch streaming:

  • Euclidean Counterpart: 1.21 ms/frame (826.4 FPS), 1.06M trainable projection params, 2.37G FLOPs.
  • HyperDetect (Lorentz): 1.29 ms/frame (772.1 FPS), 1.06M trainable projection params, 2.38G FLOPs.
  • Pure Hyperbolic Overhead: Only 0.08 ms (+6.6%) per frame and <0.01G FLOPs. For 32-frame video clips, batch throughput reaches 1,840 FPS (only 4.2% slower than Euclidean). Iterative Riemannian centroid updates (Eq. 8) and multi-prompt text encoding are computed once offline and cached as static vectors at test time, ensuring zero scaling bottleneck for streaming video inference.
AuthorsOpenReview ↗

Response to Reviewer ignG (Part 3 of 3: Q6, Q7 & Q8)

Response to Reviewer ignG (Part 3 of 3: Q6–Q8)

Q6: Simulating "Out-of-Taxonomy" Forgery Generalization

We removed all Level-2 face-swapping prompts from T\mathcal{T} (retaining only Level-1 GENERIC-FAKE and expression-reenactment prompts) and evaluated on unseen face-swapping benchmarks:

  • Full Taxonomy: CDF-v2 AUC = 96.3%, SimSwap AUC = 94.2%.
  • Out-of-Taxonomy (Face-Swap branch removed): CDF-v2 AUC = 93.8% (-2.5%), SimSwap AUC = 91.5% (-2.7%).

The bounded ∼2.6\sim 2.6\\% drop confirms that the Level-1 GENERIC-FAKE cone serves as an effective semantic safety net for novel forgery branches outside the predefined taxonomy.


Q7 & Q8: User Study Paradigm Discussion & Supplementing FPR@95

  1. User Study Discussion: We have added a dedicated discussion in Section A.3 acknowledging the paradigm difference (template-based geometric retrieval vs. SIDA-13B's end-to-end generative decoding). Furthermore, against a matched Euclidean CLIP-RAG retrieval baseline on CDF-v2 (1,500 judgments), HyperDetect's hierarchical hyperbolic retrieval still achieved a 68.4% win rate due to patch-level artifact localization.
  2. Supplementing FPR@95 (at TPR = 95%, lower is better, FF++ trained):
MethodCDF-v2 FPR@95 (↓\downarrow)DFDC FPR@95 (↓\downarrow)DFDCP FPR@95 (↓\downarrow)Avg. FPR@95 (↓\downarrow)
ForensicsAdapter (CVPR'25)38.4%54.2%41.7%44.8%
Effort (ICML'25)24.6%51.8%36.9%37.8%
HyperDetect (Ours)19.2%47.5%31.4%32.7%
Reviewer_VokmOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies generalizable deepfake detection by modeling hierarchical relationships among different forgery types. The authors argue that existing methods often treat forgery types as independent categories, overlooking shared characteristics that could improve generalization to unseen manipulations. To address this limitation, they propose HyperDetect, a CLIP-based framework that represents forgery concepts in hyperbolic Lorentz space.

HyperDetect combines three objectives: Angular Contrastive Learning to distinguish forgery types, Hierarchical Entailment Loss to preserve parent-child relationships, and Domain Dispersion Regularization to separate forgery domains. The framework uses text prompts as semantic anchors for hierarchical representation learning. Experiments on cross-dataset and cross-manipulation benchmarks report improvements over existing deepfake detectors, with ablation studies evaluating the proposed components.

Critical strengths
  1. The paper provides an interesting perspective on learning hierarchical relationships among forgery types.

    Rather than treating different manipulation methods as independent categories, HyperDetect models their relationships through a hierarchy of real, generic fake, and specific forgery types. The proposed framework combines angular contrastive learning, hierarchical entailment, and domain dispersion to capture both shared and method-specific forgery characteristics. This offers a meaningful direction for improving generalizable deepfake detection.


  1. The paper reports promising generalization performance with extensive experimental comparisons.

    HyperDetect achieves competitive results across several cross-dataset and cross-manipulation benchmarks, outperforming recent methods on most evaluated settings. The ablation studies in Tables 4, 7, and 8 also examine different loss components, geometric formulations, and hierarchy designs, providing useful evidence for the effectiveness of the proposed framework.

Critical weaknesses
  1. The text-guided hierarchical learning strategy is insufficiently analyzed and specified.

    The initial discriminability of text anchors in the CLIP embedding space is not examined. The proposed text prompts may not provide sufficient separation between forgery concepts, potentially affecting the stability of contrastive learning. A similar issue was observed in prior work [0], which reported high cosine similarity not only among different forgery types but also between real and fake categories. Although the taxonomy differs, this issue may also arise in HyperDetect and potentially hinder the learning of distinct hierarchical representations. However, the paper does not examine this possibility or explain how the proposed framework addresses it.

    The contrastive pair construction in Eq. (6) is insufficiently specified. Although the paper describes same-concept and ancestor-related pairs as positives and sibling concepts as negatives, the positive pair set P\mathcal P and candidate set B\mathcal B are not explicitly defined. It is also unclear how image and text embeddings are paired and how positive and negative samples are selected during training. These missing details make it difficult to understand and reproduce a central component of the proposed method.


  1. The training details and generalization results require further clarification.

    Several important training details are unclear. The paper does not clearly explain which CLIP components are optimized, particularly whether the image encoder is fully fine-tuned or kept frozen. Furthermore, the settings of key hyperparameters, including curvature, temperature, cone aperture, and margin parameters, are not provided. Although Figure 5(b) examines the sensitivity to loss weights, the effects of these hyperparameters remain unexplored.

    The unusually strong generalization results require further analysis. Appendix Tables 10–12 report nearly perfect performance on several GAN- and diffusion-generated image datasets, including multiple cases of 100% accuracy. Given that the model is trained on facial manipulations from FF++, a more detailed analysis is needed to understand how it achieves such strong performance on general AI-generated images, including non-facial content.


REFERENCES
[0] Jung, Inho, et al. "A rich knowledge space for scalable deepfake detection." International Conference on Learning Representations. Vol. 2026. 2026.
[1] Grommelt, Patrick, et al. "Fake or jpeg? revealing common biases in generated image detection datasets." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024.
[2] Guillaro, Fabrizio, et al. "A bias-free training paradigm for more general ai-generated image detection." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.
[3] Chen, Ruoxin, et al. "Dual data alignment makes ai-generated image detector easier generalizable." arXiv preprint arXiv:2505.14359 (2025).

Questions for authors
  1. Initial text-anchor relationships: Could the authors provide an analysis of the pairwise cosine similarities among the initial text anchors in the CLIP embedding space? How does the proposed framework handle potentially high similarities between different forgery concepts during training?

  1. Contrastive pair construction: How are the positive pair set P\mathcal P and candidate set B\mathcal B in Eq. (6) constructed? Specifically, are image-image pairs, image-text pairs, or both used, and how are ancestor-related positives and sibling negatives selected?

  1. Training settings: Which components are trainable, including the CLIP encoders, text prototypes, and projection layers? Could the authors also provide the settings of the key hyperparameters (curvature, temperature, cone aperture, and margins) and clarify how sensitive the performance is to these choices?

  1. Generalization to non-facial images: Were the results in Appendix Tables 10–12 obtained using only the FF++-trained model, without additional training or calibration? How are non-facial images processed, and what explains the exceptionally strong performance on these datasets?

  1. Numerical inconsistency:
    The average performance reported in Appendix Tables 12 and 15 appears inconsistent, despite using the same per-dataset results. Could the authors double-check the average calculations and correct any errors?

  1. Dataset bias in generalization evaluation:
    Prior works [1–3] have highlighted various dataset biases in AI-generated image detection, including JPEG compression and image-size biases in GenImage [1]. Given the exceptionally high accuracies reported in Appendix Tables 10–12, could the authors evaluate whether the performance remains consistent when JPEG compression and resizing biases are controlled? Such an analysis could help better understand how your approach, primarily developed for facial deepfake detection, achieves strong generalization performance on broader AI-generated image benchmarks.
5 comments
AuthorsOpenReview ↗

Response to Reviewer Vokm (Part 1 of 4: Q1)

We thank Reviewer Vokm for the thorough review, constructive feedback, and careful verification of our tables. Due to OpenReview's character limit, we divide our response into four parts: Part 1 (Q1), Part 2 (Q2), Part 3 (Q3–Q4), and Part 4 (Q5–Q6).

Q1: Initial Text-Anchor Similarities and Separation Dynamics

We measured the pairwise cosine similarities of the initial frozen CLIP (ViT-B/16) text anchors before training. Because all prompts share common facial nouns (e.g., "face", "portrait"), initial embeddings indeed exhibit high cosine similarities (consistent with prior VLM forensics work [0]):

  • cos⁡(tReal,tGeneric-Fake)=0.842\cos(t_{\text{Real}}, t_{\text{Generic-Fake}}) = 0.842
  • cos⁡(tGeneric-Fake,tDeepFakes)=0.891\cos(t_{\text{Generic-Fake}}, t_{\text{DeepFakes}}) = 0.891, cos⁡(tGeneric-Fake,tFaceSwap)=0.876\cos(t_{\text{Generic-Fake}}, t_{\text{FaceSwap}}) = 0.876
  • cos⁡(tDeepFakes,tFaceSwap)=0.914\cos(t_{\text{DeepFakes}}, t_{\text{FaceSwap}}) = 0.914, cos⁡(tFace2Face,tNeuralTextures)=0.883\cos(t_{\text{Face2Face}}, t_{\text{NeuralTextures}}) = 0.883

How HyperDetect resolves this: HyperDetect projects visual and textual embeddings onto the Lorentz hyperboloid via exp⁡0κ(⋅)\exp_{\mathbf{0}}^\kappa(\cdot) and jointly optimizes a learnable hyperbolic projection head alongside the visual encoder. Under LhFC\mathcal{L}_{hFC} (angular sibling repulsion) and LhFE\mathcal{L}_{hFE} (parent-child cone entailment with effective parent half-aperture ω(cGF)+ξ2≈41.7∘\omega(c_{\text{GF}}) + \xi_2 \approx 41.7^\circ), the representations converge to a geometrically consistent hierarchy on FF++:

Concept Pair (u,v)(u, v)Initial CLIP CosineTrained Tangent Angle α(u,v)\alpha(u,v)Trained Tangent Cosine cos⁡α(u,v)\cos\alpha(u,v)Geometric Role
(Real, Generic-Fake)0.842106.4∘106.4^\circ-0.282Obtuse separation (outside Real cone)
(Generic-Fake, DeepFakes)0.89121.3∘21.3^\circ0.932Entailed inside parent cone (≤41.7∘\le 41.7^\circ)
(Generic-Fake, FaceSwap)0.87623.8∘23.8^\circ0.915Entailed inside parent cone (≤41.7∘\le 41.7^\circ)
(DeepFakes, FaceSwap)0.91451.4∘51.4^\circ0.624Angularly separated sibling domains
(Face2Face, NeuralTextures)0.88354.8∘54.8^\circ0.576Angularly separated sibling domains
AuthorsOpenReview ↗

Response to Reviewer Vokm (Part 2 of 4: Q2)

Response to Reviewer Vokm (Part 2 of 4: Q2)

Q2: Explicit Definition of Contrastive Pair Set P\mathcal{P} and Candidate Set B\mathcal{B} in Eq. (6)

We will include the exact mathematical formulation in Section 2.2 of the revision. In each mini-batch of size BB, we have image embeddings zi=g(Ii)i=1B\\{z_i = g(I_i)\\}_{i=1}^B and hierarchy text prototypes A=cReal,cGF∪cdd∈S\mathcal{A} = \\{c_{\text{Real}}, c_{\text{GF}}\\} \cup \\{c_d\\}_{d \in \mathcal{S}}.

  • Candidate Set B(u)\mathcal{B}(u): For an anchor image embedding u=ziu = z_i, B(u)=(zjj≠i)∪A\mathcal{B}(u) = (\\{z_j\\}_{j \neq i}) \cup \mathcal{A}, combining both image-image and image-text pairs.
  • Positive Set P(u)\mathcal{P}(u): Defined via the taxonomy partial order ≺\prec:
    1. If IiI_i is Real, P(u)\mathcal{P}(u) contains all other Real images in the batch plus the Level-0 prototype cRealc_{\text{Real}}.
    2. If IiI_i belongs to forgery domain d∈Sd \in \mathcal{S}, P(u)\mathcal{P}(u) contains same-concept positives (other images from domain dd and the Level-2 prototype cdc_d) and ancestor-related positives (the Level-1 prototype cGFc_{\text{GF}}, weighted by 0.50.5 to balance gradient scale).
  • Negatives (B(u)∖P(u)\mathcal{B}(u) \setminus \mathcal{P}(u)): For a fake sample of domain dd, negatives include all Real instances/cRealc_{\text{Real}} and all sibling forgery instances/prototypes (d′∈S,d′≠dd' \in \mathcal{S}, d' \neq d).
AuthorsOpenReview ↗

Response to Reviewer Vokm (Part 3 of 4: Q3 & Q4)

Response to Reviewer Vokm (Part 3 of 4: Q3–Q4)

Q3: Trainable Components, Hyperparameter Settings, and Sensitivity

  • Trainable Components: The CLIP text encoder is frozen to preserve linguistic semantic priors. We fine-tune the CLIP visual backbone (ViT-B/16, lr =3×10−6= 3 \times 10^{-6}) together with a 2-layer MLP hyperbolic projection head (512→512512 \rightarrow 512, lr =3×10−5= 3 \times 10^{-5}).
  • Hyperparameter Configuration: Across all experiments, we fix curvature κ=1.0\kappa = 1.0, temperature τ=0.07\tau = 0.07, initial cone aperture ω0=π/4\omega_0 = \pi/4, decay coefficient β=0.1\beta = 0.1, slack margins ξ1=0.05,ξ2=0.10\xi_1 = 0.05, \xi_2 = 0.10, and domain dispersion margin δ=1.5\delta = 1.5.
  • Sensitivity & Multi-Seed Stability (FF++ →\rightarrow CDF-v2 / DFDC Frame-level AUC %): Over 3 independent seeds, our default configuration achieves 96.3±0.296.3 \pm 0.2\\% on CDF-v2 and 84.7±0.384.7 \pm 0.3\\% on DFDC. Varying individual hyperparameters confirms smooth stability:
HyperparameterTested ValuesCDF-v2 AUC (%)DFDC AUC (%)HyperparameterTested ValuesCDF-v2 AUC (%)DFDC AUC (%)
Curvature κ\kappa0.1 / 0.5 / 1.0 / 2.0 / 5.094.8 / 95.9 / 96.3 / 95.7 / 93.482.9 / 84.1 / 84.7 / 83.8 / 81.2Domain Margin δ\delta0.5 / 1.0 / 1.5 / 2.5 / 3.595.6 / 96.1 / 96.3 / 96.0 / 95.583.9 / 84.5 / 84.7 / 84.3 / 83.7
Aperture ω0\omega_0π/6\pi/6 / π/4\pi/4 / π/3\pi/3 / 5π/125\pi/1294.9 / 96.3 / 95.8 / 94.283.1 / 84.7 / 84.2 / 82.4Slack (ξ1,ξ2)(\xi_1, \xi_2)(0,0) / (0.05,0.10) / (0.1,0.2) / (0.15,0.3)94.1 / 96.3 / 95.7 / 94.581.8 / 84.7 / 84.0 / 82.6

Q4: Evaluation Protocols for Non-Facial AIGC Benchmarks (Appendix Tables 10–12)

Thank you for highlighting this needed clarification. Unlike the face-centric deepfake benchmarks (Tables 1–6), general AIGC evaluation in Appendix B follows standard universal synthetic image detection protocols:

  1. Full-Image Input (No RetinaFace): RetinaFace cropping is disabled; entire images are resized/padded to 224×224224 \times 224.
  2. Standard Protocol Alignment: Following CNNDetection (Wang et al., CVPR'20) for Table 10 and GenImage (Zhu et al., NeurIPS'23) / FatFormer (Liu et al., CVPR'24) for Tables 11–12, the model is trained on the respective standard AIGC training splits (ProGAN 20-class for Table 10; SD v1.4 / ProGAN 4-class for Tables 11–12) using general semantic prompts ("a real photograph" vs. "an AI-generated image" / "a GAN-synthesized image" / "a diffusion-generated image"). This also explains the expected difference on BigGAN between Table 10 (99.6%, evaluated under the 20-class ProGAN protocol) and Table 12 (84.9%, evaluated under the cross-resolution GenImage protocol, where baselines like FatFormer score 55.8%). We will state this protocol distinction explicitly in Appendix B.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.