- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes CAEIR, a cross-architecture mixture-of-experts framework for generalizable deepfake detection. Unlike existing MoE-based forgery detectors that build experts within one single architectural family, this work constructs a competitive expert pool containing CNN, Transformer, and two Mamba-based state-space experts with different scan directions. Instead of purely routing from input representations, the router leverages response evidence computed from all heterogeneous experts. A lightweight cross-expert interaction module generates routing corrections to calibrate CLS-based routing logits, followed by Top-1 sparse residual injection. The backbone is ViT-S/16, whose last 9 blocks are augmented with CAEIR branches. Besides, a Detector-to-LLM interface encodes prediction confidence and routing traces as evidence tokens for LLM to produce human‑readable explanations.
Critical strengths
(1)Prior MoE deepfake detectors mostly build experts within one architectural family. This paper observes CNN, Transformer and Mamba state-space models own distinct inductive biases for heterogeneous forensic artifacts. It further argues that routing should condition on actual expert responses instead of only input features, which is a reasonable observation for forgery detection. (2)The authors systematically compare inputconditioned routing, Top-k dispatch, soft mixture, within-family expert pools, MoE insertion depth and hyper-parameter sensitivity. Results verify that performance gain comes largely from cross-expert interaction calibration rather than merely evaluating multiple experts. Post-hoc analysis on HydraFake in appendix offers insightful observations on expert preference for different manipulation types. The Detector-to-LLM module provides a qualitative interpretability prototype. (3)Models are trained only on FF++ real videos with online synthetic fakes, evaluated across four widely-used cross-domain benchmarks. Both framelevel and video-level AUC are reported. Comprehensive comparison against recent SOTA methods is provided.
Critical weaknesses
1.The method is output-sparse yet computationally dense, incurring substantial efficiency overhead. The paper repeatedly emphasizes that the proposed method is output-sparse (only one selected expert’s output is injected into the residual branch). However, to obtain responses from all experts for the interaction module, all four experts must be fully executed in the forward pass. As shown in Table 3: running only the selected expert yields 31.14 GFLOPs with latency of 10.46 ms; the full CAEIR execution with all four experts consumes 44.42 GFLOPs and 23.74 ms, nearly doubling the latency. The core goal of conventional sparse MoE in the MoE community is to reduce computational cost. This work is merely output-sparse, since all experts must be evaluated during inference, resulting in high computational overhead. The authors should candidly highlight this limitation in the main text instead of only mentioning it in the discussion section. 2.Section 4.1 explicitly states that Table 1 provides contextual comparisons rather than comparisons under strictly identical protocols. Many competing methods adopt different training data, augmentation pipelines and video aggregation strategies. We recommend the authors reproduce 2–3 representative MoE baselines (DFMoE, MoE-FFD) under exactly the same training/testing protocol, with matched training data, augmentations and video aggregation, and place these results in the main table. This would strongly verify CAEIR’s advantages, instead of only citing reported numbers from prior papers. The current table only demonstrates competitive metrics within published literature and cannot fully rule out performance differences introduced by protocol mismatches. 3.The Detector-to-LLM module is presented as one contribution of this paper, yet only qualitative case figures are provided without any quantitative metrics. 4.The HydraFake analysis in the appendix is offline post-hoc analysis and imposes no constraints during training. The observed expert preference across different forgery types is an emergent phenomenon rather than an explicit design objective of the model. This distinction should be clearly stated to avoid misleading readers into believing the model explicitly assigns experts conditioned on forgery categories. 5.For video-level evaluation: “the average of the top 20% fake probabilities within a video is taken as the video score”. This scoring strategy is custom-designed for this work, and its potential to introduce unfairness in comparisons needs to be discussed.
Questions for authors
1.The interaction module uses a simple 4-head self-attention layer to model interactions over four expert evidence nodes. Why is self-attention the most suitable choice here? Why not adopt alternative interaction modules? 2.The “All candidates, no interaction” setting already requires forward computation for all experts but only achieves marginal performance gains. Beyond generating routing correction logits, does the interaction module have other indirect effects? Please explain the underlying mechanism behind this ablation result. 3.Could you reproduce MoE-FFD or DFMoE under the identical experimental protocol and include these comparisons in the main result table? 4.Do you plan to add quantitative or human evaluation for the Detector-to-LLM module? If quantitative evaluation will not be provided, we suggest revising the abstract and contribution list to reduce the prominence of this component. 5.To address the high inference cost, beyond future work, can you provide preliminary exploratory experiments, such as an evidence predictor that approximates evidence nodes without fully running all experts, and report the resulting performance degradation? 6.Since the HydraFake analysis is offline and post-hoc, have you observed routing shifts over the training process? Are there noticeable differences in expert selection between early and late layers? 7.Please elaborate on the impacts of the top-20% video scoring strategy.
7 comments
Response to Reviewer VyGt [Part 2/7]
Q2. What the interaction module contributes beyond candidate computation
This question distinguishes the cost of computing all candidate responses from the benefit of using them. We added controlled execution checks and separate component ablations to clarify why evaluating all candidates alone provides little benefit and what the interaction module contributes beyond routing-logit correction.
Q2.1. Why evaluating all candidates alone provides little benefit
In the “All candidates, no interaction” setting, all four experts are evaluated, but routing remains input-conditioned and only the selected expert’s response is injected. The unselected responses are neither used to correct routing nor fused into the residual update. Both input-conditioned Top-1 (without response correction or modulation) and this all-candidates/no-interaction control retain the selected expert’s gating probability and disable the additional response modulation.
In the implementations examined here, both input-conditioned Top-1 and the all-candidates/no-interaction control evaluate all experts with Top-1 masking during training. Their inference paths differ in whether unselected experts are executed. To isolate this execution difference, we compared the two inference paths using identical weights and inputs in evaluation mode for each of three seeds. They selected the same experts on the tested inputs, with maximum absolute differences of 1.19 × 10⁻⁶ in logits and 3.58 × 10⁻⁷ in probabilities across these checks.
These results support the explanation that merely computing unused candidate responses does not materially change the prediction. Accordingly, we do not interpret the historical single-run difference of 94.82% versus 94.88% reported in Appendix C, Table 11 as evidence of a reliable benefit from all-candidate computation itself. The revised manuscript clarifies this distinction in Section 4.4 and Appendix C; the component comparisons are reported in Section 4.5, Table 5.
Q2.2. What interaction contributes beyond routing-logit correction
CAEIR makes the candidate responses available to two mechanisms:
-
Response-conditioned routing correction: Interaction features generate expert-specific corrections to the input-conditioned logits, affecting both the Top-1 choice and its gating probability.
-
Response modulation: After selection, a separate head uses the interaction features to generate a bounded multiplier for the selected response. This changes the strength of the residual contribution without changing the expert selected at that layer.
Consequently, unselected experts can contribute evidence to the routing and modulation decisions even though their response tensors are not directly injected into the backbone. The all-candidates/no-interaction control lacks these pathways.
We separated the two mechanisms using the same three seeds (3407–3409). The input-conditioned Top-1 configuration disables both mechanisms; cross-expert self-attention with response modulation disabled enables routing correction only; input-conditioned routing with response modulation enables modulation only; and CAEIR enables both. The table reports video-level AUC (%) as mean ± sample standard deviation.
| Configuration | CDF | DFDC | FFIW |
|---|---|---|---|
| Input-conditioned Top-1 (α = 0, β = 0) | 95.27 ± 0.44 | 79.38 ± 1.25 | 89.14 ± 1.59 |
| Cross-expert self-attention; correction only (α = 0.1, β = 0) | 95.96 ± 0.30 | 81.01 ± 0.26 | 90.38 ± 0.99 |
| Input-conditioned routing with response modulation (α = 0, β = 0.075) | 95.08 ± 0.52 | 80.47 ± 0.80 | 89.90 ± 1.15 |
| CAEIR (α = 0.1, β = 0.075) | 96.26 ± 0.38 | 81.66 ± 0.49 | 90.46 ± 1.00 |
Adding cross-expert routing correction to input-conditioned Top-1 improves the mean AUC by 0.69, 1.63, and 1.24 percentage points on CDF, DFDC, and FFIW, respectively. Adding response modulation to the corrected routing (CAEIR versus correction-only) yields further mean improvements of 0.30, 0.65, and 0.08 points. Response modulation with input-conditioned routing improves DFDC and FFIW but slightly decreases CDF, indicating that its benefit is dataset-dependent and that it does not independently explain CAEIR's gains.
These ablations support two contributions of the interaction features: informing expert selection and gating, and modulating the selected residual response. They do not establish a separate causal claim about indirect effects on expert optimization during training. The component ablations are reported in Section 4.5, Table 5; execution-path checks and diagnostic limitations are in Appendix C.
Response to Reviewer VyGt [Part 3/7]
Q3. MoE baseline comparison under shared protocol settings
We appreciate the request for a controlled external comparison. We reproduced MoE-FFD, one of the two methods named in the question, matching our training source, augmentations, face-sampling procedure, test manifests, and video-aggregation protocol. As noted in Section 4.1, MoE-FFD retains its native 224 × 224 input resolution, while CAEIR uses 336 × 336, so the input resolutions are not identical. Official code for DFMoE was not publicly available to us, so we could not establish a faithful reproduction of that method for this response. We report the MoE-FFD reproduction in the main cross-dataset comparison and distinguish it from the score reported in the original MoE-FFD paper (Section 4.2, Table 2).
We also evaluated Plain ViT-S/16 and a FLOPs-matched Dense ViT under the same study protocol. These controls assess the contribution beyond the backbone and whether a conventional dense Transformer at a comparable FLOPs budget obtains similar accuracy. They complement the external MoE comparison; they are not additional MoE reproductions.
The table reports video-level AUC (%) as mean ± sample standard deviation over three independently trained seeds (3407, 3408, and 3409).
| Method | CDF | DFD | DFDC | FFIW |
|---|---|---|---|---|
| MoE-FFD (reproduced) | 91.75 ± 0.67 | 90.65 ± 0.57 | 77.22 ± 1.12 | 86.83 ± 0.75 |
| Plain ViT-S/16 | 94.97 ± 0.37 | 93.08 ± 0.85 | 79.48 ± 0.91 | 87.98 ± 1.07 |
| FLOPs-matched Dense ViT | 94.96 ± 0.21 | 92.37 ± 0.32 | 78.81 ± 0.43 | 87.61 ± 2.27 |
| CAEIR | 96.26 ± 0.38 | 95.22 ± 0.52 | 81.66 ± 0.49 | 90.46 ± 1.00 |
CAEIR exceeds the reproduced MoE-FFD by 4.51, 4.57, 4.44, and 3.63 AUC percentage points on CDF, DFD, DFDC, and FFIW, respectively. It also exceeds Plain ViT-S/16 by 1.29, 2.14, 2.18, and 2.48 points, and the FLOPs-matched Dense ViT by 1.30, 2.85, 2.85, and 2.85 points. These are descriptive differences across the evaluated seeds; we do not infer statistical significance from three runs.
Response to Reviewer VyGt [Part 1/7]
We appreciate the request for evidence supporting this design choice. We evaluated independent response-scoring MLPs and a parameter-matched MLP over concatenated expert responses to compare self-attention with alternative interaction modules.
Self-attention provides a natural way to construct content-dependent relations among the four evidence nodes: each expert’s evidence representation can incorporate information from the other candidates before producing its routing correction. This motivates its use, but does not establish that it is uniquely suitable or universally optimal.
To evaluate this choice, we compared three routing-correction implementations:
Independent-response MLP: Scores each expert’s pooled response independently, without cross-expert interaction.
Concatenated-response MLP: Jointly processes all four pooled responses and therefore also permits cross-expert interaction.
Cross-expert self-attention: Applies four-head self-attention to the four response nodes to generate routing corrections.
All three use the same pooled expert-response inputs, expert pool, and final Top-1 rule. Their corrections are added to the original CLS-based routing logits with α = 0.1. Response modulation is disabled (β = 0) to isolate the routing-correction mechanism. Training data, training budget, checkpoint-selection rule, and evaluation settings are held fixed.
For the direct comparison between the concatenated-response MLP and self-attention, the MLP uses a hidden width of 481, giving 741,225 parameters per routing-correction branch, compared with 741,313 for attention—a difference of approximately 0.012%.
The following table reports video-level AUC (%) as mean ± sample standard deviation over seeds 3407, 3408, and 3409.
Self-attention achieved higher mean AUC than the parameter-matched concatenated-response MLP on all three datasets, with gains of 0.78, 0.55, and 0.16 percentage points on CDF, DFDC, and FFIW, respectively. The gains were larger on CDF and DFDC; on FFIW, the small difference indicates that the MLP remained competitive.
These comparisons are reported in Section 4.5, Table 4. We therefore describe self-attention as an empirically supported implementation of response-conditioned interaction, rather than an indispensable component. These experiments evaluate the four-head attention module as a whole; they do not establish that four heads are optimal or that attention is superior to every possible interaction operator.