acceptodds
Under review as a conference paper at ICLR 2027

CAEIR: Cross-Architecture Experts with Interaction-aware Routing for Deepfake Detection

Abstract

Generalizable deepfake detection requires identifying weak and heterogeneous forensic cues that vary across manipulation methods and data distributions. Existing mixture-of-experts detectors, however, typically specialize experts within a single architectural family, while expert-selection decisions are often based solely on the input representation, without explicitly comparing the forensic evidence produced by heterogeneous experts. Our key insight is that routing should depend not only on the input, but also on how heterogeneous experts respond to each sample. We introduce CAEIR, a cross-architecture framework that places convolutional, Transformer, and state-space experts in a unified competitive pool, summarizes their responses as evidence nodes, and models interactions among these nodes to produce expert-specific routing corrections that refine an input-conditioned Top-1 ranking. A separate bounded modulator scales the selected response without altering the routing decision. All candidate responses are computed to support interaction-aware expert selection, while only the selected response is injected into the detection residual. Trained on FF++ real videos with online synthetic fake supervision, CAEIR achieves video-level AUCs of 96.61%, 95.71%, 82.19%, and 89.76% on Celeb-DFv2, DFD, DFDC, and FFIW, respectively. Controlled ablations further support the benefits of cross-architecture diversity and response-conditioned routing beyond full-candidate evaluation alone.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Deepfake Detection, heterogeneous expert selection, interaction-aware routing

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_VyGtOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CAEIR, a cross-architecture mixture-of-experts framework for generalizable deepfake detection. Unlike existing MoE-based forgery detectors that build experts within one single architectural family, this work constructs a competitive expert pool containing CNN, Transformer, and two Mamba-based state-space experts with different scan directions. Instead of purely routing from input representations, the router leverages response evidence computed from all heterogeneous experts. A lightweight cross-expert interaction module generates routing corrections to calibrate CLS-based routing logits, followed by Top-1 sparse residual injection. The backbone is ViT-S/16, whose last 9 blocks are augmented with CAEIR branches. Besides, a Detector-to-LLM interface encodes prediction confidence and routing traces as evidence tokens for LLM to produce human‑readable explanations.

Critical strengths

(1)Prior MoE deepfake detectors mostly build experts within one architectural family. This paper observes CNN, Transformer and Mamba state-space models own distinct inductive biases for heterogeneous forensic artifacts. It further argues that routing should condition on actual expert responses instead of only input features, which is a reasonable observation for forgery detection. (2)The authors systematically compare inputconditioned routing, Top-k dispatch, soft mixture, within-family expert pools, MoE insertion depth and hyper-parameter sensitivity. Results verify that performance gain comes largely from cross-expert interaction calibration rather than merely evaluating multiple experts. Post-hoc analysis on HydraFake in appendix offers insightful observations on expert preference for different manipulation types. The Detector-to-LLM module provides a qualitative interpretability prototype. (3)Models are trained only on FF++ real videos with online synthetic fakes, evaluated across four widely-used cross-domain benchmarks. Both framelevel and video-level AUC are reported. Comprehensive comparison against recent SOTA methods is provided.

Critical weaknesses

1.The method is output-sparse yet computationally dense, incurring substantial efficiency overhead. The paper repeatedly emphasizes that the proposed method is output-sparse (only one selected expert’s output is injected into the residual branch). However, to obtain responses from all experts for the interaction module, all four experts must be fully executed in the forward pass. As shown in Table 3: running only the selected expert yields 31.14 GFLOPs with latency of 10.46 ms; the full CAEIR execution with all four experts consumes 44.42 GFLOPs and 23.74 ms, nearly doubling the latency. The core goal of conventional sparse MoE in the MoE community is to reduce computational cost. This work is merely output-sparse, since all experts must be evaluated during inference, resulting in high computational overhead. The authors should candidly highlight this limitation in the main text instead of only mentioning it in the discussion section. 2.Section 4.1 explicitly states that Table 1 provides contextual comparisons rather than comparisons under strictly identical protocols. Many competing methods adopt different training data, augmentation pipelines and video aggregation strategies. We recommend the authors reproduce 2–3 representative MoE baselines (DFMoE, MoE-FFD) under exactly the same training/testing protocol, with matched training data, augmentations and video aggregation, and place these results in the main table. This would strongly verify CAEIR’s advantages, instead of only citing reported numbers from prior papers. The current table only demonstrates competitive metrics within published literature and cannot fully rule out performance differences introduced by protocol mismatches. 3.The Detector-to-LLM module is presented as one contribution of this paper, yet only qualitative case figures are provided without any quantitative metrics. 4.The HydraFake analysis in the appendix is offline post-hoc analysis and imposes no constraints during training. The observed expert preference across different forgery types is an emergent phenomenon rather than an explicit design objective of the model. This distinction should be clearly stated to avoid misleading readers into believing the model explicitly assigns experts conditioned on forgery categories. 5.For video-level evaluation: “the average of the top 20% fake probabilities within a video is taken as the video score”. This scoring strategy is custom-designed for this work, and its potential to introduce unfairness in comparisons needs to be discussed.

Questions for authors

1.The interaction module uses a simple 4-head self-attention layer to model interactions over four expert evidence nodes. Why is self-attention the most suitable choice here? Why not adopt alternative interaction modules? 2.The “All candidates, no interaction” setting already requires forward computation for all experts but only achieves marginal performance gains. Beyond generating routing correction logits, does the interaction module have other indirect effects? Please explain the underlying mechanism behind this ablation result. 3.Could you reproduce MoE-FFD or DFMoE under the identical experimental protocol and include these comparisons in the main result table? 4.Do you plan to add quantitative or human evaluation for the Detector-to-LLM module? If quantitative evaluation will not be provided, we suggest revising the abstract and contribution list to reduce the prominence of this component. 5.To address the high inference cost, beyond future work, can you provide preliminary exploratory experiments, such as an evidence predictor that approximates evidence nodes without fully running all experts, and report the resulting performance degradation? 6.Since the HydraFake analysis is offline and post-hoc, have you observed routing shifts over the training process? Are there noticeable differences in expert selection between early and late layers? 7.Please elaborate on the impacts of the top-20% video scoring strategy.

7 comments
AuthorsOpenReview ↗

Response to Reviewer VyGt [Part 1/7]

Q1. Rationale for the self-attention interaction module

We appreciate the request for evidence supporting this design choice. We evaluated independent response-scoring MLPs and a parameter-matched MLP over concatenated expert responses to compare self-attention with alternative interaction modules.

Self-attention provides a natural way to construct content-dependent relations among the four evidence nodes: each expert’s evidence representation can incorporate information from the other candidates before producing its routing correction. This motivates its use, but does not establish that it is uniquely suitable or universally optimal.

To evaluate this choice, we compared three routing-correction implementations:

  • Independent-response MLP: Scores each expert’s pooled response independently, without cross-expert interaction.

  • Concatenated-response MLP: Jointly processes all four pooled responses and therefore also permits cross-expert interaction.

  • Cross-expert self-attention: Applies four-head self-attention to the four response nodes to generate routing corrections.

All three use the same pooled expert-response inputs, expert pool, and final Top-1 rule. Their corrections are added to the original CLS-based routing logits with α = 0.1. Response modulation is disabled (β = 0) to isolate the routing-correction mechanism. Training data, training budget, checkpoint-selection rule, and evaluation settings are held fixed.

For the direct comparison between the concatenated-response MLP and self-attention, the MLP uses a hidden width of 481, giving 741,225 parameters per routing-correction branch, compared with 741,313 for attention—a difference of approximately 0.012%.

The following table reports video-level AUC (%) as mean ± sample standard deviation over seeds 3407, 3408, and 3409.

Routing correctionCDFDFDCFFIW
Independent-response MLP94.86 ± 0.6680.00 ± 0.2688.34 ± 2.39
Concatenated-response MLP95.18 ± 0.3980.46 ± 0.4390.22 ± 1.40
Cross-expert self-attention95.96 ± 0.3081.01 ± 0.2690.38 ± 0.99

Self-attention achieved higher mean AUC than the parameter-matched concatenated-response MLP on all three datasets, with gains of 0.78, 0.55, and 0.16 percentage points on CDF, DFDC, and FFIW, respectively. The gains were larger on CDF and DFDC; on FFIW, the small difference indicates that the MLP remained competitive.

These comparisons are reported in Section 4.5, Table 4. We therefore describe self-attention as an empirically supported implementation of response-conditioned interaction, rather than an indispensable component. These experiments evaluate the four-head attention module as a whole; they do not establish that four heads are optimal or that attention is superior to every possible interaction operator.

AuthorsOpenReview ↗

Response to Reviewer VyGt [Part 2/7]

Q2. What the interaction module contributes beyond candidate computation

This question distinguishes the cost of computing all candidate responses from the benefit of using them. We added controlled execution checks and separate component ablations to clarify why evaluating all candidates alone provides little benefit and what the interaction module contributes beyond routing-logit correction.

Q2.1. Why evaluating all candidates alone provides little benefit

In the “All candidates, no interaction” setting, all four experts are evaluated, but routing remains input-conditioned and only the selected expert’s response is injected. The unselected responses are neither used to correct routing nor fused into the residual update. Both input-conditioned Top-1 (without response correction or modulation) and this all-candidates/no-interaction control retain the selected expert’s gating probability and disable the additional response modulation.

In the implementations examined here, both input-conditioned Top-1 and the all-candidates/no-interaction control evaluate all experts with Top-1 masking during training. Their inference paths differ in whether unselected experts are executed. To isolate this execution difference, we compared the two inference paths using identical weights and inputs in evaluation mode for each of three seeds. They selected the same experts on the tested inputs, with maximum absolute differences of 1.19 × 10⁻⁶ in logits and 3.58 × 10⁻⁷ in probabilities across these checks.

These results support the explanation that merely computing unused candidate responses does not materially change the prediction. Accordingly, we do not interpret the historical single-run difference of 94.82% versus 94.88% reported in Appendix C, Table 11 as evidence of a reliable benefit from all-candidate computation itself. The revised manuscript clarifies this distinction in Section 4.4 and Appendix C; the component comparisons are reported in Section 4.5, Table 5.

Q2.2. What interaction contributes beyond routing-logit correction

CAEIR makes the candidate responses available to two mechanisms:

  • Response-conditioned routing correction: Interaction features generate expert-specific corrections to the input-conditioned logits, affecting both the Top-1 choice and its gating probability.

  • Response modulation: After selection, a separate head uses the interaction features to generate a bounded multiplier for the selected response. This changes the strength of the residual contribution without changing the expert selected at that layer.

Consequently, unselected experts can contribute evidence to the routing and modulation decisions even though their response tensors are not directly injected into the backbone. The all-candidates/no-interaction control lacks these pathways.

We separated the two mechanisms using the same three seeds (3407–3409). The input-conditioned Top-1 configuration disables both mechanisms; cross-expert self-attention with response modulation disabled enables routing correction only; input-conditioned routing with response modulation enables modulation only; and CAEIR enables both. The table reports video-level AUC (%) as mean ± sample standard deviation.

ConfigurationCDFDFDCFFIW
Input-conditioned Top-1 (α = 0, β = 0)95.27 ± 0.4479.38 ± 1.2589.14 ± 1.59
Cross-expert self-attention; correction only (α = 0.1, β = 0)95.96 ± 0.3081.01 ± 0.2690.38 ± 0.99
Input-conditioned routing with response modulation (α = 0, β = 0.075)95.08 ± 0.5280.47 ± 0.8089.90 ± 1.15
CAEIR (α = 0.1, β = 0.075)96.26 ± 0.3881.66 ± 0.4990.46 ± 1.00

Adding cross-expert routing correction to input-conditioned Top-1 improves the mean AUC by 0.69, 1.63, and 1.24 percentage points on CDF, DFDC, and FFIW, respectively. Adding response modulation to the corrected routing (CAEIR versus correction-only) yields further mean improvements of 0.30, 0.65, and 0.08 points. Response modulation with input-conditioned routing improves DFDC and FFIW but slightly decreases CDF, indicating that its benefit is dataset-dependent and that it does not independently explain CAEIR's gains.

These ablations support two contributions of the interaction features: informing expert selection and gating, and modulating the selected residual response. They do not establish a separate causal claim about indirect effects on expert optimization during training. The component ablations are reported in Section 4.5, Table 5; execution-path checks and diagnostic limitations are in Appendix C.

AuthorsOpenReview ↗

Response to Reviewer VyGt [Part 3/7]

Q3. MoE baseline comparison under shared protocol settings

We appreciate the request for a controlled external comparison. We reproduced MoE-FFD, one of the two methods named in the question, matching our training source, augmentations, face-sampling procedure, test manifests, and video-aggregation protocol. As noted in Section 4.1, MoE-FFD retains its native 224 × 224 input resolution, while CAEIR uses 336 × 336, so the input resolutions are not identical. Official code for DFMoE was not publicly available to us, so we could not establish a faithful reproduction of that method for this response. We report the MoE-FFD reproduction in the main cross-dataset comparison and distinguish it from the score reported in the original MoE-FFD paper (Section 4.2, Table 2).

We also evaluated Plain ViT-S/16 and a FLOPs-matched Dense ViT under the same study protocol. These controls assess the contribution beyond the backbone and whether a conventional dense Transformer at a comparable FLOPs budget obtains similar accuracy. They complement the external MoE comparison; they are not additional MoE reproductions.

The table reports video-level AUC (%) as mean ± sample standard deviation over three independently trained seeds (3407, 3408, and 3409).

MethodCDFDFDDFDCFFIW
MoE-FFD (reproduced)91.75 ± 0.6790.65 ± 0.5777.22 ± 1.1286.83 ± 0.75
Plain ViT-S/1694.97 ± 0.3793.08 ± 0.8579.48 ± 0.9187.98 ± 1.07
FLOPs-matched Dense ViT94.96 ± 0.2192.37 ± 0.3278.81 ± 0.4387.61 ± 2.27
CAEIR96.26 ± 0.3895.22 ± 0.5281.66 ± 0.4990.46 ± 1.00

CAEIR exceeds the reproduced MoE-FFD by 4.51, 4.57, 4.44, and 3.63 AUC percentage points on CDF, DFD, DFDC, and FFIW, respectively. It also exceeds Plain ViT-S/16 by 1.29, 2.14, 2.18, and 2.48 points, and the FLOPs-matched Dense ViT by 1.30, 2.85, 2.85, and 2.85 points. These are descriptive differences across the evaluated seeds; we do not infer statistical significance from three runs.

Reviewer_beYDOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

CAEIR integrates CNN, Transformer, and Mamba experts into a ViT backbone for generalizable deepfake detection. Expert-response interactions refine Top-1 routing. The method reports competitive results on four cross-dataset benchmarks.

Critical strengths
  1. Competitive cross-dataset performance under single-source training.

  2. A well-motivated design, supported by routing and expert-pool ablations and source-dependent routing analysis.

Critical weaknesses

W1. Insufficient validation of the core contribution

The contribution centers on combining heterogeneous experts with response-conditioned interaction. Table 3 supports a benefit beyond full-candidate evaluation alone, but does not separate the contributions of cross-expert interaction, additional parameters, and response modulation. Simpler response-conditioned MLP routers and separate component ablations are needed to establish the value of the proposed mechanism over simpler alternatives. Table 4 and Appendix A support architectural diversity, but differing routing preferences do not directly establish that the experts learn complementary forensic features.

W2. Computational cost and the rationale for Top-1 dispatch

CAEIR computes all experts before routing, so Top-1 selection does not provide sparse-computation savings. With all expert responses already available, the advantage of retaining only one output over feature fusion needs stronger justification. Although Table 2 includes a soft-mixture baseline, the two output strategies should be compared while keeping response-conditioned routing and modulation otherwise fixed. Batch-throughput measurements and comparisons with dense models of comparable cost would further clarify the accuracy–efficiency trade-off.

W3. Limited generalization validation and baseline comparability

Key ablations are confined to CDF, and Table 4 uses one seed, leaving it unclear whether the design gains are stable and transfer across datasets. A plain ViT-S/16 baseline under the same training protocol is also missing. The paper explicitly acknowledges that Table 1 mixes training and video-aggregation protocols; consequently, the published-score comparison cannot isolate the method's advantage from protocol differences. Controlled comparisons and ablations on additional datasets are needed.

Final Comment

The new experiments clarify the benefit of Top-1 selection over soft fusion. I’m keeping my score at 2 because the gain from attention over a simpler MLP router is modest, and the lower throughput compared with the FLOPs-matched dense ViT remains a concern for me.

Questions for authors

Q1. Necessity of expert interaction

Could the authors compare independent MLP scoring of each expert response and a parameter-matched MLP router over concatenated responses? These controls would distinguish response information from cross-expert interaction and test the need for attention. Please also ablate route correction and response modulation separately.

Q2. Backbone baseline and expert complementarity

Could the authors provide a plain ViT-S/16 baseline under the same training protocol? Beyond the routing patterns in Appendix A, could feature-redundancy analysis or targeted expert ablations further substantiate complementary forensic representations?

Q3. Top-1 dispatch and inference efficiency

Building on the soft-mixture baseline in Table 2, could the authors compare Top-1 dispatch and soft fusion while keeping response-conditioned routing and modulation otherwise fixed? Please also report batch throughput and compare against dense models of comparable computational cost on the same hardware.

Q4. Generalization and evaluation consistency

Could the authors extend key ablations to DFDC and FFIW and report means and standard deviations across seeds? Re-evaluating a representative strong baseline under the same training, preprocessing, and video-aggregation protocol would also strengthen the external comparison.

Q5. Clarity of Figure 2

Figure 2 depicts faces at the expert-pool input, although the modules process intermediate ViT spatial tokens. Please distinguish feature representations from raw images to avoid ambiguity.

5 comments
AuthorsOpenReview ↗

Response to Reviewer beYD [Part 1/5]

Q1. Response-interaction baselines and component ablations

We appreciate the focus on separating the effect of response information from cross-expert interaction and assessing routing correction independently from response modulation. The requested controls address these distinctions. We therefore conducted both MLP comparisons and the separate component ablations. All configurations were trained independently under the same training and evaluation protocol, including the same training data, training budget, checkpoint-selection rule, and test sets. We report video-level AUC (%) as mean ± sample standard deviation over three training seeds (3407, 3408, and 3409). All configurations retain Top-1 output dispatch.

Q1.1. Independent-response and parameter-matched MLP controls

The independent-response MLP, concatenated-response MLP, and cross-expert self-attention use the same pooled expert-response representations and add response-derived corrections to the original CLS-based routing logits with α = 0.1. Response modulation is disabled in all three variants (β = 0). They differ in how the correction is generated: the independent-response MLP scores each response separately, the concatenated-response MLP jointly processes all four pooled responses, and self-attention models interactions among the four response nodes.

Routing correctionCDFDFDCFFIW
Independent-response MLP94.86 ± 0.6680.00 ± 0.2688.34 ± 2.39
Concatenated-response MLP95.18 ± 0.3980.46 ± 0.4390.22 ± 1.40
Cross-expert self-attention (β = 0)95.96 ± 0.3081.01 ± 0.2690.38 ± 0.99

For parameter matching, the concatenated-response MLP uses a hidden dimension of 481, yielding 741,225 parameters per routing-correction branch versus 741,313 for self-attention, a difference of approximately 0.012%.

Self-attention achieved higher mean AUC than the parameter-matched concatenated-response MLP on all three datasets, with gains of 0.78, 0.55, and 0.16 percentage points on CDF, DFDC, and FFIW, respectively. The gains were larger on CDF and DFDC; on FFIW, the small difference indicates that the MLP remained competitive. Relative to independent-response scoring, self-attention achieved higher mean AUC by 1.10, 1.01, and 2.04 percentage points, respectively. Because the concatenated-response MLP can also model cross-expert dependencies, its comparison with self-attention evaluates two interaction mechanisms at essentially the same parameter budget. These results support attention-based interaction among the evaluated alternatives, including when response modulation is disabled.

Q1.2. Separate ablations of routing correction and response modulation

We additionally evaluate a 2 × 2 ablation using α ∈ {0, 0.1} for routing correction and β ∈ {0, 0.075} for response modulation. Disabling modulation sets its multiplicative factor to one while retaining ordinary routing-probability weighting. In the response-modulation-only configuration (α = 0, β = 0.075), the interaction module still supplies the modulation signal, but its routing correction is disabled.

ConfigurationCDFDFDCFFIW
Input-conditioned Top-1 (α = 0, β = 0)95.27 ± 0.4479.38 ± 1.2589.14 ± 1.59
Cross-expert self-attention; correction only (α = 0.1, β = 0)95.96 ± 0.3081.01 ± 0.2690.38 ± 0.99
Input-conditioned routing with response modulation (α = 0, β = 0.075)95.08 ± 0.5280.47 ± 0.8089.90 ± 1.15
CAEIR (α = 0.1, β = 0.075)96.26 ± 0.3881.66 ± 0.4990.46 ± 1.00

Adding cross-expert routing correction to input-conditioned Top-1 (correction-only versus input-conditioned Top-1) improves mean AUC by 0.69, 1.63, and 1.24 points on CDF, DFDC, and FFIW. Adding response modulation to the corrected routing (CAEIR versus correction-only) provides further mean gains of 0.30, 0.65, and 0.08 points, respectively. Response modulation with input-conditioned routing has dataset-dependent effects: it improves DFDC and FFIW but decreases CDF AUC by 0.19 points.

Together, these results identify routing correction as the larger source of improvement in these comparisons, with modulation providing a smaller additional benefit when combined with it. CAEIR achieves the highest mean AUC on all three datasets. The two sets of controls therefore support the value of attention-based routing correction beyond independent response scoring and parameter count, while separating its contribution from response modulation. These results are reported in Section 4.5, Tables 4–5.

AuthorsOpenReview ↗

Response to Reviewer beYD [Part 2/5]

Q2. Backbone baseline and expert complementarity

We evaluated the requested Plain ViT-S/16 baseline and a FLOPs-matched Dense ViT under the same training and evaluation protocol. The dense model controls for computational budget. The table reports video-level AUC (%) as the mean ± sample standard deviation over seeds 3407–3409.

ModelCDFDFDDFDCFFIW
Plain ViT-S/1694.97 ± 0.3793.08 ± 0.8579.48 ± 0.9187.98 ± 1.07
Dense ViT (FLOPs-matched)94.96 ± 0.2192.37 ± 0.3278.81 ± 0.4387.61 ± 2.27
CAEIR96.26 ± 0.3895.22 ± 0.5281.66 ± 0.4990.46 ± 1.00

CAEIR has the highest mean on all four datasets. The expert-pool comparison below uses the same seeds.

Expert poolCDFDFDCFFIW
CAEIR96.26 ± 0.3881.66 ± 0.4990.46 ± 1.00
CNN-only94.68 ± 0.2580.14 ± 0.9288.98 ± 1.07
Transformer-only95.41 ± 0.5379.96 ± 0.4690.18 ± 1.43
State-space-only95.54 ± 0.4579.66 ± 0.3089.89 ± 1.25

The heterogeneous pool has the highest mean AUC on all three datasets. Its FFIW margin over the best homogeneous pool is less than 0.3 percentage points.

We further ablated the scan-direction composition by replacing H+V with H+H or V+V while retaining a four-expert pool. All three configurations were evaluated with seeds 3407–3409 under the same training and evaluation protocol. The table reports video-level AUC (%) as mean ± sample standard deviation.

Scan compositionCDFDFDCFFIW
H+H95.42 ± 0.5280.00 ± 0.1989.30 ± 0.84
V+V95.47 ± 0.1280.49 ± 0.7190.08 ± 1.36
H+V (CAEIR)96.26 ± 0.3881.66 ± 0.4990.46 ± 1.00

The H+V composition achieves the highest mean AUC on all three datasets, exceeding the stronger same-direction control, V+V, by 0.79, 1.17, and 0.38 percentage points on CDF, DFDC, and FFIW, respectively. Because all configurations retain four experts, these results support the benefit of mixing horizontal and vertical scan directions beyond expert count alone. The FFIW gain over V+V is modest and does not hold for every seed; we interpret these results as descriptive evidence of an average performance benefit, without claiming statistical significance from three runs.

For a post hoc response-similarity check, we analyzed the archived CAEIR checkpoint trained with seed 3407 on 256 CDF test videos (128 real and 128 fake; sampling seed 3407), using the middle face crop from each video. At ViT blocks 4, 8, and 12, all four experts received the same inputs in the same order. We pooled their raw responses identically and computed pairwise linear CKA before applying routing weights, response modulation, or residual injection.

Off-diagonal CKA ranges from 0.77 to 0.91 at block 4 and from 0.44 to 0.82 at block 8. At block 12, CNN and Transformer remain highly similar (0.97), whereas their similarities to Mamba-H and Mamba-V range from 0.52 to 0.61; the similarity between Mamba-H and Mamba-V is 0.62. These results show non-uniform, depth-dependent response similarity, not uniformly low redundancy.

The results show non-uniform, depth-dependent response similarity, while the pool ablations demonstrate a performance benefit from heterogeneous composition. The baseline comparison is reported in Section 4.2, Table 2; the expert-pool and scan-direction controls are in Section 4.5, Table 6; and CKA is shown in Appendix C, Figure A4. Per-seed values are provided in Appendix E. These results do not establish that every expert contributes unique forensic information or is individually necessary.

AuthorsOpenReview ↗

Response to Reviewer beYD [Part 3/5]

Q3. Top-1 versus soft fusion and inference efficiency

We appreciate the focus on both predictive performance and practical efficiency. We conducted a controlled soft-fusion comparison, measured batch throughput, and evaluated a FLOPs-matched dense Transformer baseline. These results are reported in Section 4.5, Tables 7–8.

Q3.1. Controlled comparison with soft fusion

The new soft-fusion variant retains the same four-expert pool, insertion layers (blocks 4–12), response-conditioned routing (α = 0.1), and response modulation (β = 0.075). Only the final output aggregation changes: Top-1 retains the selected expert response with its gating weight, whereas soft fusion combines all four responses using their gating weights. This control is distinct from the original soft-mixture baseline in Table 2.

All variants follow the training and evaluation protocol of our submission. Results below are video-level AUC (%) as mean ± sample standard deviation over three training seeds (3407, 3408, and 3409).

ModelCDFDFDCFFIW
FLOPs-matched dense ViT94.96 ± 0.2178.81 ± 0.4387.61 ± 2.27
CAEIR with soft fusion95.02 ± 1.2880.12 ± 0.5689.42 ± 1.23
CAEIR with Top-196.26 ± 0.3881.66 ± 0.4990.46 ± 1.00

With routing and modulation held fixed, Top-1 improves mean AUC over soft fusion by 1.24, 1.54, and 1.04 percentage points on CDF, DFDC, and FFIW, respectively. These results support the accuracy benefit of retaining a selected expert response under the evaluated setting. Both variants still compute all candidate responses; Top-1 does not save candidate-expert computation.

Q3.2. FLOPs-matched dense baseline

The dense baseline retains the 12-layer ViT-S/16 backbone and inserts nine independently parameterized Transformer residual blocks after backbone blocks 4–12, giving 21 executed blocks. Each added block has feature dimension 384, six attention heads, and MLP hidden width 2368. Its update is xout=x+γl(Blockl(x)−x)x_{\mathrm{out}} = x + \gamma_l(\mathrm{Block}_l(x) - x), with a learnable scalar γ_l initialized to zero. All added blocks are executed, and a final LayerNorm and CLS-based binary classification head produces the prediction.

We selected the added blocks’ MLP width from 1024–4096 in increments of 64 solely to minimize the FLOPs difference, without using AUC for architecture selection. Their number, positions, feature dimension, and head count were fixed. The resulting model has 43.49M parameters and 44.49 GFLOPs, approximately 0.25% below CAEIR’s 44.60 GFLOPs.

Reviewer_xaw8OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CAEIR, a cross-architecture mixture-of-experts framework for generalizable deepfake detection. The method integrates CNN, Transformer, and two direction-specific Mamba experts into a unified expert pool. Unlike conventional input-conditioned routing, CAEIR computes all candidate expert responses, summarizes them into evidence nodes, and models cross-expert interactions to refine the Top-1 routing decision. The selected expert response is then modulated and injected into the ViT backbone. The model is trained using real videos from FaceForensics++ with online self-blending supervision. Experiments on four external datasets demonstrate competitive cross-dataset generalization, and several ablation studies investigate the effectiveness of the proposed routing mechanism and heterogeneous expert design.

Critical strengths
  1. Strong cross-dataset generalization performance. CAEIR achieves competitive video-level AUCs of 96.61%, 95.71%, 82.19%, and 89.76% on Celeb-DF v2, DFD, DFDC, and FFIW, respectively. These results demonstrate the effectiveness of the proposed framework across different manipulation methods and data distributions.
  2. Comprehensive ablation studies supporting the proposed design. The paper provides extensive ablation studies covering routing strategies, expert architectures, insertion depths, and response modulation. In particular, Table 3 shows that CAEIR improves the Celeb-DF video-level AUC from 94.88% to 96.61% compared with the all-candidate baseline without interaction, providing evidence for the effectiveness of interaction-aware routing.
Critical weaknesses
  1. Limited methodological novelty. The proposed framework primarily combines established components, including heterogeneous CNN/Transformer/Mamba experts, self-attention-based feature interaction, and Top-1 routing. Although using candidate expert responses to calibrate routing decisions is a reasonable and potentially useful design, the overall methodological contribution appears incremental. The paper would benefit from a clearer explanation of what fundamentally distinguishes the proposed interaction-aware routing mechanism from existing heterogeneous MoE approaches and simpler response-conditioned routing alternatives.
  2. Insufficient statistical validation of the experimental improvements. The paper reports results primarily from single experimental runs without providing means, standard deviations, or confidence intervals across multiple random seeds. Some improvements supporting the architectural design are relatively modest. For example, Table 4 shows a 0.65 percentage-point improvement in video-level AUC over the strongest homogeneous expert pool, while Table 2 shows a 0.61 percentage-point improvement over the soft expert mixture. Without statistical analysis, it is difficult to determine whether these improvements are consistent and statistically significant.
Questions for authors
  1. Could the authors further clarify the methodological novelty of interaction-aware routing compared with existing heterogeneous MoE approaches? In particular, would a simpler response-conditioned routing mechanism without cross-expert self-attention achieve comparable performance?
  2. Could the authors report results over multiple random seeds, including means and standard deviations or confidence intervals, for the key ablation experiments in Tables 2–4? This would help establish the robustness of the reported improvements.
2 comments
AuthorsOpenReview ↗

Response to Reviewer xaw8 [Part 1/2]

Q1. Methodological novelty and simpler response-conditioned routing

We appreciate this question because it separates two issues that require distinct evidence: how CAEIR differs from existing heterogeneous MoE approaches and whether cross-expert attention adds value. We therefore clarify the routing mechanism and compare it with simpler response-conditioned alternatives.

Q1.1. How architectural diversity enters the routing decision

CAEIR places CNN, Transformer, and two direction-specific state-space experts in a common pool at each augmented ViT layer. All four process the same spatial tokens. Their pooled responses serve as evidence for selecting the spatial response to inject. Architectural diversity therefore affects both the candidate transformations and their selection.

MoE-FFD (Kong et al., 2026) uses separate pools of LoRA experts with different ranks and convolutional adapter experts, gated by input features. DFMoE (Wang and Liu, 2025) selects among experts with different capacities and scales based on input characteristics. Response-conditioned combination has also been explored: Forensic-MoE (Fang et al., 2025) derives fusion weights from expert features for final classification. CAEIR instead uses interactions among candidate responses to correct an input-conditioned prior before making a Top-1 spatial-residual selection at each augmented layer.

Specifically, the CLS representation supplies initial expert logits. Interacting pooled responses produce expert-specific logit corrections, and the corrected scores select the response. A separate bounded head modulates its residual strength. We clarify this coupling of architecture-level candidates and layerwise selection in Sections 3.2–3.4 of the revised manuscript.

Q1.2. Comparison with simpler response-conditioned routers

We tested three response-conditioned routing corrections: an independent-response MLP, a concatenated-response MLP, and cross-expert self-attention. They use the same pooled responses and add corrections to CLS-based logits with α = 0.1; response modulation is disabled (β = 0). Each is trained under the same protocol. The concatenated-response MLP can model cross-expert dependencies and closely matches the attention module in parameters (741,225 versus 741,313 per layer).

Video-level AUC (%) is reported as mean ± sample standard deviation over seeds 3407–3409.

Routing correctionCDFDFDCFFIW
Independent-response MLP94.86 ± 0.6680.00 ± 0.2688.34 ± 2.39
Concatenated-response MLP95.18 ± 0.3980.46 ± 0.4390.22 ± 1.40
Cross-expert self-attention (β = 0)95.96 ± 0.3081.01 ± 0.2690.38 ± 0.99

Cross-expert self-attention achieved higher mean AUC than the parameter-matched concatenated-response MLP on all three datasets, with gains of 0.78, 0.55, and 0.16 percentage points on CDF, DFDC, and FFIW, respectively. The gains were larger on CDF and DFDC; on FFIW, the small difference indicates that the MLP remained competitive. Relative to the independent-response MLP, self-attention achieved higher mean AUC by 1.10, 1.01, and 2.04 percentage points, respectively. These results support attention as an effective implementation in our setting, but do not establish that it is necessary. The comparison is reported in Section 4.5, Table 4.

The three-seed architecture-pool comparison also favors the heterogeneous pool over the evaluated homogeneous four-expert pools on CDF, DFDC, and FFIW. The FFIW margin over the strongest homogeneous pool is small; AUC comparisons alone do not establish that the learned forensic features are complementary.

AuthorsOpenReview ↗

Response to Reviewer xaw8 [Part 2/2]

Q2. Multi-seed robustness of key ablations

We appreciate the emphasis on robustness. To address it, we re-evaluated the principal configurations from Tables 2–4 of the submission using seeds 3407, 3408, and 3409 under the same training and evaluation protocol. We report the same-weight execution control and newly added response-conditioned routing controls separately. The table reports CDF video-level AUC (%) as the mean ± sample standard deviation. The single-run values in the submission should not be interpreted as three-seed means.

Comparison groupConfigurationCDF AUC (%)
Submitted Table 2, repeated over seedsInput-conditioned Top-195.27 ± 0.44
Submitted Table 2, repeated over seedsOrthogonality-regularized routing95.53 ± 0.25
Submitted Table 2, repeated over seedsTop-2 cooperative routing95.13 ± 0.27
Submitted Table 2, repeated over seedsLoad-biased routing95.36 ± 0.59
Submitted Table 2, repeated over seedsOriginal soft expert mixture95.37 ± 0.57
Submitted Tables 2 and 3, repeated over seedsCAEIR96.26 ± 0.38
Table 3 same-weight execution controlAll candidates, no interaction95.27 ± 0.44
Added response-conditioned routing controlIndependent-response MLP (α = 0.1, β = 0)94.86 ± 0.66
Added response-conditioned routing controlConcatenated-response MLP (α = 0.1, β = 0)95.18 ± 0.39
Added response-conditioned routing controlCross-expert self-attention (α = 0.1, β = 0)95.96 ± 0.30
Submitted Table 4(a), repeated over seedsCNN-only expert pool94.68 ± 0.25
Submitted Table 4(a), repeated over seedsTransformer-only expert pool95.41 ± 0.53
Submitted Table 4(a), repeated over seedsState-space-only expert pool95.54 ± 0.45
Added four-expert scan-direction controlH+H95.42 ± 0.52
Added four-expert scan-direction controlV+V95.47 ± 0.12

CAEIR has the highest mean CDF AUC among the independently trained configurations in these comparisons. Table 3 of the submission reported 94.88% for “All candidates, no interaction” in a separate single run. The 95.27% value above comes from the current same-weight execution control: for each seed, this path and input-conditioned Top-1 select the same experts and produce matching predictions on the tested inputs. It is not a three-seed estimate from independently trained repetitions of the original Table 3 configuration. The Shared-CNN, conflict-entropy, and three-expert single-scan variants have results only for seed 3407; we identify these as single-run diagnostic results in Appendix C and do not treat them as repeated-run estimates. The revised abstract reports the four-dataset video AUC as mean ± sample standard deviation over seeds 3407–3409, consistent with the shared-protocol results in Section 4.2, Table 2. Repeated-seed ablations are summarized in Sections 4.4–4.5, Tables 3–6, with individual-seed values in Appendix E. These three-seed summaries are descriptive and are not presented as evidence of statistical significance.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.