acceptodds
Withdrawn from ICLR 2027

AGDT: Audio Deepfake Detection via Asymmetric Gated Dual-Stage Temporal Modeling

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

In audio deepfake detection under unseen generators, channel distortions, and real-world recording conditions, discriminative evidence may be distributed across representation depths, temporal scales, and utterance-level statistics. To address these characteristics, we propose AGDT, a framework that aggregates evidence across layer depth, temporal scale, and distributional statistics, using XLS-R as its frontend. First, non-competitive sensitive layer gating is applied to the frame-level representations of the XLS-R Transformer layers, allowing multiple layers with complementary discriminative information to contribute to feature fusion. Next, a multi-scale temporal convolution module applies sequential channel and temporal gates to local residual updates before a Conformer backend, while a second module refines the contextual representations without additional gating. This asymmetric configuration places selective local processing before the backend encoder and ungated refinement after it. Finally, attentive statistics pooling combines weighted means and standard deviations to capture both the central tendency and temporal dispersion of frame-level features. For the downstream detection task, the model is trained only on the ASVspoof 2019 LA training set and evaluated on multiple benchmarks involving shifts across datasets, channels, and languages. It achieves equal error rates of 2.48%, 1.56%, 13.57%, and 5.83% on ASVspoof 2021 LA, ASVspoof 2021 DF, ASVspoof 5, and In-the-Wild, respectively. The experimental results demonstrate competitive performance across multiple evaluation domains.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio deepfake detection, Cross-domain generalization, XLS-R

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_mm55OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AGDT, an audio deepfake detection framework that combines sensitive layer fusion, asymmetric dual-stage multi-scale temporal modeling, and attentive statistics pooling. Specifically, XLS-R representations are aggregated through layer-wise gating, followed by a gated multi-scale temporal convolution module (MSTCM), a Conformer encoder, and an ungated MSTCM. The model is trained on ASVspoof 2019 LA and evaluated on multiple cross-domain benchmarks, including ASVspoof 2021 LA/DF, ASVspoof 5, In-the-Wild, and other datasets.

Critical strengths
  1. The paper addresses the important problem of generalization in audio deepfake detection, particularly under unseen synthesis methods, channel distortions, and cross-domain conditions.

  2. The experimental evaluation covers a relatively broad range of datasets, including ASVspoof 5 and several out-of-domain benchmarks. The proposed method achieves competitive results on multiple evaluation sets.

  3. The paper includes ablation studies on temporal module placement, convolution kernel configurations, and encoder architectures, providing useful empirical observations about different temporal modeling strategies.

Critical weaknesses
  1. The methodological novelty is relatively limited. The proposed framework mainly combines existing components, including XLS-R sensitive layer fusion, multi-scale temporal convolutions, channel and temporal attention, a Conformer encoder, and attentive statistics pooling. The main novelty appears to lie in the asymmetric placement of gated and ungated convolution modules around the Conformer. However, this is largely an architectural combination of established techniques, and the paper does not sufficiently demonstrate a fundamentally new modeling mechanism.

  2. The necessity of the asymmetric dual-stage design is not sufficiently justified. Although Table 3 compares different temporal module placements, it does not directly investigate whether the asymmetric gating strategy itself is responsible for the improvements. For example, comparisons between both-gated, both-ungated, and reversed-gating configurations are missing. Without these controls, it is difficult to determine whether the gains arise from the proposed asymmetric design or simply from introducing additional multi-scale convolution modules and model capacity.

  3. The performance improvements over strong baselines are relatively limited, and the contribution of individual components remains unclear. In Table 1, the proposed method improves the best reported EER from 1.77% to 1.56% on ASVspoof 2021 DF and from 14.40% to 13.57% on ASVspoof 5, while performing substantially worse than several baselines on ASVspoof 2021 LA. Although improvements on some additional datasets are more noticeable, the overall advantage is not consistently demonstrated. Moreover, Table 3 indicates that data augmentation contributes substantially to cross-domain performance, particularly on In-the-Wild and ASVspoof 5. Given the strong XLS-R frontend and RawBoost augmentation, the additional benefits attributable specifically to the proposed architectural design require clearer analysis. Tables 1 and 2 also lack aggregate performance measures, and no multi-seed results or statistical variance are reported to establish the reliability of the improvements.

  4. Several experimental configurations and ablation settings require clarification. In Table 3, the variant "w/o SLS" is not sufficiently explained. It is unclear whether removing SLS means using only the final XLS-R layer, averaging representations across layers, or adopting another feature aggregation strategy. This distinction is important for interpreting the contribution of layer fusion. The manuscript should also explicitly clarify whether the XLS-R frontend is frozen or fine-tuned during training and ensure that the comparisons use consistent training and augmentation protocols.

  5. Some presentation and technical details should be improved. The abbreviation "ASP" is used in the ablation study without being explicitly introduced in its abbreviated form. Although attentive statistics pooling is described in the methodology, the terminology should be defined consistently. In addition, the paper would benefit from clearer reporting of model complexity, including total/trainable parameters and computational overhead introduced by the two MSTCM modules. Such information is important for evaluating whether the additional architectural complexity is justified by the observed performance gains.

Questions for authors
  1. The methodological novelty is relatively limited. The proposed framework mainly combines existing components, including XLS-R sensitive layer fusion, multi-scale temporal convolutions, channel and temporal attention, a Conformer encoder, and attentive statistics pooling. The main novelty appears to lie in the asymmetric placement of gated and ungated convolution modules around the Conformer. However, this is largely an architectural combination of established techniques, and the paper does not sufficiently demonstrate a fundamentally new modeling mechanism.

  2. The necessity of the asymmetric dual-stage design is not sufficiently justified. Although Table 3 compares different temporal module placements, it does not directly investigate whether the asymmetric gating strategy itself is responsible for the improvements. For example, comparisons between both-gated, both-ungated, and reversed-gating configurations are missing. Without these controls, it is difficult to determine whether the gains arise from the proposed asymmetric design or simply from introducing additional multi-scale convolution modules and model capacity.

  3. The performance improvements over strong baselines are relatively limited, and the contribution of individual components remains unclear. In Table 1, the proposed method improves the best reported EER from 1.77% to 1.56% on ASVspoof 2021 DF and from 14.40% to 13.57% on ASVspoof 5, while performing substantially worse than several baselines on ASVspoof 2021 LA. Although improvements on some additional datasets are more noticeable, the overall advantage is not consistently demonstrated. Moreover, Table 3 indicates that data augmentation contributes substantially to cross-domain performance, particularly on In-the-Wild and ASVspoof 5. Given the strong XLS-R frontend and RawBoost augmentation, the additional benefits attributable specifically to the proposed architectural design require clearer analysis. Tables 1 and 2 also lack aggregate performance measures, and no multi-seed results or statistical variance are reported to establish the reliability of the improvements.

  4. Several experimental configurations and ablation settings require clarification. In Table 3, the variant "w/o SLS" is not sufficiently explained. It is unclear whether removing SLS means using only the final XLS-R layer, averaging representations across layers, or adopting another feature aggregation strategy. This distinction is important for interpreting the contribution of layer fusion. The manuscript should also explicitly clarify whether the XLS-R frontend is frozen or fine-tuned during training and ensure that the comparisons use consistent training and augmentation protocols.

  5. Some presentation and technical details should be improved. The abbreviation "ASP" is used in the ablation study without being explicitly introduced in its abbreviated form. Although attentive statistics pooling is described in the methodology, the terminology should be defined consistently. In addition, the paper would benefit from clearer reporting of model complexity, including total/trainable parameters and computational overhead introduced by the two MSTCM modules. Such information is important for evaluating whether the additional architectural complexity is justified by the observed performance gains.

Reviewer_uBCyOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AGDT, which fuses multilayer representations from XLS-R and incorporates multiscale temporal convolutions before and after a Conformer, using an asymmetric design with gating before the Conformer and no gating after it. The model is trained on ASVspoof 2019 LA and evaluated on multiple cross-domain datasets.

Critical strengths
  • The evaluation covers nine test sets spanning different languages, channels, and generation methods, providing broad experimental coverage.
  • The results are competitive on several datasets. For example, compared with XLSR-Conformer, EER decreases from 2.58% to 1.56% on 21DF and from 8.42% to 5.83% on ITW.
  • The paper includes component and architectural ablations and explicitly acknowledges that the full model degrades performance on some datasets.
Critical weaknesses

The central asymmetric design lacks direct validation. Table 3 compares modules placed before, after, and on both sides of the Conformer, but does not hold the two-stage structure fixed while varying only the gating configuration. It therefore remains unclear whether the improvements come from asymmetric gating or simply from adding multiscale modules. The model selection procedure is insufficiently clear. Section 4.1 states that models are selected using the 2019 LA development set, but subsequent discussions justify the choices of convolution kernels and encoder depth based on performance on 21DF and ITW. The authors should clarify whether test results informed architectural choices; otherwise, the cross-domain generalization gains may be overstated.

Questions for authors

Which datasets and metrics were used to select the final architecture and checkpoint, respectively? Did test-set results inform either decision? Have the authors conducted experiments that keep the two-stage structure fixed and vary only the gating configurations before and after the Conformer? Are the reported results based on a single training run or multiple runs? Are the key comparisons supported by repeated experiments?

Reviewer_S4jJOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes Asymmetric Gated Dual-Stage Temporal Modeling (AGDT) for robust audio deepfake detection against unseen speech generation methods and recording conditions. The proposed model fuses features from multiple layers and places Multi-Scale Temporal Convolution Modules (MSTCMs) before and after the encoder. The first MSTCM selectively enhances local features through channel gating and temporal gating, while the second MSTCM refines contextualized features without using gates. The model is evaluated on multiple benchmarks.

Critical strengths
  • [S1] Practically Important Problem Setting

Generalization to unseen generation methods and recording conditions is a critical challenge in audio deepfake detection.

  • [S2] Evaluation Across a Wide Range of Benchmarks

Tables 1 and 2 compare the proposed method with several representative detectors. The proposed method achieves the lowest EER among the compared methods on ASVspoof 2021 DF, ASVspoof 5, and In-the-Wild.

  • [S3] Comprehensive Ablation Studies

Tables 3–5 investigate the effects of removing individual modules, varying the placement of MSTCMs, and changing the kernel size, encoder architecture, and encoder depth.

Critical weaknesses
  • [W1] Limited Technical Novelty of the Proposed Method (Sections 2–3)

The main components of the proposed method are all based on existing techniques. The primary novelty of the proposed method lies in placing MSTCMs before and after the Conformer encoder and applying gating only to the first MSTCM.

Although synergistically combining existing architectural components can itself constitute a meaningful research contribution, the paper does not sufficiently present a clear design principle, a new theoretical insight, or the discovery of a phenomenon that cannot be explained by existing methods. From a reviewer’s perspective, the technical contribution remains unclear beyond demonstrating that combining existing modules can improve performance. To strengthen the contribution, the authors should clearly explain why the asymmetric design is necessary and experimentally isolate its specific effects.

  • [W2] The Effectiveness of the Asymmetric Design Is Not Validated (Sections 3.2 and 4.4)

The authors present the asymmetric architecture, in which gating is applied to the pre-MSTCM but not to the post-MSTCM, as a key contribution of their method. However, Table 3 primarily compares different MSTCM placements. These experiments alone are insufficient to determine whether the asymmetry in the gating mechanism itself is important.

  • [W3] Lack of Evidence Supporting the Proposed Mechanism (Sections 3.2 and 4.4)

The authors hypothesize the following mechanism:

  • Pre-MSTCM selectively emphasizes local features indicative of spoofed speech.
  • Conformer integrates long-range contextual information.
  • Post-MSTCM re-extracts local features that have been obscured by contextual information.

However, no analyses are provided to directly support these explanations.

  • [W4] Unclear Statistical Reliability of the Experimental Results (Sections 4.3–4.4)

The paper reports only a single EER value for each experimental condition; however, neither the mean and standard deviation across multiple random seeds nor confidence intervals are provided.

Questions for authors
  • 1 Why asymmetric gating? Could the authors conduct a 2×2 ablation study in which gating in Pre-MSTCM and Post-MSTCM is independently enabled or disabled? Does the Gated–Ungated configuration consistently outperform the other configurations?

  • 2 Computational fairness: What are the parameter count, FLOPs, and inference latency of the proposed method? Could the authors provide comparisons with baselines having comparable parameter counts?

  • 3 Statistical significance: Are the main performance improvements statistically significant, as assessed through multiple random seeds or bootstrap confidence intervals?

  • 4 Frontend training: Is XLS-R 300M frozen or fine-tuned end-to-end? Were the same frontend training conditions applied to the compared baselines?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.