- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes ASNet, an audio deepfake detection framework designed to improve reliability under distribution shifts and adversarial perturbations. The central idea is to derive a class-symmetric energy score from the detector, stabilize this score before using it as a training signal, and then reuse the same score for inference-time abstention. Experiments under single-source training show improvements in cross-corpus robustness and calibration over compute-matched baselines, while adaptive attacks also reveal limitations of the shared energy-based reliability mechanism.
Critical strengths
- Deepfake detectors can remain highly confident despite substantial degradation under unseen generators, codec/channel shifts, or adversarial perturbations. The attempt to jointly consider generalization, adversarial robustness, calibration, and abstention is important.
- The experimental evaluation is relatively thorough.
- I appreciate that the authors explicitly evaluate an attacker targeting the proposed energy-based rejection mechanism. The substantial degradation under this attack is an important negative result and gives a more realistic picture of the method’s security limitations.
Critical weaknesses
-
The novelty is somewhat incremental and distributed across several existing ideas. Energy/confidence scoring, adversarial training, uncertainty-based abstention, instance-dependent robustness weighting, multi-view representations, and consistency regularization are all established techniques.
-
The shared energy score creates a single point of failure. The same quantity controls robustness allocation and deployment-time rejection. Once an adaptive attacker explicitly targets this score, error catch drops dramatically (to 48%, and 35% under the stronger joint attack). This raises an important question about whether coupling training and inference through the same reliability signal is ultimately preferable to using separate or independently learned signals.
-
External robustness remains fairly limited despite the improvement over baselines. In particular, CodecFake remains difficult, with a Robust AUC of only 0.56, and a nontrivial fraction of errors remain below the fixed abstention threshold. Thus, the relative improvements are encouraging, but the practical reliability of the resulting detector under substantial distribution shift remains unclear.
-
The baselines considered in the paper are not the state-of-the-art detector for audio deepfake detection thus the comparison is not that fair.
-
The paper only evaluates the robustness against adversarial perturbation. For audio deepfake detection, however, the robustness against real-world corruption[1] is equally important.
[1] Li, Xiang, Pin-Yu Chen, and Wenqi Wei. "Measuring the Robustness of Audio Deepfake Detection under Real-World Corruption." Proceedings of the Sixteenth ACM Conference on Data and Application Security and Privacy. 2026.
Questions for authors
- Can the author compare with more advanced audio deepfake detectors using foundation models?
- Can the authors provide more evidences to support the proposed method's robustness against real-world perturbations?
- Given the relatively low absolute robustness on CodecFake, could the authors provide more analysis of which types of codec/generator shifts cause ASNet to fail and whether those failures correlate with instability of the energy score? This would help establish whether the proposed stabilization mechanism addresses the intended failure mode rather than primarily improving average performance.
5 comments
W5/Q2 — Non-adversarial corruption robustness. We evaluate 18 corruption types—noise (3), modification (8), compression (7)—on LibriSeVoc and WaveFake under a matched protocol (Sec. 5.1; Table 26; App. D.10). Against compute-matched Ren, ASNet has smaller clean-relative AUC drops in every family: equal-family means 0.043 vs 0.062 (LibriSeVoc) and 0.073 vs 0.103 (WaveFake). Noise, absent from the specified consistency augmentation, also favors ASNet (0.030 vs 0.043; 0.057 vs 0.076). Weighting by operator counts preserves the advantage (0.047 vs 0.067; 0.077 vs 0.110, derived from rounded family means). We use the three corruption families of Li et al., who also study 18 types. Although severities and corpora differ, our family-level results are consistent with their broader observation that noise is generally less damaging than modification and compression; this is not a reproduction of their evaluation. Per-severity and physical-playback robustness are not established.
Thank you for the detailed response and additional experiments. I appreciate the authors' efforts to strengthen the paper, particularly through the analysis of Codecfake failure modes and comparisons with stronger pretrained detectors.
However, my main concerns remain. First, the novelty is still largely incremental. The additional ablations demonstrate the effectiveness of combining and ordering established components, but do not establish a sufficiently distinct methodological contribution. Second, while the alternative rejection experiments are informative, they do not directly evaluate decoupling training-time robustness allocation from inference-time rejection. The shared-score vulnerability under adaptive attacks therefore remains a concern, and the benefit of coupling these mechanisms is not sufficiently established.
I therefore keep my original score.
We thank the reviewer for highlighting the adaptive-attack result. The revised manuscript evaluates foundation-model baselines, non-adversarial corruptions, CodecFake failure conditions, and alternative rejection rules. All corresponding revisions are highlighted in violet, and overlapping methodological controls are cross-referenced where relevant.
W1 — Incremental novelty. Individual ingredients are established, and binary energy has the same uncertainty ordering as entropy and direction-corrected margin on a fixed detector (Eqs. 4, 29). The claim is the tested stabilization–allocation dependency, not a new score: consistency bounds gate drift (Eqs. 7, 21); conditioning adds +0.07 source Robust AUC after stabilization versus +0.04 without it (descriptive interaction +0.03; Table 28). Uniform / fixed-positive / learned gating gives 0.830 / 0.872 / 0.900 (Table 29), and matched nominal exposure yields stabilization-first 0.880 versus allocation-first 0.840 (Table 31). Controller-only transfer on the Ren backbone adds +0.045 external Robust AUC (Table 40). These controls identify a meaningful complete-system and allocation-rule effect, without claiming unique optimality or independently novel primitives (Secs. 1, 5.3; Apps. E.2–E.3/E.9).
W2 — Shared-score vulnerability. Matched attacks reduce induced-error catch 61% to 48% (energy-aware PGD) and 56% to 35% (joint APGD), with UASR reaching 17.7%/31.2% (Table 15). Under rule-aware APGD and fixed ECRM training, auxiliary-only and Mahalanobis rejection yield UASR 33.8%/37.4%, versus 31.2% for shared energy; energy+auxiliary reduces it to 26.1% (Table 16). ECRM improves both shared (36.8% to 31.2%) and combined (31.9% to 26.1%) rejection relative to Uniform. The 5.1/4.9-point selector gains do not establish an interaction. In separate reruns, substituting an auxiliary or Mahalanobis score for both allocation and abstention yields source Robust AUC 0.88/0.86, versus 0.90 for energy (Table 39). These variants still share their selected signal across both roles. The tested separate rejection signals increase rule-aware attack success, whereas combining energy with the auxiliary signal reduces it.Clean source coverage is 79.2–80.5%; on CodecFake, coverage decreases from 70.8% with shared-energy rejection to 68.6% with energy–auxiliary rejection. The finite-budget evidence does not establish secure abstention, and an independently trained rejection model is not evaluated (Secs. 5.2, 6; Apps. D.2, E.8).
W3/Q3 — CodecFake failure modes and reliability. CodecFake is hard for every detector evaluated: ASNet reaches 0.560 PGD-10 Robust AUC against 0.496 (compute-matched Ren) and 0.512–0.534 (standalone SSL) (Tables 18–19). Comparable degradation across the tested detectors indicates a challenging transfer setting beyond ASNet alone. Condition-level analysis localizes it: benchmark-held-out DAC/C7 and AudioGen/A3, a non-speech shift, give Robust AUC 0.389/0.441 and fake-miss rates 46.3%/41.7% (Table 23). Failures track energy instability: across ten conditions, drift and fake-miss rank-correlate at ρ=+0.939 (+0.24 per utterance). On identical pairs, removing scheduled consistency raises prediction flips 8.3%→11.5% and correct-to-wrong transitions 6.2%→8.8% (Table 24); stratified by within-model drift, transitions rise 2.1%→12.9% (ASNet) and 3.8%→15.9% without it, ASNet lower at every rank and the difference larger in the upper quintiles (Table 25). Quintiles are defined per model, so rank-wise differences are descriptive. Reliability stays bounded: 34% of clean CodecFake errors fall below the source-fixed threshold (Table 14). These are associations, not causal mediation.
W4/Q1 — Are stronger pretrained detectors evaluated fairly? Wav2Vec2-BERT, WavLM-Large+AASIST, and XLS-R+AASIST are fine-tuned on LibriSeVoc and evaluated with the same external splits and waveform PGD-10 protocol (Tables 18–19; App. D.4). They exceed original ASNet on external clean AUC (0.901/0.913/0.924 versus 0.881). Original ASNet reaches external Robust AUC 0.650 versus 0.604/0.618/0.631, the XLS-R margin being within reported dispersion, and source ECE 0.101 versus 0.172/0.159/0.151. WavLM+ASNet reaches 0.686 external Robust AUC but changes the system. SSL pretraining and resource budgets differ; SSL rows have one downstream fine-tuning run, so these are evaluation-matched system comparisons, not compute-matched tests or significance claims.