acceptodds
Under review as a conference paper at ICLR 2027

VOICE OR LENGTH? COUNTERFACTUAL AUDITING OF TIMING SHORTCUTS IN AUDIO DEEPFAKE DETECTION

173 views

Abstract

A benchmark can remove a shortcut from its evaluation data while leaving it in its training data. A detector may then learn a highly predictive nuisance cue that the benchmark neither rewards nor exposes. We study this gap for timing in audio deepfake detection, and separate a shortcut’s availability in a corpus from a detector’s reliance on it. Availability: ASVspoof 5 equalises timing across classes, by design, only in the partitions it tests on; on held-out speakers of its training partition, timing alone separates the classes at AUC 95.6, against chance on its evaluation set. Across four corpora the predictive component differs, and on ASVspoof 2019 LA (LA19) the components that survive boundary trimming, the standard control, retain AUC 87.7 of the 88.3 available. Reliance: matched-length evaluation confounds speech amount with boundary context and a shifting population, so we intervene instead, holding the speech byte identical. Planting real bonafide boundary silence on spoofed ASVspoof 5 clips raises a released XLS-R Conformer from 5.8 to 9.4% EER, against 7.1% when every clip receives it. Incentive: fine-tuning that checkpoint further on its natural view lowers its LA19 benchmark EER from 0.35 to 0.17% while raising its length-matched In-the-Wild EER by 4.7 points, so, in this setup, selecting on the benchmark picks the detector that relies more on the shortcut. Randomly placed training crops reverse this and remove the planted-silence reliance, at a cost in LA19 EER, and turning the intervention into an objective — a consistency loss between each crop and the same speech with label-independent silence added — gives the lowest length-matched In-the-Wild EER of any LA19-trained detector (6.6%, from 9.2%); a duration-aware architecture, tested over twenty seeds, does not help. For timing, removing the cue from a test set does not test for it; intervening on it does. We release the code, ports of five released checkpoints, and per-clip scores for every condition.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Audio Deepfake Detection, Shortcut Learning, Temporal Bias, Speech Representation Learning, Robustness, Dataset Bias, Evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NaV5OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies timing shortcuts in audio deepfake detection, focusing on distinguishing whether timing cues are merely available in a dataset from whether trained detectors actually rely on them. The authors analyze duration and silence statistics across several datasets, evaluate released detectors under matched-length conditions, and introduce counterfactual interventions that modify boundary silence while keeping the speech unchanged. They further investigate different training views and propose counterfactual timing invariance (CTI) to reduce reliance on timing cues. The main finding is that standard benchmark performance can hide shortcut reliance, while randomized cropping and consistency-based training can improve robustness to such shortcuts.

Critical strengths
  • The analysis of the detectors' shortcut reliance on silence is interesting.
  • The experiments are fairly comprehensive.
Critical weaknesses
  • The paper is quite difficult to follow. The presentation is dense with many experimental conditions, abbreviations, metrics, controls and numerical comparisons introduced in a messy way. A substantial revision/restructuring is necessary.
  • The overall novelty and significance of the findings are somewhat limited. Timing and silence artifacts in audio deepfake benchmarks have already been studied, as the paper itself acknowledged. The counterfactual framing makes the evidence cleaner, but I am less convinced that the empirical findings themselves substantially change our understanding of the problem.
  • CTI improves some cross-dataset results but introduces clear trade-offs, including worse performance on some benchmark/transfer settings. This makes it unclear whether CTI should be viewed as a generally useful mitigation strategy or primarily as evidence supporting the authors’ counterfactual methodology.
Questions for authors
  1. Since timing and silence shortcuts in ASVspoof have already been established, what do the authors consider the main new scientific conclusion beyond providing a cleaner causal test of detector reliance? In particular, which finding would not reasonably be expected from the existing shortcut-learning literature?
  2. Random cropping appears to account for a substantial portion of the robustness improvement, even when silence is retained. Could the authors clarify what CTI provides beyond sufficiently diverse temporal cropping/augmentation, especially considering that CTI does not consistently improve performance across datasets?
  3. The paper shows cases where improving the standard benchmark score increases shortcut reliance. How sensitive is this conclusion to the particular fine-tuning setup and training budget? Would the same relationship hold when models are trained from scratch or selected using a more diverse validation distribution?
  4. Could the authors include additional experiments with more recent models to further validate the generality of the findings?
Reviewer_aTuVOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether audio deepfake detectors rely on timing, and it separates two questions: does a corpus offer a timing cue (availability), and does a detector use it (reliance)? Overall, the results show that random training crops remove this reliance, a consistency loss gives the best length-matched ITW EER, and ta duration-aware architecture does not help

Critical strengths
  1. The reliance test is a genuine intervention. Planting bona fide silence on spoofs only, against an every-clip control and with the speech held fixed, separates real reliance from mere sensitivity to extra non-speech, and a digital-silence control confirms the effect comes from timing.
  2. The ASVspoof 5 finding changes how the benchmark should be read. Its training partition carries a stronger timing cue than LA19, so a detector can learn it without the benchmark ever exposing it, and the validated ports and per-clip scores make the audit easy to build on.
Critical weaknesses

1.Some headline claims exceed the paper's own outcomes. The abstract says benchmark selection favours the more shortcut-reliant detector, yet the transplant effect falls after natural-view fine-tuning (net about 2.4 to 1.4, Table 2). Likewise, CTI's "invariance" is tested only on the intervention it was trained against, and the claim that ASVspoof 5 evaluation data carry no timing cue rests on transfer from training data alone. 2. The training results lack key controls and uncertainty. No CTI arm keeps the paired views with lambda = 0, so not sure if its 1.2-point gain (three seeds, at a 5.2-point cost on FoR) can be credited to the consistency term.

Questions for authors
  1. Which reliance measure increases after natural-view fine-tuning? If none, will the authors restate the incentive claim as a selection effect between training views?
  2. Does CTI beat the same paired views with lambda = 0 at matched compute?
  3. Do the Table 2 ITW contrasts and the ASVspoof 5 transplant effects survive speaker-clustered resampling?
Reviewer_c83ROpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper studies the effect of timing shortcuts on audio deepfake detection performance. These shortcuts, such as clip duration and boundary silence, may be easier to learn, leading detectors to rely on them rather than learning domain-invariant features. Through experiments, the paper shows that the dominant timing cues differ across corpora. The paper also demonstrates the reliance of existing detectors on these shortcuts by inserting silence from real clips into fake ones. It further shows that fine-tuning on the benchmark increases reliance on these timing shortcuts. However, random cropping and a timing-invariant consistency loss can reduce this effect.

Critical strengths
  1. The empirical analysis is carefully conduct and is rigorous.
  2. The timing shortcuts are clearly isolated, and their effects are thoroughly investigated.
Critical weaknesses
  1. My feeling is that this paper reads more like documentation than a method-focused paper. There are many facts and findings listed across the paper, but is the paper presented as an evaluation and benchmark study or a method-focused paper?
  2. Even though multiple datasets are used, they are from the ASV family and ITW, which do not include the most recent TTS systems and VC models, particularly those that leverage pretrained large models.
  3. Leading silence is not a new pitfall in audio deepfake evaluation, as it has been previously reported and is well known in the community. Random cropping will eliminate some shortcuts, leaving a mixture of training samples with and without such shortcuts, which is not surprising and is a fairly standard approach.
  4. Other forms of shortcuts, such as model-specific artefacts, also prevent models from learning domain-invariant features. Would the proposed methods also improve cross-corpus generalisation, or do they only help when samples from other corpora also contain the same shortcuts?
  5. The selection of baseline methods is fairly limited, making it difficult to substantiate the findings.
Questions for authors

Please see my weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.