- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper revisits vocal-tract-based audio deepfake detection and evaluates whether its underlying assumptions still hold for modern TTS and voice conversion models. The authors show that detection performance degrades substantially on several recent synthesis models, suggesting that modern generators can reproduce more realistic vocal-tract characteristics even without explicitly modeling the vocal tract. Through ablation studies, the paper attributes this improvement primarily to learned linguistic representations in TTS models and explicit F0 modeling in VC models. The paper further proposes VoicePatch, a post-generation attack that replaces detector-sensitive phoneme segments with acoustically matched real-speech segments. VoicePatch increases detection bypass rates from around 1% to over 80% even under surrogate/target detector mismatch.
Critical strengths
- The evaluation and analysis are fairly comprehensive.
- The proposed VoicePatch attack is interesting. It exploits the detector’s reliance on specific phonetic regions rather than treating the detector as a generic black box. Its effectiveness under feature-selection mismatch is particularly useful, as it shows that the attacker does not need to reproduce the deployed detector exactly.
Critical weaknesses
-
The finding from the experiments is not surprising.
-
Some causal claims from the ablation studies appear stronger than the evidence supports. The TTS experiments show that perturbing linguistic representations substantially affects the proposed vocal-tract fidelity metric, but this does not necessarily establish that linguistic representations themselves are the primary mechanism responsible for realistic vocal-tract characteristics. Perturbing these representations may also substantially change phonetic realization or other downstream acoustic properties. Similarly, the F0 experiments show that perturbing F0 increases variation in estimated vocal-tract features, but this is indirect evidence for the stronger claim that explicit F0 modeling explains why certain VC models are harder to detect.
-
The conclusions are tied closely to one specific vocal-tract detector. Most of the paper’s conclusions are based on the detector of Blue et al. (2022). It is unclear whether the observed failure is a fundamental limitation of vocal-tract-based detection or primarily a limitation of this particular reconstruction and feature-selection pipeline.
-
The practicality of VoicePatch depends on relatively favorable attacker assumptions. The attacker is assumed to have a large real-speech dataset and knowledge of the target detector design. More importantly, the attack requires finding real segments containing matching phoneme sequences with sufficiently similar timbre and prosody. The experiments use controlled speech corpora (TIMIT/VCTK), so it is unclear how well the method scales to more diverse speakers, languages, accents, or unconstrained real-world speech.
Questions for authors
- Can the authors provide stronger evidence that the linguistic representation and F0 findings are causal rather than consequences of broader changes to the generated speech caused by the perturbations?
- To what extent do the findings generalize beyond the specific Blue et al. detector? Have the authors considered evaluating another vocal-tract/physiology-based detection method or alternative vocal-tract estimation procedure?
- How sensitive is VoicePatch to the size and diversity of the real-speech database? In particular, what happens when suitable phoneme segments with similar timbre/prosody are scarce, as may occur for unseen speakers, accents, or languages?
- Given the noticeable MOS decrease for FreeVC and NeuralVC, could the authors provide human perceptual evaluation of VoicePatch, particularly focusing on whether listeners can detect splice boundaries or changes in speaker identity?