acceptodds
Withdrawn from ICLR 2027

Revisiting Vocal-Tract-Based Detection for Modern Audio Deepfake Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vocal-tract-based audio deepfake detection relies on the premise that deepfake generators may fail to reproduce the vocal-tract characteristics of natural human speech. However, its robustness against modern synthesis models and adversarial manipulation remains insufficiently understood. We systematically evaluate vocal-tract detection against recent text-to-speech (TTS) and voice conversion (VC) models and show that its effectiveness degrades substantially on modern models. Through ablation studies, we identify two major contributors to vocal-tract realism: learned linguistic representations in TTS models and explicit modeling of source-speech fundamental frequency in VC models. We further introduce VoicePatch, a model-agnostic post-generation editing attack that evades detection by replacing detector-sensitive phoneme segments with real speech. Across three evaluated deepfake models, VoicePatch increases detection bypass rates from below 2% to above 80%.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Audio deepfake, deepfake detection

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RJzjOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper revisits vocal-tract-based audio deepfake detection and evaluates whether its underlying assumptions still hold for modern TTS and voice conversion models. The authors show that detection performance degrades substantially on several recent synthesis models, suggesting that modern generators can reproduce more realistic vocal-tract characteristics even without explicitly modeling the vocal tract. Through ablation studies, the paper attributes this improvement primarily to learned linguistic representations in TTS models and explicit F0 modeling in VC models. The paper further proposes VoicePatch, a post-generation attack that replaces detector-sensitive phoneme segments with acoustically matched real-speech segments. VoicePatch increases detection bypass rates from around 1% to over 80% even under surrogate/target detector mismatch.

Critical strengths
  • The evaluation and analysis are fairly comprehensive.
  • The proposed VoicePatch attack is interesting. It exploits the detector’s reliance on specific phonetic regions rather than treating the detector as a generic black box. Its effectiveness under feature-selection mismatch is particularly useful, as it shows that the attacker does not need to reproduce the deployed detector exactly.
Critical weaknesses
  1. The finding from the experiments is not surprising.

  2. Some causal claims from the ablation studies appear stronger than the evidence supports. The TTS experiments show that perturbing linguistic representations substantially affects the proposed vocal-tract fidelity metric, but this does not necessarily establish that linguistic representations themselves are the primary mechanism responsible for realistic vocal-tract characteristics. Perturbing these representations may also substantially change phonetic realization or other downstream acoustic properties. Similarly, the F0 experiments show that perturbing F0 increases variation in estimated vocal-tract features, but this is indirect evidence for the stronger claim that explicit F0 modeling explains why certain VC models are harder to detect.

  3. The conclusions are tied closely to one specific vocal-tract detector. Most of the paper’s conclusions are based on the detector of Blue et al. (2022). It is unclear whether the observed failure is a fundamental limitation of vocal-tract-based detection or primarily a limitation of this particular reconstruction and feature-selection pipeline.

  4. The practicality of VoicePatch depends on relatively favorable attacker assumptions. The attacker is assumed to have a large real-speech dataset and knowledge of the target detector design. More importantly, the attack requires finding real segments containing matching phoneme sequences with sufficiently similar timbre and prosody. The experiments use controlled speech corpora (TIMIT/VCTK), so it is unclear how well the method scales to more diverse speakers, languages, accents, or unconstrained real-world speech.

Questions for authors
  1. Can the authors provide stronger evidence that the linguistic representation and F0 findings are causal rather than consequences of broader changes to the generated speech caused by the perturbations?
  2. To what extent do the findings generalize beyond the specific Blue et al. detector? Have the authors considered evaluating another vocal-tract/physiology-based detection method or alternative vocal-tract estimation procedure?
  3. How sensitive is VoicePatch to the size and diversity of the real-speech database? In particular, what happens when suitable phoneme segments with similar timbre/prosody are scarce, as may occur for unseen speakers, accents, or languages?
  4. Given the noticeable MOS decrease for FreeVC and NeuralVC, could the authors provide human perceptual evaluation of VoicePatch, particularly focusing on whether listeners can detect splice boundaries or changes in speaker identity?
Reviewer_HJu8OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper revisits the vocal-tract reconstruction (VTR) deepfake detector of Blue et al. (2022) against modern zero-shot TTS (YourTTS, StyleTTS2, XTTS) and VC (PPG-VC, FreeVC, NeuralVC, HierSpeech++) models. The detector largely fails on modern models, with near-chance accuracy and high false-positive rates, though it remains effective on FreeVC and NeuralVC. Ablation studies identify the increased vocal-tract realism to learned linguistic representations in TTS and explicit F0 conditioning in VC. Finally, the paper introduces VoicePatch, a post-processing attack that involves swapping of segments identified by the detector with real speech pieces, matching their timbre and prosody, cross-fading the boundaries, and brings the bypass rate from near zero to over 90%.

Critical strengths
  1. Timely and convincing negative result: physics-based detectors are often assumed to generalize better than neural ones, and the paper shows VTR fails even in the detector-favorable setting of per-generator feature selection. I am not aware of prior work evaluating VTR against this generation of zero-shot models.

  2. Faithful reproduction of the original detector with a sound protocol; the feature-selection stability and temporal-coverage analysis (Sec. 3.1) explains why detection fails rather than only reporting numbers.

  3. VoicePatch is simple, cheap online, and robust to surrogate/target feature mismatch; the ethics statement is appropriate.

Critical weaknesses
  1. Both attribution ablations are confounded, and they are the paper’s main mechanistic contribution. Tube-model vocal-tract estimation is known to be F0-sensitive, so greater inter-frame variation after F0 perturbation cannot separate “the model synthesizes a more realistic tract” from “the estimator jitters with F0”; PPG-VC and FreeVC also differ in far more than F0 use. In the TTS ablation, randomized embeddings yield non-speech, and filtering vowel frames by wav2vec2 confidence introduces selection bias (surviving-frame counts are not reported). Neutralizing the speaker embedding with a corpus mean still yields a plausible average speaker, so the “speaker embedding has little effect” claim is weakly supported. Shin et al. (arXiv 2609.30983) provide the kind of fixed-vocoder control missing here and should be discussed.

  2. VoicePatch is a partial-fake/splicing attack, but the paper neither cites nor evaluates against that literature. PartialSpoof, Half-truth, ADD, Alali & Theodorakopoulos (2025), and SPADE (2026) study spliced partial fakes systematically, and SPADE’s splicing recipe is nearly identical to VoicePatch’s. Negroni et al. (ASVspoof 2024) further show splicing artifacts are detectable with a training-free threshold detector, yet VoicePatch is never tested against any partial-spoof, splicing-artifact, or neural detector; the attack is generator-agnostic but detector-specific.

  3. The “modern” scope is narrower than claimed, and an alternative explanation is untested. No codec-LM or flow-matching TTS is evaluated despite being the motivating examples, nearly all vocoders are GAN-based, and real speech is TIMIT only. Sec. 3.1’s own finding that selected features are rare, low-sample bigrams equally supports “threshold-based feature selection overfits” over “generators produce realistic tracts”; HierSpeech++’s below-chance accuracy points the same way and is not discussed.

Questions for authors
  1. Flatten or shuffle the F0 input to HierSpeech++/PPG-VC and retrain the detector: does accuracy recover? Please also report, per TTS ablation configuration, how many vowel frames survive the wav2vec2 filter. These directly test the two hypotheses in W1.
  2. What are VoicePatch’s bypass rates against (a) a partial-spoof detector, (b) a Negroni-style splicing-artifact detector, and (c) a mainstream neural detector (AASIST / XLS-R-based)? Please add external ASV similarity (ECAPA / WavLM-SV) and a small AB listening test.
  3. Is the failure due to generator realism or to overfitting of the threshold-based feature selection? Does a simple classifier (e.g., logistic regression on tube-area features) also fail on modern models? Please explain HierSpeech++’s below-chance accuracy.
Reviewer_fwRtOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper re-implements the vocal-tract (acoustic-tube) deepfake detector of Blue et al. (2022). The detector uses bigram tube-area features with thresholds selected for precision and recall ≥ 0.9, then a majority vote. The authors report 99.48% accuracy on TIMIT + Real-Time Voice Cloning and describe this as comparable to the original. They then fit per-generator detectors for three zero-shot TTS models and four VC models. Each detector is evaluated over five sessions; within each session, the feature-selection speakers (50) and evaluation speakers (250) are disjoint.

  • TTS. Accuracy falls to 52–64% for YourTTS, StyleTTS2 and XTTS, with FPR up to 59.8%.
  • VC. FreeVC and NeuralVC are still caught (~98.8%); PPG-VC (63.4%) and HierSpeech++ (45.1%) are not.
  • Feature-selection stability. For the modern TTS models, the selected bigrams are less stable across sessions and cover less of each utterance (32–35% vs 65% for RTVC).
  • TTS ablation. Linguistic embeddings are randomized (three schemes) and speaker embeddings neutralized. The effect is scored with a new vowel-frame "vocal-tract fidelity" metric.
  • VC ablation. F0 is independently perturbed at vowel frames in the two F0-conditioned VC models, and the authors measure the increase in within-vowel variation of reflection coefficients.
  • VoicePatch. The attack replaces detector-sensitive bigram segments (60% coverage) with phoneme-matched real speech from TIMIT/VCTK. Segments are ranked by FACodec timbre/prosody similarity and blended with 20 ms raised-cosine crossfades. The bypass rate rises from about 1% to 91–93%, and to 78–87% when the attacker's surrogate detector selects different bigrams. Quality is checked with UTMOS, generator speaker-encoder similarity and Whisper CER/WER.
Critical strengths
  • Per-generator detectors, five sessions with disjoint selection and evaluation speakers, and the stability/coverage analysis (Fig. 1–2) support the interpretation that the selected features generalize poorly. For the hard generators, detection rests on less frequent bigrams whose selection varies across speaker sets. The VC results are also informative because they are not simply "newer is harder": FreeVC and NeuralVC are still caught while PPG-VC and HierSpeech++ are not. This detector is often cited as hard to fool, so the result is worth having on record.
  • The attack is evaluated against detectors whose bigram selection differs from the attacker's surrogate (Table 5: 85.5/81.6/85.8% average bypass). The authors state that anonymized code is included.
Critical weaknesses
  1. The conclusions about "vocal-tract-based detection" rest on one detector and one decision rule, and the ablations do not isolate vocal-tract realism.

    • One detector, one rule. Only one implementation is tested, with one feature-selection rule (P, R ≥ 0.9 on 50 speakers, then a majority vote). HierSpeech++ reaches 45.12% accuracy, below chance. Together with the unstable feature selection, this raises doubts about the selection-and-thresholding procedure, but it does not tell us whether tube areas carry discriminative information. A stronger classifier on the same features (logistic regression or GBDT with speaker-disjoint CV, plus ROC/EER) would help separate the decision rule's limits from the features'.
    • No learned detector for reference. A learned detector (AASIST, wav2vec2/XLS-R + AASIST) run on the same generated audio would show how hard these generators are in general.
    • TTS ablation (Table 2). Randomizing linguistic embeddings can change phoneme recognizability, the accuracy of the wav2vec 2.0 labels, and which vowel intervals survive the filter (intervals of ≥ 2 frames with average confidence > 0.5, from which two central frames are compared with real frames of the same label). Lower fidelity could therefore reflect selection and labelling effects rather than less realistic tracts, and retention rates per condition are not reported. As written, the DD recurrence vn=βvn−1+1−β2wnv_n = \beta v_{n-1} + \sqrt{1-\beta^2} w_n also changes the coordinate means relative to D whenever μ≠0\mu \neq 0 and 0<β<10 < \beta < 1 (stationary mean μ1−β2/(1−β)\mu\sqrt{1-\beta^2}/(1-\beta)). Unless this was controlled, DD vs D is not a pure temporal-dependence test. The text's "3–12 percentage points" does not match Table 2, where LDD − LD ranges from 1.93 to 13.19.
    • F0 ablation (Table 4). The authors perturb F0 conditioning and measure more within-vowel variation in the estimated reflection coefficients. Tube/LPC estimates are known to depend on F0, because harmonics sample the spectral envelope. The experiment therefore cannot distinguish changes in the generated vocal tract from F0-dependent estimation effects. It does not show why explicit F0 conditioning raises the detector's difficulty. The causal test would compare the same model with and without F0 conditioning (or with constant F0) and report detection accuracy. As written, the hypothesis comes from comparing two architectures against two others that differ in many ways.
  2. VoicePatch lacks the controls needed to credit its design, and its quality cost is not measured in the way that matters.

    • Missing controls. The majority-vote rule gives a direct motivation for replacing detector-sensitive segments with real speech, and Table 9 already shows 87–90% bypass at 55% coverage. Segment coverage does not mathematically guarantee a passing majority, but it is unclear how much the method adds beyond inserting real speech where the detector looks. The missing controls are random-location replacement at the same budget, no FACodec ranking, and no smoothing.
    • Quality. UTMOS drops by 0.39 for FreeVC and NeuralVC (4.06 → 3.67, 4.05 → 3.66), which the text calls "negligible". Speaker similarity is measured only with each generator's own speaker encoder, and the relation between donor and target speakers is unspecified. An independent ASV model and a listening test are missing.
    • Other detectors. The spliced audio is never tested against a learned or partial-spoof detector, even though splice boundaries are a known cue. The partial-spoof literature (PartialSpoof, TASLP 2023; Half-Truth/HAD, Interspeech 2021) studies exactly this kind of mixed real/synthetic audio.
    • Data separation. TIMIT supplies both the detector's real-speech statistics and part of the donor bank. Whether those speakers or recordings are disjoint is not stated.

Smaller points:

  • The RTVC numbers disagree. Appendix A gives P/R 99.9/99.5, Table 1 gives 99.49/98.84. Unedited RTVC speaker similarity is 0.78 in Table 6 and 0.82 in Table 9.
  • Per-session standard deviations and real/fake evaluation counts are not reported; the session-averaged metrics do not establish class balance.
  • What sampling and resampling rates are used before tube-area estimation, given that N=2LF/CN = 2LF/C depends on FF?
  • Sec. 3.2 obtains phoneme intervals by grouping wav2vec 2.0 frame predictions. Is the same procedure used for detector feature extraction and VoicePatch matching, and how are bigrams built from those intervals?
  • Several citations are formatted as narrative citations where parenthetical ones are meant (e.g. "synthesis Stupp; Tian et al. (2020)"). Typos: "segmen" (Sec. 4.2), "ICLM" (references).
  • Related work should cover partial-spoof and splicing detection, and ASVspoof 5.
  • Blue et al. (§8.4) already report that features selected for one generator do not transfer to another (Lyrebird). That limitation should be discussed. This paper's finding is different and stronger: detection can stay poor even with generator-specific feature selection. Also, Blue et al. report 2.5% FPR on RTVC while Table 1 has 0.51%, so "comparable to the original" needs a word of explanation.
Questions for authors
  1. (Could change my score.) Train a stronger classifier on the same tube-area features with speaker-disjoint CV and report AUC/EER per generator. If AUC stays near 0.5 for XTTS and HierSpeech++, the paper's main claim becomes much stronger.
  2. (Could change my score.) How do AASIST or an SSL-based detector do on the same generated sets, and on the VoicePatch outputs?
  3. (Clarification.) What is the vowel-interval retention per condition in Table 2? Was the DD noise centred (and the original mean restored)?
  4. (Suggestion.) Run a constant-F0 or F0-free variant of PPG-VC/HierSpeech++ and report detection accuracy.
  5. (Clarification.) Are donor speakers and recordings disjoint from the detector's real-speech set? Appendix C motivates 60% with a coverage sweep; was that choice made on data separate from the final attack evaluation?
  6. (Suggestion.) Can you add matched-budget random replacement, an independent ASV model (e.g. ECAPA-TDNN) and a small listening test for patched speech?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.