acceptodds
Under review as a conference paper at ICLR 2027

Probabilistic Verification of Voice Anti-Spoofing Models

Abstract

Recent advances in generative models have amplified the risk of malicious misuse of speech synthesis technologies, enabling adversaries to impersonate target speakers and access sensitive resources. Although speech deepfake detection has progressed rapidly, most existing countermeasures lack formal robustness guarantees or fail to generalize to unseen generation techniques. We propose PV-VASM, a probabilistic framework for verifying the robustness of voice anti-spoofing models (VASMs). PV-VASM estimates the probability of misclassification under text-to-speech (TTS), voice cloning (VC), and parametric signal transformations. The approach is model-agnostic, builds on known concentration inequalities, and enables robustness verification against speech synthesis techniques and input perturbations. We improve existing high-confidence estimation methods with an adaptive selection mechanism without sacrificing their performance, and validate the framework across diverse experimental settings, demonstrating its effectiveness as a practical robustness verification tool.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
probabilistic verification, speech deepfake detection, voice anti-spoofing

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_PysHOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to assess the robustness of voice anti-spoofing models beyond accuracy. The authors propose PV-VASM, a model-agnostic framework that estimates how likely a detector is to fail under audio transformations, text-to-speech systems, and voice-cloning models. The framework combines several existing probabilistic bounds and introduces an adaptive method AdaVAU to select among them. The results show that verification is generally easier for simple audio transformations than for generated speech, while fine-tuning on a target generator can substantially improve robustness.

Critical strengths
  • The framework covers a much broader range of attacks than many existing verification methods. PV-VASM can also be applied to text-to-speech and voice-cloning systems, including generators that were not used during training.

  • Since PV-VASM treats the anti-spoofing model as a black box, it can in principle be used with different detector architectures. This makes the method easier to reuse and potentially more useful as an evaluation tool.

Critical weaknesses
  • One of my major concerns is that the evaluation is too narrow to support the broad claims of a general verification framework. Although PV-VASM is presented as model-agnostic, the experiments use only one anti-spoofing model. The paper indeed shows that the framework can work on one detector, but it remains unclear to other detectors with very different architectures or failure patterns.

  • PV-VASM estimates failure probability under a predefined distribution of transformations or generated samples. If that distribution does not match the attacks encountered in practice, the bound may not match/reflect real-world robustness meaningfully. This is important for voice attacks, where the space of possible generators, speakers, and settings is generally very large.

Questions for authors
  • How sensitive are the results to the choice of attack distribution? For example, if the distribution used for verification differs from the deployment, how should the guarantee be interpreted?

  • For the TTS and voice-cloning experiments, how much overlap is there between the generators used for verification and the data used to train the detector?

2 comments
Reviewer_PysHOpenReview ↗

No response from the authors

The authors chose not to include any response, so I will keep my original assessments.

AuthorsOpenReview ↗

Response to reviewer PysH

We thank the reviewer for their detailed feedback on our work. Below, we provide our answers to the weaknesses and questions raised by the reviewer. We also updated the text of the manuscript according to the provided answers (to include, where possible, additional experimental results and clarifications). We realize that the discussion phase itself is coming to an end, and posting our answers shortly before the deadline may feel inconvenient. However, we motivate it by the necessity of additional experiments and text rework. We would appreciate the reconsideration of the assessment of our work if the concerns of the reviewer are addressed sufficiently.

W1: We added verification of 2 more open-source models [Appendix A.2 Table 6]. We understand that evaluating the method across different models would strengthen the model-agnosticism claim, so we included it in the appendix.

W2: You are correct on that. Changing the parameter distribution directly impacts the upper bound of the expectation of r.v. Z, as it impacts the expectation itself.

We do not claim that the method can yield bounds that hold across multiple parameter distributions. Instead, we provide the framework that allows its user to choose an arbitrary parameter distribution that they find particularly applicable in their respective setting.

Q1: Please see the answer to W2.

Q2: TTS and VC models were present in the training set of the main model

Reviewer_PAniOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies probabilistic robustness verification for voice anti-spoofing models. The authors propose PV-VASM, a framework that estimates upper bounds on misclassification probabilities under audio transformations and speech generation methods, including TTS and voice cloning. The method builds on existing concentration inequalities and introduces AdaVAU to adaptively select among different bounds. Experiments on Wav2Vec2-AASIST evaluate the approach across various audio transformations and speech generators.

Critical strengths
  1. The paper addresses a relevant problem in speech deepfake detection. Probabilistic robustness verification could provide useful information beyond conventional empirical evaluation.

  2. I appreciate that the authors consider both conventional audio transformations and speech generation methods, including TTS and voice cloning. This makes the evaluation more relevant to practical anti-spoofing systems.

Critical weaknesses
  1. I find the connection between the proposed method and its main application somewhat weak. The TTS and voice cloning experiments use the existing CC-Cert method rather than AdaVAU. While these results demonstrate that probabilistic verification can be applied to speech generators, I am not convinced that they establish the value of the proposed method in these settings.

  2. I also find the methodological contribution is limited. AdaVAU mainly selects among existing probabilistic bounds using heuristic thresholds. Although this improves the results in some settings, I find it difficult to identify what substantial new understanding or capability it provides beyond existing verification methods.

Questions for authors
  1. Why is AdaVAU not evaluated in the TTS and voice cloning experiments? Does it offer any advantage over CC-Cert in these settings?

  2. Does the confidence guarantee remain valid when the parameter tt is selected using the same samples used to estimate the bound?

3 comments
AuthorsOpenReview ↗

Response to reviewer PAni

We thank the reviewer for their detailed feedback on our work. Below, we provide our answers to the weaknesses and questions raised by the reviewer. We also updated the text of the manuscript according to the provided answers (to include, where possible, additional experimental results and clarifications). We realise that the discussion phase itself is coming to an end, and posting our answers shortly before the deadline may feel inconvenient. However, we motivate it by the necessity of additional experiments and text rework. We would appreciate the reconsideration of the assessment of our work if the concerns of the reviewer are addressed sufficiently.

W1: We improved the connection of AdaVAU with the tts/vc robustness formulation by adding its performance in comparison with the CC-Cert method on robustness verification against tts/vc [See updated tables 2 & 3]

W2: Note that we do not just propose a heuristic selection rule for AdaVAU:

  1. We improve the CC-Cert bound by proposing an adaptive selection for its denominator [See Lemma 1 in Appendix B.1]
  2. We employ the Bernoulli-Cantelli concentration inequality to construct a probabilistic bound; noteworthy, we couldn’t find any method in the literature doing so. We added a theorem proving its correctness. [See theorem 1 in appendix B.2]
  3. We propose adaptive method selection to get better bounds while keeping the same confidence level. Q1. Please see the answer to W1.

Q2. We agree with the concern. Confidence guarantee does not hold in general:

  1. In the original CC-Cert, the bound holds only for t that is fixed in advance, so one can’t select different ones per batch.
  2. If you try to use t minimized on the same sample, after computing the bound, it would break the independence of Y’s, which is required in one of the steps of the proof. We fixed this mistake prior to all experiments by choosing this t on a portion of the sample (15%), and then running correct bounding on the rest of the sample. But we omitted it from the algorithms mistakenly. We now edited that algorithm, please[See algorithm 1 in appendix B.5]
Reviewer_PAniOpenReview ↗

Revised PDF missing

It seems that the authors have not submitted the revised PDF mentioned in their rebuttal. I cannot evaluate the rebuttal without the supporting evidence.

AuthorsOpenReview ↗

Response to reviewer PAni

We thank the reviewer for mentioning the the text of the manuscript is outdated. We update it now.

Reviewer_VBb6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This work proposes a probabilistic framework for verifying the robustness of binary speech anti-spoofing systems. This paper estimates upper bounds on the probability of misclassification under distributions of input transformations or generated speech samples. It combines three statistical verification approaches: CC-Cert, the Empirical Bernstein inequality, and the Bernoulli–Cantelli inequality; which are used to introduce an adaptive selection strategy intended to select an appropriate verification method based on preliminary observations.

Critical strengths

The paper addresses an important problem in speech anti-spoofing: evaluating not only the empirical performance of a detector, but also providing probabilistic guarantees on its error under data transformations or synthetic samples.

The paper considers three different statistical approaches to obtaining upper bounds on the probability of misclassification and proposes an adaptive strategy to select among them.

Critical weaknesses

The experimental evaluation relies on a single anti-spoofing model. This makes it difficult to determine whether the proposed verification framework is broadly applicable.

The dataset description does not sufficiently clarify the linguistic and speaker composition of the evaluation data. In particular, it is unclear whether the experiments use only English speech or include other languages. If multiple languages are included, the number of samples and speakers per language should be reported.

It is unclear whether the same transformation distributions and parameter ranges are used during training and robustness evaluation.

Comparing a model trained without augmentation vs. augmentation would help establish whether the proposed augmentations actually improve robustness under input transformations (Table 1). Additionally, evaluating transformations that are not used during training, such as standard MP3 compression, echo distortion, reverberation, or other operations, would provide a more convincing assessment of generalization to previously unseen perturbations.

Questions for authors

The abstract and introduction suggest that anti-spoofing systems fail to generalize to unseen speech-generation techniques. Could you please provide experimental evidence supporting this claim? Previous research has reported promising generalization to unseen synthetic speech generation methods:

Pizarro, M., Laszkiewicz, M., Kolossa, D., Fischer, A. (2026) Lightweight Detection and Model Attribution of Synthetic Speech via Residual Statistical Fingerprints. Proc. Interspeech 2026 [Long Track], 3071-3080, doi: 10.21437/Interspeech.2026-1361

Stan, A., Combei, D., Oneata, D., Cucu, H. (2025) TADA: Training-free Attribution and Out-of-Domain Detection of Audio Deepfakes. Proc. Interspeech 2025, 1543-1547, doi: 10.21437/Interspeech.2025-472

Müller, N., Diekmann, F., Williams, J. (2022) Attacker Attribution of Audio Deepfakes. Proc. Interspeech 2022, 2788-2792, doi: 10.21437/Interspeech.2022-129

Clarify whether the same transformations are used during training and robustness evaluation.

The results in Table 1 report substantially lower computational costs for AVU than for the individual verification methods in several settings. Since AVU is based on adaptively selecting among these methods, could you please explain which computations are performed during the pilot stage and after method selection?

Why are the generated TTS and VC samples not also evaluated under the input transformations considered in the first experiment?

1 comment
AuthorsOpenReview ↗

Response to reviewer VBb6

We thank the reviewer for their detailed feedback on our work. Below, we provide our answers to the weaknesses and questions raised by the reviewer. We also updated the text of the manuscript according to the provided answers (to include, where possible, additional experimental results and clarifications).

We realize that the discussion phase itself is coming to an end, and posting our answers shortly before the deadline may feel inconvenient. However, we motivate it by the necessity of additional experiments and text rework. We would appreciate the reconsideration of the assessment of our work if the concerns of the reviewer are addressed sufficiently.

Q1: One can see experiments evidence supporting claim about generalization problem in second paragraph of related works section (Section 2). Still, we agree that a lot of research is done in this field, so we also added suggested references, as they show relevance of this problem.

Q2: Augmentations used during training are listed in subsection “Datasets and augmentation” of section 4, below the Table 1. For some augmentations, the parameter ranges were wider in robustness verification than in training: semitones for pitch shift in training were coming from [-5, +5], highpass cutoff frequency in training was coming from [100, 400]. For other augmentations, the parameters were either matched in training and robustness verification, or ranges in robustness verification were narrower: this is empirically validated by the column (p) in Table 1: Those with wider parameters have higher empirical error rate. The table with the parameters are included in the appendix section B.3.

Q3: Cost is a crucial parameter for comparison of different concentration inequalities: when fixing a group of methods, cost of a given method for a given audio x is defined as the difference of bound of this method and minimum bound of all methods. We report this metric averaged across all test audios. Q4: We assess robustness to TTS and VC as a distribution, so it is a notably different setup. Please refer to Section 3.4, where it is explained in detail. W1. We added verification of 2 more open-source models [Appendix A.2 Table 6]. We understand that evaluation of the method on different models would strengthen the model-agnosticism claim, so we included it in the appendix. W2. For the test set, we used a subset of 300 audios from the ASVspoof5 evaluation set as it serves as a hard challenge. The language used is English.

W3. Please see answer to Q2.

W4. We added verification results across 5 different augmentations both for the model that was trained without augmentations, and for the model that had those augmentation in it’s training. Other training parameters were the same. [Appendix A.3, Table 7]

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.