acceptodds
Withdrawn from ICLR 2027

On the Vulnerability of Multilingual Speech Recognition to Adversarial and Backdoor Attacks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multilingual automatic speech recognition (ASR) models have achieved remarkable performance, yet their robustness under adversarial and backdoor attacks remains insufficiently explored, especially for low-resource languages. This paper presents a systematic empirical study across seven languages spanning high, medium, and low resource levels, using state-of-the-art models including Whisper (small, base, medium), wav2vec2-XLSR, and omniASR-CTC. We evaluate untargeted adversarial attacks, preprocessing defenses, and imperceptible backdoor injection with varying poisoning ratios, trigger types, and model scales. Our key findings include: (1) Low-resource languages exhibit significantly larger degradation under adversarial perturbations, with Hausa showing an increase in word error rate (WER) from 41.3% to 222.3% at , while English degrades from 10.2% to 112.6%; (2) simple low-pass filtering reduces adversarial WER by up to 71% for high-resource languages but is less effective for low-resource ones; (3) backdoor attacks require much lower poisoning ratios to succeed on low-resource languages—0.3% poisoning yields 87% attack success rate (ASR) for Hausa versus 0.5% for 71% ASR in English; (4) cross-lingual backdoor transfer is weak, with ASR dropping from 98.8% to 14.3% when transferring from English to Swahili; (5) adversarial training provides meaningful robustness gains for low-resource languages, reducing adversarial WER by 23.5% for Swahili and 12.0% for Hausa with minor clean accuracy loss. These results highlight disproportionate security risks for low-resource language communities and provide insights for building more robust multilingual ASR systems.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Adversarial Attacks, Backdoor Attacks, Multilingual ASR, Speech Recognition, Low-Resource Languages, Robustness, Whisper, wav2vec2-XLSR

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_FAxnOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents an empirical evaluation of adversarial attacks on multilingual speech recognition. Its main contribution is characterizing language-specific adversarial attacks. The results suggest that the low-resource languages suffer greater WER growth and are easier to attack. This paper also presents several defense methods and shows the trade-offs between benefits and accuracy.

Critical strengths
  1. This paper presents an innovative study on language-specific adversarial attacks.
  2. This paper proposes several defense mechanisms and demonstrates the trade-offs between benefits and accuracy in detail.
Critical weaknesses
  1. The perceptual evidence for trigger signals is based on PESQ values rather than human evaluation.
  2. In Section 4.1, the experimental results show that Hausa is more vulnerable to English, but since the two languages have different clean WERs, the conclusions drawn from this comparison are not entirely convincing.
  3. Table 3 shows that larger models tend to have slightly lower ASR. However, the experimental data is limited and the ASR for English does not increase monotonically.
  4. This paper concludes that low-resource languages are more vulnerable, but experiments were conducted on only two such languages.
Questions for authors
  1. Are trigger signals with low PESQ values also difficult for humans to hear? It is better to conduct human evaluations.
  2. Is it possible to conduct a comparative experiment in which the clean WER for the two languages in Section 4.1 is the same?
Reviewer_3r5YOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This is an empirical paper. It does not propose a new method or module; its contribution is an analysis of adversarial and backdoor attacks on automatic speech recognition (ASR). The authors find that multilingual ASR models are more vulnerable under both threat scenarios in low-resource languages and more robust in high-resource languages.

Critical strengths
  1. The experiments are systematic, covering seven languages and three multilingual ASR model families.
  2. The study is broad: it covers several attacks, triggers, poisoning ratios, model scales, and defenses.
Critical weaknesses
  1. The paper does not propose a new attack, defense, or method; adversarial and backdoor vulnerabilities of ASR are well established. Its contribution therefore rests entirely on the empirical finding that low-resource languages are more vulnerable. However, this finding is only partially new (e.g., [1]) and is not convincingly supported (see points 2, 3, and 11 on the noise control, metric choice, and test-set size). As a result, the paper’s significance for the ICLR main track is limited.

  2. Low-resource languages already have much higher clean WER, so their larger degradation under PGD may simply reflect weaker overall robustness rather than a specific adversarial vulnerability. To separate the two, the authors should report a random-noise control (e.g., Gaussian noise at the same SNR as each ε) for all languages. If the gap between low- and high-resource languages is similar under random noise, the finding is about general noise robustness, not adversarial robustness. The natural-noise experiment in the appendix is incomplete, and it undercuts the adversarial story. Table 12 compares Gaussian, background, and adversarial noise, but only for Hausa and Swahili. Without high-resource languages, it cannot show whether the low-resource gap reflects adversarial vulnerability or general noise sensitivity. Moreover, Swahili is far more sensitive to Gaussian noise than Hausa (153.6% vs. 97.3% WER at 10 dB), yet less adversarially vulnerable (136% vs. 222%). The authors should also add the clean WER (no attack, no defense) to Figures 2 and 3, so the comparison is easy to make rather than buried in the appendix.

  3. The headline adversarial claim depends on the choice of metric. The paper states that “Hausa WER rises from 41.3% to 222.3% (+181.0), while English only increases from 10.0% to 112.6% (+102.6).” In relative terms, however, English WER increases about 11.3× (+1026%), while Hausa WER increases only about 5.4× (+438%). Under this metric, the low-resource languages appear more robust.

  4. In Table 2, adversarial WER remains above 100% even after adversarial training. WER exceeds 100% because insertions are counted as errors, so the number of errors is larger than the number of reference words. In my opinion, both 136.5% (before) and 104.6% (after) for Swahili are unacceptable, and the statement “This suggests that even limited adversarial training can improve robustness for low-resource languages” is overstated. I agree there is a small gain, but in practice the model remains unusable under attack.

  5. What exactly is the re-training defense? Re-training on what — a clean version of the dataset? In the poisoning setup, the defender does not have access to such a clean set, so how are the clean samples identified?

  6. For a study presented as systematic, more ASR models should be evaluated. The adversarial part uses only a single model family (Whisper small).

  7. Only a simple PGD attack is evaluated. A complete study should include stronger attacks, such as an adapted version of AutoAttack [2].

  8. Unsupported contribution: the first contribution bullet claims “cross-model comparisons on representative CTC-based systems” for adversarial robustness, but the main text contains no adversarial results on wav2vec2 or omniASR, only backdoor results.

  9. The attack combines a step size of α = 0.02 with the Adam optimizer, which is unusual for PGD. Please clarify how Adam is used.

  10. The adversarial training details are missing: ε, number of attack steps, number of epochs, etc.

  11. What is the test set? According to the caption of Table 8, the adversarial results are computed on only 30 utterances per language. This is too few to support a cross-language ranking.

  12. Several reported statistics appear internally inconsistent:

(a) WERs of 110–245% are reported with a standard deviation of only about ±1. In my experience, per-utterance WER varies far more than this across 30 utterances. Am I missing something? Please clarify what the standard deviation is computed over (utterances, seeds, or runs) and, if over seeds, how many.

(b) The same means are reported with different standard deviations across tables. Hausa adversarial WER at ε = 0.1 is 222.34 with ±0.94 (Table 1), ±0.59 (Table 2), ±1.03 (Table 8), and ±1.09 (Table 9). Swahili shows the same pattern (136.45 with ±0.88, ±1.17, ±0.89, and ±1.01). Why do identical means appear with different standard deviations?

(c) Clean WER varies across the ε rows of Table 8 for the same 30 utterances (EN: 10.18 / 9.96 / 10.21, each with a different standard deviation). Since the attack does not affect clean inputs, these values should be identical. Why do they differ?

(d) Zero-shot WER varies across the poisoning-ratio rows of Table 11 (HA: 98.38–98.52, each with a different standard deviation). Zero-shot WER is measured before fine-tuning, so it cannot depend on the poisoning ratio. Why does it vary?

References

[1] R. Olivier and B. Raj. There is more than one kind of robustness: Fooling Whisper with adversarial examples. Interspeech, 2023. [2] F. Croce and M. Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks. ICML, 2020.

Questions for authors

See weaknesses section!

Reviewer_Pbs3OpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the vulnerability of multilingual automatic speech recognition (ASR) models to adversarial and backdoor attacks, focusing on differences across language resource levels. Using several languages from FLEURS, the authors evaluate untargeted adversarial attacks, backdoor attacks, and several defenses. The experiments examine perturbation strength, poisoning ratio, trigger design, model scale, and cross-lingual backdoor transfer. The authors report that, under the evaluated settings, low-resource languages generally exhibit greater adversarial degradation and achieve higher backdoor attack success rates at lower poisoning ratios.

Critical strengths
  • The research question addresses an evaluation gap. Existing attack studies often provide limited coverage of languages and resource levels. Evaluating multilingual ASR security across seven languages helps broaden our understanding of risks faced by underrepresented language communities.

  • The evaluation considers multiple relevant dimensions. The paper examines both adversarial and backdoor attacks, together with several defenses, model scales, trigger configurations, and cross-lingual transfer settings.

Critical weaknesses
  • The related-work discussion is insufficient to distinguish this study from existing work.

ASR adversarial attacks have a substantial literature, but Section 2 provides only brief descriptions of a few representative studies. Although additional papers are cited, their methods, settings, and findings receive little substantive discussion. More importantly, the authors explicitly acknowledge existing work on multilingual adversarial perturbations and audio backdoors, yet largely group these studies into a single sentence without further details.

Consequently, it is difficult to determine precisely what previous multilingual studies have already established and what remains unresolved. The paper should compare the closest studies in terms of multilingual model coverage, evaluated languages and resource levels, threat models, and research questions. This would help distinguish the paper’s unique contribution more clearly.

  • The distinctive contribution and depth of the empirical findings need further development.

Extending security evaluation to multilingual ASR is worthwhile. However, the current study primarily appears to apply established attacks and defenses to additional languages and models. Its findings, such as greater vulnerability in low-resource languages and partial mitigation through standard defenses, are broadly consistent with expectations.

For an empirical study, a stronger contribution could come from identifying distinctive technical or methodological challenges in the multilingual setting, uncovering less intuitive findings, or deriving concrete implications for future research. Section 5 mentions weaker representations, tokenization, and acoustic coverage as possible explanations, but does not investigate their relevance sufficiently to provide such insight. Also, general recommendations to use data augmentation or adversarial training do not yet fully demonstrate the value of the reported findings.

Some observations, such as weak cross-lingual backdoor transfer and strong trigger-position dependence, could support a more informative analysis. The authors should consider strengthen these findings and make them clearer.

Questions for authors
  1. Positioning against prior work: What are the closest existing studies on multilingual ASR attacks and audio backdoors, and which research questions or findings in this paper are not covered by them? Could the authors provide a substantive comparison beyond differences in the number of evaluated languages or models?

  2. Distinctive insights and implications: Does the multilingual setting introduce technical or methodological challenges beyond applying existing attacks independently to each language? Which findings provide the strongest new insight, and what specific recommendations for future evaluation or defense development follow from them? A focused analysis of one or two such findings would help establish the contribution.

  3. Consistency of the defense comparison: The contributions state that adversarial training is more effective for low-resource languages, but Tables 2 and 9 show lower adversarial WER with low-pass filtering for both Swahili (99.23% versus 104.56%) and Hausa (168.45% versus 195.67%). Are these results obtained under comparable protocols? If so, the corresponding conclusion should be revised.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.