acceptodds
Withdrawn from ICLR 2027

Never Stop Speaking: a Denial-of-Service Attack on End-to-End Speech Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Many studies have shown that specially crafted inputs can induce large language models (LLMs) to generate excessively long outputs, resulting in significant computational overhead and resource consumption. While most existing denial-of-service (DoS) attacks target text-only LLMs, end-to-end (E2E) speech LLMs are rapidly emerging. Existing text-based DoS attacks primarily rely on prompt engineering, such as adversarial suffixes or semantic inducement, which exploit the discrete nature of text inputs and therefore cannot be directly transferred to continuous speech inputs. Moreover, prior studies on speech model security mainly focus on ASR or TTS systems, leaving the DoS vulnerability of E2E speech LLMs largely unexplored. To address this gap, we propose the perturbation-based DoS attack targeting E2E speech models. Instead of inducing long outputs through prompt manipulation, our method optimizes imperceptible acoustic perturbations to directly influence the model's autoregressive generation process while preserving the original input length. Specifically, we formulate the attack as a composite optimization objective that jointly suppresses EOS generation, encourages prolonged decoding, and preserves semantic consistency by integrating weighted EOS loss, top-k logit loss, length loss, and semantic alignment loss. To further improve stealthiness, we employ voice activity detection (VAD) to inject perturbations only into voiced regions. Extensive experiments on three open-source E2E speech LLMs demonstrate that our method achieves stable attack success rate while significantly increasing generation length and GPU resource consumption, revealing security risks in modern ALLMs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Speech Language Models, Denial-of-Service Attack, Generative AI Safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_ZiD9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates denial-of-service (DoS) attacks against end-to-end speech language models by optimizing acoustic perturbations to induce prolonged autoregressive generation while preserving the original input length. The proposed white-box attack uses projected gradient descent with a composite objective comprising weighted EOS loss, top-k logit loss, length loss, and semantic alignment loss. Voice activity detection (VAD) further restricts perturbations to voiced regions. Experiments on three open-source audio language models demonstrate the effectiveness of the proposed attack.

Critical strengths
  1. The research problem is meaningful. Studying DoS attacks on End-to-End speech language models contributes to highlighting practical vulnerabilities in ALLM systems.
  2. The methodology is straightforward and easy to understand. The attack constructs adversarial perturbations through a multi-objective loss designed to prolong generation and enable DoS attacks on ALLMs.
  3. The paper is generally well-organized and easy to read.
Critical weaknesses
  1. The novelty of the method is limited. The authors simply adapt audio adversarial perturbation attacks for ALLM DoS. As audio adversarial attacks have been widely studied, I could not find what makes the proposed method advanced over existing adversarial attack frameworks.
  2. The discussion of related works is insufficient. Though prior adversarial attacks works on ASR, TTS and ALLMs typically aim at misprediction or jailbreaking rather than DoS, they share relevant adversarial perturbation mechanisms. These related works should be discussed to justify the contribution.
  3. The imperceptibility claim is not well supported. The paper claims its perturbations are imperceptible, yet I could not find SNR, perceptual metrics or listening tests to back this statement.
  4. Section 4.5 only tests against additive random noise, which provides a narrow assessment of attack robustness. I would like to see evaluations across common audio processing operations, such as denoising, speech enhancement, resampling and compression for a more realistic evaluation.
  5. The paper evaluates three models under the same stated experimental setup, yet reports only two in the main results and places Qwen2-Audio in Appendix C. It is not clear to me why the authors chose this presentation layout.
  6. Figures 2 and 3 are blurry and contain small text that is difficult to read.
Questions for authors
  1. What technical challenges distinguish DoS attacks on ALLMs from existing audio adversarial attacks, and could the authors clarify how their method address these beyond multi-objective optimization?
  2. Can the authors provide evaluations of perturbation imperceptibility, such as SNR, PESQ, or listening tests, to assess the trade-off between perturbation magnitude and attack performance?
Reviewer_k7KFOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates denial-of-service (DoS) attacks against end-to-end speech language models. The authors propose a white-box adversarial attack that combines multiple loss functions to suppress EOS generation and induce excessively long outputs. The method also employs VAD to restrict perturbations to voiced regions of the input audio. Experiments on three audio-language models, together with ablation studies and comparisons against existing baselines, demonstrate that the proposed method can substantially increase output length and GPU memory consumption during inference.

Critical strengths
  1. The paper extends adversarial DoS attacks to end-to-end speech language models, addressing a relatively underexplored security problem.

  2. Experiments across multiple models demonstrate substantial increases in generation length and GPU memory consumption, show that the proposed method works.

Critical weaknesses
  1. Although the attack appears effective, the effectiveness largely builds on EOS suppression, and provides limited new insight into the underlying vulnerability of end-to-end speech language models. I buy that the method works, but I find it difficult to identify what new understanding this work brings beyond existing DoS attacks.

  2. The practical impact of the attack is not sufficiently evaluated. The experiments show increased generation length and GPU memory usage, but I would like to see how these translate into actual service degradation, such as increased latency or reduced throughput. At the very least, a more concrete discussion of the practical implications would strengthen the paper.

  3. I also have concerns about the formulation of the loss functions in Section 3.3. The EOS probability is computed by applying a sigmoid to the EOS logit, rather than using the token probability from the softmax distribution. I am concerned that these issues may affect the validity of the proposed length loss.

Questions for authors
  1. The formulation of the length loss in Section 3.3 seems questionable. The EOS probability is estimated using sigmoid rather than softmax. I would appreciate clarification on whether these issues affect the implementation or experimental results.

  2. Given that simple EOS suppression already achieves relatively high attack success rates, could the authors clarify the necessity of the additional loss terms and their contributions beyond the existing approach?

Reviewer_SNJVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates denial-of-service (DoS) attacks against end-to-end speech language models by optimizing small acoustic perturbations to induce excessively long generation. The authors propose a joint optimization framework combining dynamically weighted EOS loss, Top-k logit loss, length loss, and semantic alignment loss, with VAD used to restrict perturbations to voiced regions. Experiments demonstrate improvements over existing baselines across multiple models, while ablation studies and robustness evaluations further support the effectiveness of the proposed method.

Critical strengths
  1. This paper addresses an underexplored DoS security problem in end-to-end speech language models, making it a valuable research contribution. Unlike existing attacks that rely on discrete text manipulation, the proposed approach directly constructs adversarial perturbations in the continuous audio space, providing a new direction for evaluating the security of speech language models.
  2. The method offers a degree of novelty by integrating termination control, length optimization, and semantic preservation into a unified framework. The experimental evaluation is relatively comprehensive, covering baseline comparisons, component ablations, different decoding strategies, and robustness analyses, providing solid empirical support for the main findings.
Critical weaknesses
  1. The independent contribution of the dynamically weighted EOS loss remains unclear. Simple Loss already achieves attack success rates of 77%–79% in Table 1, while the full method improves these results by only 7–8 percentage points. The paper does not directly compare standard and dynamically weighted EOS losses while keeping the other three loss terms unchanged. Therefore, it remains unclear how much of the improvement is attributable to dynamic weighting rather than the additional loss terms.
  2. Equation (6) defines the EOS termination probability as pt=σ(zt)p_t=\sigma(z_t), where ztz_t is the EOS logit. However, in standard autoregressive language models, token probabilities are obtained through a softmax over the vocabulary, not by applying a sigmoid to an individual logit. These formulations are not equivalent, and the paper does not clarify whether Equation (6) reflects the actual implementation. Since the length loss depends directly on this probability estimate, this ambiguity affects the interpretation of the optimization objective.
  3. The authors claim that the adversarial perturbations are imperceptible, but the evaluation does not report objective audio quality metrics such as PESQ or STOI, or human perceptual tests such as MOS or ABX. Using ChatGPT 5.5 to evaluate the attack cannot directly substitute for human perceptual evaluation of whether the perturbations are audible. Therefore, the experiments do not directly substantiate the claimed imperceptibility of the perturbations.
Questions for authors
  1. How robust is the proposed attack to common audio front-end processing in real-world speech systems, such as speech enhancement, denoising, compression, and resampling, as well as over-the-air playback and recording?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.