- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies whether lossy audio compression can serve as a reliable defense against targeted attacks on Audio LLMs. It proposes CodecAttack, which optimizes a bounded perturbation in the continuous latent space of EnCodec and trains against a multi-bitrate Opus channel using EoT and a straight-through estimator. The authors compare this method with a codec-aware waveform baseline on three Audio LLMs and four constructed application scenarios. They report substantially higher targeted string-matching ASR for the latent attack, robustness to held-out MP3 delivery, limited transfer to held-out models in a separate higher-budget experiment, and reduced success after neural re-synthesis.
Critical strengths
-
By optimizing perturbations in a neural audio codec’s latent space with multi-bitrate training, the attack reaches an average targeted ASR of 83.3% across three Audio LLMs and four scenarios, compared with 37.2% for the waveform baseline using the same optimizer, EoT configuration, and optimization step budget.
-
Besides Opus, which is used during optimization, the authors evaluate held-out MP3 and AAC-LC delivery, showing both the attack’s robustness across codecs and its limitations.
-
The authors analyze the decoder’s spectral bias and conduct band-limited waveform control experiments, finding that restricting perturbation frequencies alone does not reproduce the latent-space attack’s effectiveness.
-
The authors examine how neural audio re-synthesis reduces attack success and demonstrate nonzero transfer to models excluded from optimization, providing useful directions for further research.
Critical weaknesses
-
The evaluation provides useful proof-of-concept evidence, but the number of independent carriers is limited. S1 uses 50 speech carriers and S2 uses 25, for S3a and S3b, the reported 40 and 45 carrier-target pairs are derived from only 8 and 9 recordings, respectively, so treating them as fully independent examples may overstate statistical confidence.
-
The finance, interview, AI-content, and copyright settings are motivated by real applications, but the experiments run on three Audio LLMs with author-constructed carriers and target strings. Citations to products such as Retell AI, Fluid AI, HireVue, HeyMilo, Spotify, or YouTube provide context, not evidence that those products were tested.
-
The attacker needs the model’s weights and gradients to construct the audio, which may not be available for commercial systems. When tested on models not used to construct the attack, success rates are much lower despite allowing larger perturbations, so the results provide limited evidence of effectiveness against unknown models.
Questions for authors
-
Throughout the paper, the authors optimize audio perturbations to induce a target output, without showing that the model follows instructions embedded in the audio or overrides a trusted instruction. This is more naturally described as a targeted adversarial audio attack than prompt injection.
-
What percentage of successful target-string matches would also cause a downstream tool call or task-level decision in a realistic Audio-LLM agent? Can the authors provide a controlled tool-execution evaluation with explicit authorization checks?
-
Could you clarify the sources of audio used in experiments? Could the authors report the content distribution in each scenario and examine whether ASR varies across content categories under otherwise matched conditions?
-
Can the authors report results for commercial or API-only Audio LLMs, even if only through a limited black-box query budget? This would substantially improve the practical relevance of the claims.
4 comments
Question 1. We agree that our current experiments primarily demonstrate targeted adversarial audio attacks, where optimized perturbations induce attacker-specified outputs, rather than explicitly demonstrating that a model follows malicious instructions or overrides higher-priority instructions.
Our use of the term "injection" was intended to describe the introduction of attacker-controlled output behavior through the audio input, rather than to claim that we have established conventional prompt injection or instruction-hierarchy violations.
Question 2. To evaluate whether adversarial audio can induce downstream actions beyond target-string generation, we are conducting tool-calling experiments on Gemma 4, an audio-capable model with tool-calling support.
We evaluate two attack settings: (1) white-box attacks optimized directly against Gemma 4 and (2) transfer attacks using adversarial audio previously optimized against Qwen2.5-Omni, Audio Flamingo 3, and Phi-4 at ε=1, without using Gemma 4 during attack construction.
Both settings use an offline robot simulator with four available tools. We measure whether adversarial audio triggers the intended tool call with valid arguments under uncompressed and Opus 64 kbps delivery. We will report the results once both evaluations are complete and validated.
Reference
[1] Team, Gemma, et al. "Gemma 4 technical report." arXiv preprint arXiv:2607.02770 (2026).
Question 3
Thank you for the suggestion. We have clarified the audio sources and content distribution across our original experimental scenarios.
Table: Audio Sources and Content Distribution in the Original Evaluation.
| Scenario | Audio Source | Recordings | Carrier-target pairs |
|---|---|---|---|
| Banking (S1) | Edge TTS | 50 | 50 |
| Interview (S2) | Edge TTS | 25 | 25 |
| AI Music (S3a) | Suno AI | 8 | 40 |
| Copyright (S3b) | Official original songs | 9 | 45 |
| Total | 92 | 160 |
The banking and interview scenarios contain five distinct scripts each, synthesized using ten and five voices, respectively. Each music recording is paired with five target responses.
As described in our response to Weakness 1, we generated an additional 150 distinct recordings across banking, interview, and AI-music scenarios, using Qwen3-TTS for speech and ACE-Step/YuE2 for music.
We have completed the Qwen2-Audio evaluation on 120 held-out recordings (40 per scenario) at ε=1, using three optimization seeds and matched optimization and compression settings (Table 2 from response to Reviewer 6hsC).
The results show that attack effectiveness varies across scenarios, with the highest ASR observed on AI-generated music. Because the scenarios also differ in task prompts and generation sources, these differences cannot be attributed solely to audio content.
We are further extending the evaluation to Qwen2.5-Omni and Audio Flamingo 3, covering the same audio codecs and bitrate configurations evaluated in the original paper, with three independent optimization seeds per recording. We will report the complete results as soon as they are validated.
Follow-up: Results from Additional Experiments
Following our initial response, we conducted additional experiments to investigate whether CodecAttack can induce structured tool calls beyond target-string reproduction, whether these calls lead to downstream tool execution, and whether adversarial audio remains effective in a commercial voice-agent environment.
Question 2: Downstream Tool-Calling Evaluation
To investigate downstream agent behavior, we evaluated CodecAttack on Gemma 4 in a controlled tool-calling environment.
Experimental setup. We conducted optimization and evaluation across three independent seeds (1, 7, 42), varying the number of available tools from 1 to 128. We evaluated the resulting adversarial audio under Opus 64/128 kbps, MP3 64 kbps, and AAC 64 kbps compression.
Unlike the original target-string evaluation, this experiment measures exact-call attack success rate (ASR), requiring the model to generate the attacker-specified structured tool call rather than merely reproduce a target string in its natural-language response.
Table 1. Exact-Call Attack Success Rate (ASR, %) Across Tool-Catalog Sizes and Lossy Audio Codecs. Values Are Mean ± Standard Deviation Across Three Optimization Seeds (1, 7, 42).
| Available tools (k) | Opus 64 | Opus 128 | MP3 64 | AAC 64 |
|---|---|---|---|---|
| 1 | 100.00 ± 0.00 | 100.00 ± 0.00 | 100.00 ± 0.00 | 100.00 ± 0.00 |
| 4 | 58.50 ± 2.29 | 57.33 ± 1.61 | 57.83 ± 1.26 | 57.17 ± 2.02 |
| 8 | 47.50 ± 2.18 | 46.00 ± 1.80 | 48.00 ± 3.28 | 47.50 ± 0.87 |
| 16 | 39.00 ± 3.28 | 39.67 ± 3.55 | 39.00 ± 3.28 | 39.83 ± 1.15 |
| 32 | 24.50 ± 5.27 | 24.33 ± 2.52 | 24.17 ± 2.36 | 23.67 ± 1.44 |
| 64 | 11.67 ± 3.06 | 12.00 ± 1.00 | 11.33 ± 0.58 | 13.00 ± 0.50 |
| 128 | 3.83 ± 1.15 | 2.67 ± 1.04 | 3.00 ± 0.00 | 3.17 ± 0.29 |
The results demonstrate that CodecAttack can induce attacker-specified structured tool calls that remain effective after lossy audio compression. With four available tools, the attack achieves 58.50% ASR under Opus 64 kbps, 57.33% under Opus 128 kbps, 57.83% under MP3 64 kbps, and 57.17% under AAC 64 kbps.
Attack success decreases as the number of available tools increases, indicating that targeted tool selection becomes more difficult as the available action space expands.
Additional downstream tool-execution evaluation. To further assess whether attacker-induced tool calls lead to actual execution, we evaluated ε=1 adversarial audio on the held-out Gemma-4-12B-it model in a sandboxed robot-control environment with explicit authorization controls. The experiment used three optimization seeds (1, 7, 42) and compared adversarial audio with the corresponding unperturbed carrier under the same authenticated-session condition (L1T).
CodecAttack achieved 25.2 ± 22.3% protected-tool execution, compared with 2.2% for the unperturbed carrier, representing an observed increase of 23.0 percentage points. These results extend the exact-call evaluation by demonstrating that adversarial audio can induce actual downstream tool execution rather than only attacker-specified structured outputs. The observed executions occurred in an authenticated session and therefore do not establish authorization bypass.
Weakness 2: Preliminary Commercial Voice-Agent Evaluation (Retell AI)
To further assess practical applicability, we conducted a preliminary evaluation using Retell AI with gpt-realtime-mini-2.1 as the backbone model.
We evaluated 15 adversarial audio samples generated through ensemble optimization against Qwen2.5-Omni-7B, Audio Flamingo 3, and Phi-4, without directly optimizing against the deployed target model.
Table 2. Preliminary Target-String Attack Success Rate on Retell AI.
| Platform | Target model | Successful attacks | Target-string ASR |
|---|---|---|---|
| Retell AI | gpt-realtime-mini-2.1 | 5/15 | 33.3% |
Of the 15 evaluated samples, 5 successfully induced the attacker-specified target response, corresponding to a target-string ASR of 33.3%.
The evaluation was limited to 15 samples due to the usage constraints of Retell AI's free tier. Accordingly, we present these results as a preliminary feasibility study rather than a comprehensive assessment of the platform's robustness.
Despite the limited sample size, these results provide initial evidence that adversarial audio optimized against open-weight surrogate models can retain some effectiveness when evaluated through a commercial voice-agent platform. A larger-scale evaluation is necessary to establish the reliability and generalizability of this finding.
Thank you again for your review and suggestions. We have posted our responses and additional experimental results addressing the concerns you raised.
We understand that this is a busy time for reviewers, but wanted to follow up as the discussion period is coming to an end. We would appreciate it if you could take another look at our responses when you have a chance, and please let us know if there are any remaining questions or concerns.
Thank you again for your time and consideration.
Thank you for your detailed review and constructive comments. The following are our initial responses to your questions and concerns.
Weakness 1. We agree that carrier-target pairs and optimization seeds should not be treated as independent recordings. Our original evaluation contains 92 distinct recordings and 160 carrier-target pairs, and the reported three-seed statistics capture optimization variability rather than uncertainty across independent audio content.
To address this concern, we have generated 150 additional distinct audio recordings, separate from the original evaluation cohort, with 120 recordings reserved for held-out evaluation (40 per scenario). We have completed the Qwen2-Audio evaluation at ε=1 across three optimization seeds, achieving 81.4% ASR on uncompressed audio and 67.5% under Opus 64 kbps compression.
We will report uncertainty estimates clustered at the recording level, rather than treating repeated optimization seeds as independent samples. The detailed content distribution and scenario-specific ASR results are provided in our response to Question 3 below.
Weakness 2. Thank you for highlighting the importance of real-world evaluation. The commercial platforms mentioned in our paper serve as motivating application scenarios, while our experiments investigate codec-robust adversarial perturbations against Audio LLMs under controlled and reproducible conditions.
This evaluation approach is consistent with prior audio adversarial research. For example, Schönherr et al. (NDSS 2019) [1] investigated adversarial audio against Kaldi while discussing potential attacks on commercial voice assistants. Similarly, Carlini and Wagner (2018) [2] evaluated targeted audio attacks against DeepSpeech, and Ma et al. (EMNLP 2025) [3] studied adversarial vulnerabilities in open-weight Speech LLMs. These works demonstrate the value of studying underlying model vulnerabilities independently of end-to-end commercial deployment.
However, to further assess practical applicability, we are planning additional evaluations on commercial voice-agent platforms, particularly Vapi and Retell AI, subject to testing authorization. We aim to examine whether adversarial audio remains effective under real-world audio processing and compression pipelines. We will share the results as soon as they become available and would welcome the reviewer's suggestions on additional platforms or evaluation settings.
References
[1] Schönherr, Lea, et al. "Adversarial attacks against automatic speech recognition systems via psychoacoustic hiding." arXiv preprint arXiv:1808.05665 (2018).
[2] Carlini, Nicholas, and David Wagner. "Audio adversarial examples: Targeted attacks on speech-to-text." 2018 IEEE security and privacy workshops (SPW). IEEE, 2018.
[3] Ma, Rao, et al. "Universal acoustic adversarial attacks for flexible control of speech-llms." Association for Computational Linguistics (ACL), 2025.
Weakness 3. We agree that our current cross-model transfer results show limited effectiveness against unseen models, even at larger perturbation budgets. However, white-box attacks represent an important worst-case threat model, where an attacker has full access to model weights and gradients. This setting is particularly relevant to open-weight Audio LLMs, which can be directly deployed in downstream applications.
Furthermore, white-box access enables controlled optimization and systematic evaluation of how adversarial perturbations interact with neural codec representations and lossy compression, which is the primary focus of our work. We will clarify this threat model and distinguish white-box effectiveness from cross-model transferability in the revised manuscript.
Question 4. Following the reviewer's suggestion, we conducted additional black-box transfer evaluations on proprietary API-only Audio LLMs, including gpt-audio, gpt-audio-1.5, and gpt-audio-mini, at the primary perturbation budget (ε=1).
We evaluated adversarial audio optimized against Qwen2.5-Omni-7B, Audio Flamingo 3, and Phi-4, without using the proprietary models' weights, gradients, or feedback during attack construction. The evaluation covers 224 target entries across three optimization seeds, under uncompressed and lossy-compressed delivery.
Under Opus 64 kbps, the attacks achieved 12.05% ASR on gpt-audio, 8.78% on gpt-audio-1.5, and 3.12% on gpt-audio-mini. These results show limited but nonzero transfer to proprietary API-only models without target-specific optimization. Full cross-model transfer results are reported in our response to Reviewer izWN, Question 3.