Joint Attention-semantic Disruption Attack on Non-autoregressive Text-to-speech Models

Recent advancements in neural speech synthesis have shifted toward non-autoregressive models, but the adversarial robustness of such continuous-trajectory and non-token-based schemes remains under-explored. In this paper, we reveal that the mapping between the continuous acoustic prompt and the generation trajectory is highly susceptible to subtle cross-modal perturbations. We exploit this vulnerability and propose a novel black-box adversarial attack that simultaneously distorts acoustic embeddings, dissolves internal cross-modal alignment, and corrupts semantic representations. Extensive experiments show that the proposed attack successfully sabotages five state-of-the-art text-to-speech models with a small ${\ell }_{\infty }$ perturbation budget $ϵ=0.01$ (SNR$\approx 19$dB, audio waveform scaled to $[-1,1]$) under eight representative adversarial defense measures: total variation regularization, Kullback-Leibler regularization, $100$ adversarial training, low-pass filtering, $\mu$-law companding, randomized smoothing, the Griffin-Lim algorithm, and score-based speech enhancement. Our method can be applied to low-energy digital jamming, availability attack, and evaluation of generative audio robustness.