acceptodds
Under review as a conference paper at ICLR 2027

Joint Attention-semantic Disruption Attack on Non-autoregressive Text-to-speech Models

Abstract

Recent advancements in neural speech synthesis have shifted toward non-autoregressive models, but the adversarial robustness of such continuous-trajectory and non-token-based schemes remains under-explored. In this paper, we reveal that the mapping between the continuous acoustic prompt and the generation trajectory is highly susceptible to subtle cross-modal perturbations. We exploit this vulnerability and propose a novel black-box adversarial attack that simultaneously distorts acoustic embeddings, dissolves internal cross-modal alignment, and corrupts semantic representations. Extensive experiments show that the proposed attack successfully sabotages five state-of-the-art text-to-speech models with a small perturbation budget (SNRdB, audio waveform scaled to ) under eight representative adversarial defense measures: total variation regularization, Kullback-Leibler regularization, adversarial training, low-pass filtering, -law companding, randomized smoothing, the Griffin-Lim algorithm, and score-based speech enhancement. Our method can be applied to low-energy digital jamming, availability attack, and evaluation of generative audio robustness.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Joint attention-semantic disruption attack, non-autoregressive, text-to-speech

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_MeAdOpenReview ↗
Rating
2(+1)
Confidence
2(−1)
Informativeness
3(+2)
Presentation
2(+1)
Soundness
2(+1)
Read review
Paper summary

The paper studies attacks on reference speech used by nonautoregressive speech synthesis systems It constructs auxiliary representations with an audio codec and a language model and optimizes reference audio to increase representation differences The resulting perturbation is evaluated across speech synthesis models and defense configurations The paper uses attention differentials and trajectory sensitivity to motivate the mechanism and speaker similarity to measure its effects.

Critical strengths
  • Target query-free attacks on reference speech are a relevant robustness problem, and transfer across synthesis architectures is worth investigating.
  • The appendix includes approximately energy-matched sign randomization and temporal shifts, providing more informative evidence than a random noise comparison alone.
  • Attack settings, the reference prompt, evaluation model, and some paired uncertainty estimates are documented.
Critical weaknesses
  • The stated optimization is stationary at initialization. Section 3.1 sets x=x_ref, while Equation 15 uses L(x)=||z(x)-z(x_ref)||². Consequently its gradient is zero, and every sign-gradient update in Equation 18 leaves the input unchanged. The straight-through estimator does not remove the zero representation residual. No attack random start or other symmetry-breaking mechanism is specified in A.4.2. This is a decisive inconsistency between the stated algorithm and the reported nonzero perturbations, although it does not establish what unpublished code actually does.
  • The acoustic-to-language projection is not reproducible. The cited LLaMA is a text language model; an affine map from codec features does not establish semantic alignment. Training, initialization, data, and checkpoints for W_proj are missing. An untrained projection might still support a feature attack, but would not substantiate the semantic interpretation. LLaMA
  • Appendix uses one reference utterance and one perturbation across targets. Multiple generated sentences and datasets do not establish robustness across reference speakers or prompts. Sentence-level confidence intervals do not account for prompt-selection variability.
  • Speaker similarity does not measure semantic collapse, intelligibility, or catastrophic noise. The imperceptibility claim also needs perceptual evidence beyond the reported SNR and norm budget. Clean and adversarial prompts have different stated sampling/encoding paths that require matched controls.
  • The local linearization does not prove amplification: the propagator may contract, and no relevant lower bound or perturbation alignment is established. Auxiliary-model feature displacement does not demonstrate causal attention disruption in the target synthesizer.
  • The adversarial-training defense lacks sufficient training/checkpoint details, and most defenses lack clean-plus-defense controls. Existing local random-smoothing controls should be credited, but do not establish all defense comparisons.
Questions for authors

Q1. How does the implementation produce its first nonzero update when Section 3.1 initializes at the clean reference and Equation 15 then has zero gradient? Please provide initialization code and initial loss/gradient norms, identifying any random start or additional objective.

Q2. How are W_proj and b_proj initialized or trained, using which data, objective, and checkpoint? What evidence establishes acoustic-to-language semantic alignment?

Q3. Do all main experiments reuse the single reference utterance and perturbation in Appendix 4.2? What are the outcomes and failure cases across independently selected reference speakers and clips?

Q4. Can you provide a clean control with identical resampling/PCM processing and intelligibility/listening evidence separating identity degradation, semantic collapse, and audible noise?

Q5. What parameters, data, training budget, and held-out protocol define “100% adversarial training,” and how does each defense perform on clean inputs?

3 comments
AuthorsOpenReview ↗

Reply to Reviewer [1/3]

Thank you for identifying the implementation, reference diversity, output measurement, and defense controls that are central to evaluating this threat model. The revision provides direct evidence for each: nonzero initialization diagnostics, a precisely specified pretrained surrogate, independent reference recordings, matched audio processing, instrumented target dynamics, and clean/attacked evaluations with and without every defense. These results establish a reproducible target-query-free attack whose substantial speaker-fidelity degradation persists across the evaluated references and defensive settings.

1. Random initialization activates the objective (Q1)

The implemented attack uses a bounded random start. The revised Section 3.1, Appendix A.4.2, and Appendix A.6 now specify

\widetilde{x}^{(0)}=\operatorname{clip}_{[-1,1]}(x_{\mathrm{ref}}+u).$$ Equivalent minimal pseudocode for this operation is: ```python u = torch.empty_like(x_ref).uniform_(-eps, eps) x = (x_ref + u).clamp(-1.0, 1.0).detach().requires_grad_(True) ``` The random start provides the initial representation residual. The STE propagates its gradient through quantization. These two components have distinct roles. The seed-42 diagnostic confirms an initial loss of **369,834.5625**, an input-gradient $\ell_2$ norm of **18,562,286**, and nonzero gradients at **all 106,920 waveform coordinates**. The first maximum update is **0.002000004**, matching the specified step size $\eta=0.002$ up to floating-point rounding. An independent seed-42 process reproduces all **201 recorded states** and the final floating-point and PCM waveforms in the tested environment. With attack seeds 42, 43, and 44, **all 120 clean–attack text pairs** decrease SIM. Across-seed mean changes are $-0.4800\pm0.0189$ on F5-TTS and $-0.5571\pm0.0458$ on E2-TTS (Appendix A.7, Table A6). Seed 42 yields the smallest mean decrease of the three on both targets. # 2. Checkpoint-defined audio representations (Q2) **The surrogate uses the speech-pretrained `ICTNLP/SLED-TTS-Libriheavy` checkpoint.** `Llama1` identifies its LLaMA-style Transformer architecture: 12 layers, hidden dimension 1024, and 16 attention heads. The revised Appendix A.6 specifies exactly which components enter JASDA: the pretrained projection, normalization, and Transformer backbone. The checkpoint's TTS-specific generation modules are not used in attack construction. The interface is a checkpoint-loaded $128\rightarrow1024$ affine projection followed by affine LayerNorm with $\mathrm{eps}=10^{-6}$. Its weight, bias, and normalization tensors are loaded without reinitialization and remain frozen, together with the backbone. At attack time, these pretrained parameters provide the audio-specific mapping without additional projection fitting. JASDA optimizes only the reference waveform, using the summed squared displacement of final contextual audio representations. Independent checkpoint loads reproduce intermediate and final features in numerical precision. We also provide a speaker-disjoint audio–transcript retrieval probe using 200 recordings from 20 speakers, with training/validation/test partitions of 12/4/4 speakers (Appendix A.10, Tables A14–A17). For ten within-speaker candidate transcripts, the full surrogate and codec-only features each obtain R@1 $=0.10$ (Top-1 Recall), equal to chance among ten candidates, and all permutation $p$-values exceed $0.10$. This reports the actual transcript-discrimination behavior of the temporally pooled linear probe. The revised methodology uses the precise description **contextual audio representations**, while content effects are evaluated directly through transcription and transcript-order measurements on generated speech.
AuthorsOpenReview ↗

Reply to Reviewer [2/3]

3. Independent speakers and reference recordings (Q3)

The main five-model benchmark deliberately reuses one constructed adversarial reference across target architectures and synthesis texts, testing transfer without model-specific adaptation. The new reference-diversity experiment separately evaluates six independently selected recordings from three speakers, with two recordings per speaker, fixed selection rules, and durations of 3–6 seconds. Each recording receives its own seed-42 attack, reused across F5-TTS, E2-TTS, and ten texts per target (the revised Appendix A.7, Table A7).

All 120 paired comparisons decrease SIM. Across references, the mean changes are −0.4522-0.4522 on F5-TTS and −0.5390-0.5390 on E2-TTS, with between-reference standard deviations of 0.18570.1857 and 0.14460.1446. The weakest reference-level decreases remain 0.16130.1613 and 0.37120.3712, respectively. Table A7 reports every reference, making variation and the weakest observed cases directly inspectable.

Uncertainty is evaluated at the speaker level: 10000 cluster-bootstrap replicates retain both recordings and all generated texts for each sampled speaker. The exploratory 95% intervals are [−0.6029,−0.2257][-0.6029,-0.2257] and [−0.6670,−0.4051][-0.6670,-0.4051]. Leaving out each speaker in turn also preserves negative mean changes on both targets. These results directly establish persistence across the tested independent recordings, with speaker-cluster uncertainty distinguished from text-conditional intervals.

4. Matched processing and distinct output effects (Q4)

Identical external resampling and PCM serialization preserve the attack effect. The revised Appendix A.8 compares native clean audio (16-kHz FLOAT), processed clean audio (24-kHz PCM16), and attacked audio (24-kHz PCM16). Processing the clean reference changes mean SIM by only −0.00025-0.00025 on F5-TTS and +0.00147+0.00147 on E2-TTS. In contrast, attacked-minus-processed-clean changes are −0.4601-0.4601 and −0.5131-0.5131, corresponding to 64.81% and 65.75% relative decreases. Every matched text pair decreases SIM, and both paired 95% intervals exclude zero (Tables A8–A11).

The same outputs receive three complementary measurements (Table A12):

TargetSIM: clean → attackWER: clean → attackCER: clean → attackUTMOS22: clean → attack
F5-TTS0.7099 → 0.24988.27% → 9.00%3.40% → 3.08%2.8493 → 1.4666
E2-TTS0.7803 → 0.26729.25% → 19.46%3.46% → 8.68%2.0885 → 2.0628

WER/CER pool edit counts under fixed Whisper-base decoding and text normalization, with 411 reference words and 1821 characters per model–condition cell. F5-TTS shows pronounced predicted-quality degradation, while E2-TTS shows substantial transcription degradation. Both show strong identity degradation. These measurements distinguish the affected output dimensions.

Input strength is characterized separately across seven matched reference pairs: SNR ranges from 18.10 to 21.57 dB, with mean 19.1350 dB. The largest serialized difference is 328 PCM quantization steps, or 0.0100097656250.010009765625, following floating-point optimization at ϵ=0.01\epsilon=0.01. No input pair reaches full-scale endpoints. The revised paper therefore describes input strength in measured amplitude and energy terms, and reports UTMOS22 as predicted output quality.

5. Direct target measurements ground the mechanism

The revised Appendix A.9 measures the target synthesizer after the attack is fixed. F5-TTS clean/attack pairs share reference length, gain, initial noise, and solver grid, and instrumentation reproduces the corresponding PCM outputs. Across three texts, the maximum increases in attention mass from generation-region queries to reference-frame keys are 71.63, 75.95, and 75.26 percentage points, sampled over layers, solver evaluations, and guidance branches. This is the relevant frame-key quantity in F5-TTS's single-stream architecture.

The trajectories begin from identical synthesis states, D0=0D_0=0, and separate at every recorded point, reaching final RMS distances 3.0080, 3.4837, and 2.9179. All 192 Euler updates across the six runs are reconstructed from recorded states, velocities, and time grids. Section 3.3 expresses the local response as a conditioning-driven sensitivity integral whose magnitude depends on the trajectory Jacobians and perturbation direction. The empirical result is progressive separation under the evaluated attacks. The differential analysis identifies its propagation pathway.

The independent CTC diagnostic further records decreased intended-order advantages on all three texts: 10.73→8.4010.73\rightarrow8.40, 7.45→5.907.45\rightarrow5.90, and 11.03→9.2211.03\rightarrow9.22 nats per symbol, with intended order still preferred. This provides an additional explicitly defined output-level alignment measurement. All target instrumentation is post-construction, preserving the target-query-free attack setting.

AuthorsOpenReview ↗

Reply to Reviewer [3/3]

6. Full defense controls and specified AT checkpoints (Q5)

The revised Appendix A.11 evaluates all five models under all eight defenses in four conditions: clean (C), defended clean (CD), attack (A), and defended attack (AD). Tables A18–A27 report speaker fidelity, transcription, and predicted quality, while (CD−C)(CD-C), (AD−A)(AD-A), and (AD−CD)(AD-CD) separately quantify clean-side change, attack-side recovery, and residual attack effect.

39 of the 40 residual SIM point estimates are negative. The exception, E2-TTS with TV, accompanies a defended-clean SIM decrease from 0.78620.7862 to 0.29200.2920. The design therefore makes each defense's clean-fidelity cost visible. Recovery in one output dimension can coexist with persistent identity loss: MaskGCT AT improves attack-side WER from 12.6512.65\\% to 7.547.54\\% and UTMOS22 from 1.39651.3965 to 3.09923.0992, while retaining residual SIM −0.1334-0.1334 with paired 95% interval [−0.1595,−0.1067][-0.1595,-0.1067].

For AT, “100%” specifies offline adversarial-reference coverage of all 180 training pairs. The 244-pair LibriSpeech dev-clean manifest separates 180 training pairs/24 speakers, 32 checkpoint-validation pairs/eight other speakers, and 32 hyperparameter-development pairs/eight further speakers. Equation (58) combines adversarial and clean student-feature consistency against a frozen clean teacher, with β=1\beta=1. The training plan specifies AdamW, learning rate 10−510^{-5}, zero weight decay, gradient clipping 1.0, microbatch size one, accumulation over four microbatches, and a 100-update budget. Checkpoints are selected using mean clean/adversarial SIM on the separate 32-pair validation set.

The evaluated checkpoints are MaskGCT u050, ZipVoice step100, LEMAS-TTS u025, and F5-TTS/E2-TTS u075. All five retain negative defended attack–clean SIM differences with individually paired 95% intervals excluding zero (Table A28). This establishes persistent speaker-fidelity degradation at the explicitly documented checkpoints and training budgets.

Reviewer_UY1SOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Joint Attention-Semantic Disruption Attack (JASDA), a zero-query, single-shot adversarial attack targeting non-autoregressive (NAR) text-to-speech (TTS) models. The method constructs adversarial reference audio using a surrogate pipeline consisting of EnCodec and LLaMA-1. Specifically, it employs a Straight-Through Estimator (STE) to propagate gradients through residual vector quantization and optimizes the input waveform by maximizing the discrepancy between the final hidden representations of clean and perturbed audio. The authors further provide theoretical analyses of attention perturbation propagation and flow-matching trajectory sensitivity to explain the attack mechanism. The method is evaluated on five TTS models (F5-TTS, E2-TTS, MaskGCT, ZipVoice, and LEMAS-TTS), using text prompts from three datasets and eight defense strategies. The results demonstrate substantial reductions in speaker similarity (SIM) across different target architectures, suggesting that the constructed adversarial reference can transfer between models without target-specific optimization.

Critical strengths
  1. Meaningful problem setting and a restrictive threat model. The paper addresses an important security concern in modern zero-shot TTS systems: their vulnerability to perturbations in reference audio. The zero-query setting, in which the attacker does not require access to the target model's parameters, gradients, or inference feedback during attack construction, is challenging and practically relevant. The ability to construct an adversarial reference independently of the target model is an interesting direction for studying the transferability of speech-generation vulnerabilities.

  2. Encouraging cross-model empirical results. The authors evaluate JASDA across five TTS architectures, three text datasets, and eight defense strategies. The results consistently show considerable reductions in speaker similarity, particularly for flow-matching-based models. The ablation experiments further indicate that the optimized perturbation direction provides additional effectiveness over random perturbations and that temporal alignment influences attack performance. These findings provide meaningful empirical evidence for the effectiveness of the proposed perturbation optimization under the evaluated conditions.

Critical weaknesses
  1. Insufficient justification and validation of the proposed attention-semantic disruption mechanism. The central methodological claim is that JASDA jointly disrupts acoustic representations, attention structures, and speech semantics. However, the relationship between the proposed optimization objective and these claimed effects is not sufficiently established. First, Section 3.1 describes LLaMA-1 as a framework for speech transcription and representation learning, whereas the cited LLaMA model (Touvron et al., 2023) is originally a text-pretrained language model. Although EnCodec representations are projected into the LLaMA hidden space, the manuscript does not adequately explain how the projection parameters are obtained or whether the resulting representations are semantically aligned with speech content. Without such clarification or supporting evidence, interpreting the resulting hidden states as meaningful speech-semantic representations remains insufficiently justified. Second, the optimization objective in Eq. (15) maximizes the distance between the final hidden representations of clean and perturbed audio. Although attention operations participate in the gradient propagation, this does not necessarily establish that the resulting perturbation disrupts attention alignment or speech semantics in the target TTS models. The theoretical attention differential characterizes a possible perturbation propagation pathway, but the manuscript provides limited direct empirical evidence connecting this pathway to the observed attack outcomes. Consequently, while the reported results support the effectiveness of latent-space perturbation optimization, they do not yet sufficiently substantiate the stronger claim of joint attention-semantic disruption. This distinction is important because the proposed mechanism constitutes a central part of the paper's claimed contribution.

  2. Experimental evidence is insufficient to fully support the broader attack claims and establish the method's distinctive contribution. The evaluation demonstrates substantial speaker-similarity degradation, but several aspects limit the interpretation of these results. First, SIM is the primary evaluation metric. A reduction in speaker similarity does not necessarily imply semantic corruption, loss of intelligibility, or catastrophic speech-generation failure. Additional output-level evidence, such as ASR-based WER/CER or speech-quality measurements, would help determine whether the attack primarily alters speaker identity or causes broader generation failures. Similarly, the imperceptibility claim is not fully established by the reported perturbation bound and SNR alone. Second, the experiments use a single fixed reference utterance to construct the adversarial example. Although the results demonstrate transferability across target models and generated text prompts, they do not establish generalization across different reference speakers, utterances, or acoustic conditions. The distinction between cross-model transferability and cross-reference generalization should therefore be clarified. Third, the comparison with existing adversarial approaches remains limited. The ablation study demonstrates an advantage over random perturbations, but it does not sufficiently isolate the contribution of the proposed EnCodec–LLaMA surrogate architecture or establish its advantages over simpler representation-based or comparable zero-query transfer attacks. Notably, random perturbations already cause considerable SIM degradation, suggesting that part of the observed effect may arise from the inherent sensitivity of TTS systems to noisy reference audio. The optimized perturbation provides an additional, statistically supported improvement, but the extent to which this improvement depends on the claimed attention-semantic mechanism remains unclear. These limitations do not invalidate the reported attack effectiveness. However, they make it difficult to determine whether the results establish the paper's proposed mechanism and distinct methodological contribution, rather than primarily demonstrating the vulnerability of reference-conditioned TTS systems to optimized input perturbations.

Questions for authors

Q1. Audio-to-language representation mapping. How are the projection parameters between EnCodec and LLaMA initialized or trained? Which LLaMA checkpoint is used, and is any audio-specific adaptation involved? What evidence supports interpreting the resulting hidden representations as speech-semantic features rather than generic latent representations?

Q2. Validation of the joint attention-semantic mechanism. What direct evidence demonstrates that maximizing the final hidden-state divergence disrupts both attention alignment and speech semantics in the target TTS models? Can the authors provide analyses of attention changes or semantic-content degradation, or clarify whether the proposed mechanism should instead be interpreted primarily as a latent-representation attack?

Q3. Attack effectiveness and perceptual evaluation. Can the authors clarify whether the observed SIM reductions correspond to changes in speaker identity alone or also to reduced speech intelligibility and generation quality? Relevant existing results or limited additional evaluations using WER/CER, speech-quality metrics, or perceptual assessments would help substantiate the claims of semantic collapse and imperceptible perturbations.

Q4. Methodological contribution and generalization. What evidence establishes the necessity of the EnCodec–LLaMA surrogate architecture compared with simpler representation-based or alternative zero-query surrogate approaches? Additionally, how does the attack behave when constructed using different reference speakers or utterances? These clarifications would help distinguish the method's specific contribution from general reference-audio sensitivity and establish the scope of its transferability.

6 comments
AuthorsOpenReview ↗

Reply to Reviewer [1/3]

We thank the reviewer for recognizing the relevance of the zero-query threat model and the strength of the cross-model transfer results. The revision adds the requested checkpoint specification, direct target-model measurements, output-level evaluation, matched transfer baselines, and independent reference recordings. Together, these results establish substantial transferable speaker-fidelity loss and identify how the effects on attention, synthesis dynamics, transcription, and predicted quality vary across targets.

1. Exact pretrained mapping and representation evidence (Q1; Weakness 1)

The surrogate uses a speech-pretrained Transformer and its pretrained audio projection. The revised Section 3.1 and Appendix A.6 now identify the exact checkpoint, ICTNLP/SLED-TTS-Libriheavy. Llama1 denotes the LLaMA-style architecture of its speech-pretrained autoregressive backbone: 12 Transformer layers, hidden dimension 1024, and 16 attention heads. EnCodec supplies 128-dimensional features at 24 kHz and 6 kbps. A checkpoint-loaded 128→1024128\rightarrow1024 affine projection and affine LayerNorm, with eps=10−6\mathrm{eps}=10^{-6}, connect these features to the backbone. The projection, normalization, and Transformer parameters are loaded without reinitialization and remain frozen throughout attack construction. Independent checkpoint loads reproduce intermediate and final features within numerical precision.

Only the above pretrained components are used. The checkpoint's TTS-specific generation modules are excluded from attack construction. JASDA provides audio embeddings without appended text tokens and optimizes the input waveform through the final contextual audio representations. This specification makes the exact audio-specific pretrained mapping clearer.

We also added a speaker-disjoint transcript-retrieval pilot over 200 recordings from 20 speakers, with 12/4/4 speakers for training/validation/testing (Appendix A.10, Tables A14–A17). A validation-tuned linear readout maps mean-pooled audio features to fixed transcript embeddings. Within-speaker Top-1 recall is 0.100.10 for both the full surrogate and codec features, equal to chance among ten candidates. All reported permutation pp-values exceed 0.100.10. This probe establishes limited transcript discrimination by the temporally pooled linear readout. Accordingly, the revised method describes the hidden states as speech-pretrained contextual audio representations, and evaluates changes in generated content directly through the transcription and alignment diagnostics below.

2. Direct target attention and synthesis-dynamics evidence (Q2; Weakness 1)

Appendix A.9 now measures the target model itself after attack construction. For F5-TTS, clean and attacked runs share reference length, gain, initial synthesis noise, and solver grid. Instrumentation reproduces the corresponding PCM outputs. We probe layers 0, 11, and 21 at solver evaluations 0, 16, and 31 in both guidance branches, giving 18 probes for each of three texts.

F5-TTS incorporates text along the feature axis of single-stream attention. Its keys index frames. The measured quantity is therefore attention mass from generation-region queries to reference-frame keys. Across the sampled probes, the maximum increases are 71.63, 75.95, and 75.26 percentage points for the three texts, all at layer 0, evaluation 16, in the conditional branch. Other probes show changes in both directions. These measurements directly establish substantial redistribution of target reference attention under the fixed attack.

We additionally record all 33 synthesis states across 32 Euler evaluations. The generation-region state distance begins at D0=0D_0=0, increases at every recorded point, and reaches 3.0080, 3.4837, and 2.9179 for the three text pairs. Initial velocity differences are already 2.96342.9634, 3.08393.0839, and 3.14943.1494. The revised local sensitivity analysis explicitly includes persistent conditioning changes:

et≈∫0tΦ(t,s)JC(s)ΔC,ds,e0=0.e_t\approx\int_0^t\Phi(t,s)J_{\mathcal{C}}(s)\Delta\mathcal{C}\\,\mathrm{d}s,\qquad e_0=0.

Theorem 1 characterizes the attention differential, and this controlled experiment establishes progressive trajectory separation in the evaluated target. The controlled intervention changes the acoustic reference while fixing the text and initial state. Attention redistribution and trajectory separation characterize the resulting propagation.

A separate wav2vec2 CTC diagnostic finds that the intended-order advantage over reversed word order decreases from 10.7310.73 to 8.408.40, 7.457.45 to 5.905.90, and 11.0311.03 to 9.229.22 nats per symbol. Intended order remains preferred, so the precise finding is reduced intended-order preference. These target and output measurements give the mechanism an empirical basis beyond the surrogate feature distance, while preserving the distinction between measured attention changes and an exclusive causal attribution to attention.

AuthorsOpenReview ↗

Reply to Reviewer [2/3]

3. Speaker identity, transcription, quality, and perturbation size (Q3; Weakness 2)

Appendix A.8 separately evaluates each output dimension under matched audio processing. Resampling and serialization alone change mean SIM by only −0.00025-0.00025 on F5-TTS and +0.00147+0.00147 on E2-TTS. Comparing attacked and processed-clean references in the same 24-kHz PCM16 format gives SIM decreases of 0.46010.4601 and 0.51310.5131, or 64.81% and 65.75%. All 40 text-level pairs decrease. Their paired 95% intervals are [−0.4935,−0.4243][-0.4935,-0.4243] and [−0.5521,−0.4723][-0.5521,-0.4723] (Tables A8–A11).

The additional output metrics reveal distinct target-specific effects (Table A12):

TargetSIM: clean → attackMicro WER: clean → attackMicro CER: clean → attackUTMOS22: clean → attack
F5-TTS0.7099 → 0.24988.27% → 9.00%3.40% → 3.08%2.8493 → 1.4666
E2-TTS0.7803 → 0.26729.25% → 19.46%3.46% → 8.68%2.0885 → 2.0628

Thus F5-TTS shows pronounced predicted-quality loss, with UTMOS22 decreasing on all 20 texts, while E2-TTS exhibits substantially increased transcription errors alongside similar mean predicted quality. WER/CER use frozen Whisper-base and pooled edit counts. These results establish output degradation beyond speaker identity, with its form depending on the target.

For the original and six additional references, input SNR ranges from 18.1018.10 to 21.5721.57 dB. The floating-point attack uses ϵ=0.01\epsilon=0.01. PCM16 serialization yields a maximum difference of 328 quantization steps, approximately 0.010010.01001. The revision characterizes these perturbations through their measured amplitude and energy. UTMOS22 is explicitly a prediction of output quality.

4. Matched transfer baselines and the surrogate's contribution (Q4; Weakness 2)

The revised Appendix A.10 supplies both requested comparison types under the same starting waveform, ℓ∞\ell_\infty budget, step size, and 200 updates. All attacks are constructed without calls to either target.

Against the AntiFake-RTVC single-encoder adaptation, JASDA lowers SIM from 0.40900.4090 to 0.24980.2498 on F5-TTS and from 0.38670.3867 to 0.26720.2672 on E2-TTS (Table A13). Paired improvements favoring JASDA are 0.15910.1591 and 0.11950.1195, with text-bootstrap 95% intervals [0.1330,0.1843][0.1330,0.1843] and [0.0774,0.1596][0.0774,0.1596], conditional on this reference and attack. This establishes an advantage over a representative external transfer objective under the matched evaluation.

The codec-only controls provide a complementary result. EnCodec-Q and EnCodec-C achieve lower SIM than JASDA on both targets, but all of EnCodec-Q, EnCodec-C, and JASDA have already significantly reduced SIM to sufficiently low levels as attack methods. On E2-TTS, however, JASDA achieves SIM 0.26720.2672 while retaining UTMOS22 2.06282.0628, close to clean 2.08852.0885. The codec-only UTMOS22 scores are 1.61761.6176 and 1.72581.7258. JASDA's quality-score advantages are 0.44520.4452 and 0.33700.3370, with paired intervals [0.2753,0.6155][0.2753,0.6155] and [0.1395,0.5472][0.1395,0.5472]. The complete surrogate therefore yields a distinct combination of strong identity disruption and preserved predicted quality on E2-TTS. The codec-only result establishes that identity disruption can arise at the acoustic-representation level, while the complete surrogate changes the joint identity, transcription, and predicted-quality profile. The reported achieved ℓ2\ell_2 norms also make clear that matching ℓ∞\ell_\infty and update count does not equalize perturbation energy.

The direction controls independently address general noise sensitivity: on LEMAS-TTS, sign randomization approximately preserves perturbation energy yet raises SIM by 0.04210.0421, with 95% interval [0.0319,0.0525][0.0319,0.0525]. Temporal shifting raises it by 0.06750.0675, with interval [0.0563,0.0789][0.0563,0.0789] (Table A3). Optimized direction and temporal alignment therefore contribute beyond amplitude alone.

AuthorsOpenReview ↗

Reply to Reviewer [3/3]

5. Independent references and initialization seeds (Q4; Weakness 2)

The revised Appendix A.7 expands the evaluation across reference recordings, with one independently constructed perturbation per recording. Six additional recordings from three speakers, two recordings per speaker, are each tested on ten texts for both F5-TTS and E2-TTS. All 120 paired outputs show lower SIM. Mean decreases across references are 0.45220.4522 and 0.53900.5390. Even the weakest reference yields decreases of 0.16130.1613 and 0.37120.3712 (Table A7).

Speaker-cluster bootstrap intervals are [−0.6029,−0.2257][-0.6029,-0.2257] and [−0.6670,−0.4051][-0.6670,-0.4051], explicitly exploratory with three speaker clusters. Every leave-one-speaker-out mean remains negative. This measures reference-level variation separately from text-level uncertainty. Three optimization seeds additionally produce lower SIM in all 120 text-level pairs, with mean changes −0.4800±0.0189-0.4800\pm0.0189 and −0.5571±0.0458-0.5571\pm0.0458 (Table A6). The cross-reference and seed results strengthen the original cross-model transfer evidence while retaining the documented evaluation scope.

Reviewer_vr7JOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies transferable adversarial perturbations against non-autoregressive zero-shot text-to-speech systems. It proposes Joint Attention-Semantic Disruption Attack (JASDA), which perturbs a reference speech waveform. The attack uses a frozen EnCodec encoder and a frozen LLaMA-1 decoder as a surrogate pipeline. A straight-through estimator is used to propagate gradients through vector quantization. The objective maximizes the squared distance between the final latent representations of clean and perturbed reference audio, and the waveform is updated through projected sign-gradient steps. The adversarial prompt is evaluated without querying the target models during attack construction. Experiments indicate reductions in speaker similarity for five TTS models under an undefended setting and eight defense configurations.

Critical strengths
  1. Perturbing a reference recording before it is supplied to an unknown speech-generation system is a meaningful threat model for evaluating voice-cloning robustness. The attack is constructed without target-model queries and is tested against several TTS architectures.
  2. The use of a straight-through estimator provides a way to obtain surrogate gradients through a quantized audio representation. This is a reasonable and interesting implementation choice for a transfer attack involving an audio codec.
  3. Random bounded noise, sign-randomized perturbations, and temporally shifted perturbations help assess whether the result depends on the optimized direction and temporal alignment. The reported matched-energy sign-randomization comparison is more informative than a random-noise comparison alone.
Critical weaknesses
  1. The attack initializes the adversarial waveform at the clean waveform and maximizes the squared distance between their outputs through the same frozen surrogate. Under deterministic inference, the initial loss and its gradient are exactly zero. The projected sign-gradient update therefore cannot produce a perturbation. This is a fundamental problem for the proposed method. The authors should specify the actual initialization and computation used in the experiments, report the initial loss and gradient norm, and explain any randomness or asymmetry between the clean and adversarial branches. If a random start was used, it should be included in the algorithm and experimental settings.
  2. The optimized objective only maximizes the final surrogate latent distance. Although attention mechanism contributes to the surrogate’s hidden states, a change in the final representation does not by itself establish that attention maps changed substantially, that target-model cross-modal alignment dissolved, or that generated speech lost its intended semantics. Theorem 1 gives a general softmax differential and does not show that the proposed objective causes a particular target attention change. Likewise, the ODE linearization describes a possible propagation mechanism but does not demonstrate amplification for the evaluated TTS systems. For those questions, the authors should measure clean-versus-attacked target attention maps, text–audio alignment, trajectory deviations, and transcription accuracy before presenting these mechanisms as established findings.
  3. Key experiments need to be supplemented. The paper includes useful random and perturbation-direction controls, but does not provide a matched comparison against representative transferable audio attacks or simpler surrogate feature-disruption objectives under the same perturbation budget. The absence of target queries does not preclude comparison with other zero-query transfer methods.
Questions for authors
  1. Are the clean and adversarial surrogate branches evaluated deterministically with identical preprocessing and model states? If stochasticity or a random start is used, how sensitive are the results to random seeds?
  2. How does JASDA perform across independently selected speakers and reference utterances? Please report per-reference variation and aggregate uncertainty across reference recordings, not only across generated texts.
  3. Can the authors measure changes in attention maps or text–audio alignment inside at least one accessible target model? What evidence distinguishes attention disruption from a more general perturbation of speaker-conditioning features?
  4. How does the method compare with matched zero-query transfer baselines, such as attacks against the codec representation alone or other surrogate feature-distance objectives?
6 comments
AuthorsOpenReview ↗

Reply to Reviewer [1/2]

We thank the reviewer for recognizing the relevance of the zero-query threat model, the use of STE through the audio codec, and the value of the perturbation-direction controls. The revision establishes an active and reproducible optimization procedure, directly measures attention redistribution and trajectory separation inside F5-TTS, and adds matched zero-query baselines and independent reference recordings. We address the three weaknesses and four questions below.

1. Random-start initialization, deterministic branches, and seed sensitivity (W1, Q1)

JASDA initializes the waveform with a bounded random perturbation:

\tilde x^{(0)}=\operatorname{clip}_{[-1,1]}(x_{\mathrm{ref}}+u), \qquad \epsilon=0.01.$$ This initialization is now explicit in the revised Section 3.1, the attack configuration, and Appendix A.6, Eq. (85). The random start provides the representation residual, and STE carries its gradient through codec quantization. The diagnostic preserves the random-number draw position after model loading and clean-feature computation. Appendix A.6 reports the initial state and first updates for seed 42: the initial loss is **369,834.5625**, the input-gradient $\ell_2$ norm is **18,562,286**, and all **106,920** waveform coordinates have nonzero gradients. The first maximum coordinate update is **0.002000004**, matching the specified $\eta=0.002$ up to floating-point rounding. Exact-clean initialization is separately verified to produce zero loss, gradient, and update. These diagnostics identify precisely how the implemented attack begins to move. Appendix A.7 further evaluates seeds 42, 43, and 44 on 20 texts per target, holding synthesis seed 42 fixed and matching 24-kHz PCM16 serialization. **All 120 text-level clean–attack pairs show lower SIM.** Across the three attack seeds, mean $\Delta\mathrm{SIM}$ is $-0.4800\pm0.0189$ on F5-TTS and $-0.5571\pm0.0458$ on E2-TTS. The dispersion is the sample standard deviation across seeds (Table A6). Seed 42 produces the smallest mean reduction among the tested seeds on both models. An independent seed-42 process reproduces all 201 recorded optimization states and the final floating-point and PCM waveforms in the tested environment. # 2. Independent reference recordings and speaker-level uncertainty (Q2) Appendix A.7 adds six independently selected 3–6-second recordings from three speakers, with two recordings per speaker and fixed sampling rules. Each recording receives its own perturbation, reused across both targets and ten texts per target. The resulting 240 outputs yield 120 clean–attack comparisons. **Every reference–target combination decreases SIM on all ten texts.** Table A7 reports the complete per-reference results: | Reference | F5-TTS $\Delta\mathrm{SIM}$ | E2-TTS $\Delta\mathrm{SIM}$ | | --- | ---: | ---: | | 2803-154320-0010 | $-0.4856$ | $-0.6754$ | | 2803-154328-0021 | $-0.5701$ | $-0.6586$ | | 84-121123-0018 | $-0.1613$ | $-0.3712$ | | 84-121123-0021 | $-0.2902$ | $-0.4389$ | | 6295-244435-0019 | $-0.6271$ | $-0.4157$ | | 6295-64301-0030 | $-0.5786$ | $-0.6739$ | The mean reductions are **0.4522** and **0.5390**, with between-reference standard deviations of 0.1857 and 0.1446. A speaker-cluster bootstrap retains both recordings and every generated text within each selected speaker. Its exploratory 95% intervals are $[-0.6029,-0.2257]$ for F5-TTS and $[-0.6670,-0.4051]$ for E2-TTS. These intervals summarize the three independent speaker clusters. Every leave-one-speaker-out mean also retains a substantial reduction: $[-0.5654,-0.3768]$ and $[-0.6059,-0.4750]$, respectively. Thus the added evaluation explicitly measures both recording variation and uncertainty at the speaker level.
AuthorsOpenReview ↗

Reply to Reviewer [2/2]

3. Target attention, synthesis dynamics, and output content (W2, Q3)

Appendix A.9 directly instruments the target model after attack construction. The fixed attack remains independent of target parameters, gradients, and inference feedback during optimization. Clean and attacked F5-TTS runs share reference length, gain, initial noise, and solver grid. Instrumentation reproduces the corresponding PCM outputs.

Target attention. We probe layers 0, 11, and 21 at solver evaluations 0, 16, and 31 in both classifier-free guidance branches, giving 18 probes per text for three texts. Because F5-TTS uses single-stream attention with frame-indexed keys and text information along the feature axis, the measured quantity is attention mass from generation-region queries to reference-frame keys. The maximum increases across these probes are 71.63, 75.95, and 75.26 percentage points. All three maxima occur at layer 0, evaluation 16, in the conditional branch. Other probes exhibit both increases and decreases. These observations establish substantial redistribution of attention inside the target synthesizer.

Synthesis trajectories. We record all 33 states from 32 Euler evaluations. Paired runs start at exactly the same state, D0=0D_0=0, while their first velocity differences are 2.9634, 3.0839, and 3.1494. Generation-region state distance then increases at every recorded point, reaching 3.0080, 3.4837, and 2.9179 at the final stored state. All 192 Euler updates across the six runs are recovered from the recorded states, velocities, and grid. Section 3.3 and Appendix A.9 formulate the mechanism as reference-conditioning-driven dynamics:

\qquad e_0=0.$$ Theorem 1 provides the attention differential. The revised dynamics explain how the persistent conditioning change enters synthesis, and the controlled target runs establish the observed progressive separation. Together, these controlled measurements directly document attention redistribution and progressive trajectory separation under the same reference-conditioning change. They establish the observed target response while leaving the relative causal contributions of individual pathways open. **Transcript order and transcription accuracy.** A frozen external CTC evaluator measures intended-order advantages of $10.73\to8.40$, $7.45\to5.90$, and $11.03\to9.22$ nats per symbol for the same three texts. Intended word order remains preferred; this diagnostic quantifies its reduced advantage. Appendix A.8 additionally reports matched-clean Whisper WER/CER and UTMOS22 results on 20 texts per target (Table A12). On E2-TTS, WER rises from **9.25% to 19.46%** and CER from **3.46% to 8.68%**, alongside SIM falling from 0.7803 to 0.2672. On F5-TTS, SIM falls from 0.7099 to 0.2498 and predicted quality from **2.8493 to 1.4666**, while WER/CER change from 8.27%/3.40% to 9.00%/3.08%. These separate measurements identify model-dependent effects on identity, content, and predicted quality. # 4. Matched zero-query transfer baselines and simpler objectives (W3, Q4) The revised Appendix A.10 and Table A13 add an **AntiFake-RTVC single-encoder adaptation**, quantized codec-only disruption with STE (**EnCodec-Q**), and continuous pre-quantization codec disruption (**EnCodec-C**). All share the actual starting waveform, waveform domain, $\ell_\infty$ budget 0.01, step size 0.002, and 200 updates. All attacks are constructed without querying F5-TTS or E2-TTS. Against AntiFake-RTVC, JASDA lowers mean SIM by an additional **0.1591 on F5-TTS** and **0.1195 on E2-TTS**, with paired text-bootstrap 95% intervals $[0.1330,0.1843]$ and $[0.0774,0.1596]$. These comparisons condition on the fixed reference and attack. The codec-only objectives produce stronger SIM reductions, but all of EnCodec-Q, EnCodec-C, and JASDA have already significantly reduced SIM to sufficiently low levels as attack methods, and the complete surrogate produces a distinct output profile. On E2-TTS, JASDA combines SIM **0.2672** with UTMOS22 **2.0628**, close to the clean prediction of **2.0885**. EnCodec-Q/C yield SIM 0.1476/0.1322 and UTMOS22 1.6176/1.7258. JASDA’s predicted-quality advantages over Q/C are **0.4452/0.3370**, with paired intervals $[0.2753,0.6155]$ and $[0.1395,0.5472]$. The achieved perturbation $\ell_2$ norms are also reported: 2.3571 for JASDA, 2.6953/2.6735 for Q/C, and 2.1124 for AntiFake-RTVC. The comparison therefore makes explicit both the common optimization budget and the different identity, transcription, and quality outcomes produced by these surrogate objectives.
AuthorsOpenReview ↗

Your Comments are Highly Valuable to Us [Deadline in One Day]

Dear reviewer vr7J,

We have carefully considered all the valuable points and concerns you raised in your initial review through the initial rebuttal. As the deadline for discussion is approaching in only one day, we respectfully ask whether our response has addressed your concerns, or you still have questions to raise. Your comments are highly valuable to us. Thank you once again for your professional time and insightful effort in reviewing our submission.

Best regards,

Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Discussion

Dear Area Chair,

Thank you for overseeing the discussion. JASDA reveals a security risk in reference-conditioned speech generation. An attacker who can alter a short reference recording before synthesis can undermine the system's ability to reproduce the intended speaker, without accessing target parameters, gradients, or query feedback. A user can then submit an ordinary synthesis request and receive speech with substantially degraded speaker fidelity and, on some targets, increased transcription errors. Because the same altered recording transfers across architectures and requested texts, an attacker can interfere with multiple downstream synthesis systems through the reference material alone.

These failures can coexist with similar predicted output quality. On E2-TTS, speaker similarity falls from 0.78030.7803 to 0.26720.2672 and word error rate rises from 9.25% to 19.46%, while mean UTMOS22 remains close to its clean value (Table A12). Controlled F5-TTS measurements further show redistribution of attention to reference frames and progressive separation of synthesis trajectories from identical initial noise across three texts (Appendix A.9).

The vulnerability transfers across five models, with one altered recording reducing mean speaker similarity by 23.3% to 65.8% on LibriSpeech-PC (Table 1). Across six additional recordings from three speakers, all 120 paired comparisons on F5-TTS and E2-TTS show lower similarity (Table A7). The effect also persists at all five evaluated adversarially trained checkpoints on a speaker held out of training, checkpoint selection, and hyperparameter tuning (Table A28).

JASDA's full speech-pretrained surrogate also provides a distinct combination of speaker-fidelity disruption and predicted output quality. Across the six additional recordings, it reduces mean E2-TTS similarity from 0.72180.7218 to 0.18280.1828. At matched perturbation energy for each recording, its mean UTMOS22 exceeds EnCodec-Q and EnCodec-C by 0.37640.3764 and 0.78670.7867, respectively (Appendix A.12). The revision specifies the pretrained components, bounded random start, and adversarial-training modules, with an independent rerun reproducing the final attack waveform (Appendices A.3.1, A.6, and A.7).

Thank you for your time and consideration.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.