acceptodds
Under review as a conference paper at ICLR 2027

AudioGuard: Toward Comprehensive Audio Safety Protection Across Diverse Threat Models

Abstract

Audio has rapidly become a primary interface for foundation models, powering real-time voice assistants and widely accessible text-to-speech and voice-cloning services. Ensuring safety in these settings is inherently more complex than “unsafe text spoken aloud”: real-world failures can hinge on audio-native harmful sound events, speaker attributes (e.g., child voice), impersonation/voice-cloning misuse, and voice–content compositional harms. These challenges have left the community without standardized definitions, benchmarks, or practical guardrails that capture the full audio risk landscape. To close this gap, we conduct large-scale red teaming of modern audio-capable AI systems and voice generation pipelines, uncovering systematic vulnerabilities unique to the audio modality. Next, we build a comprehensive, policy-grounded audio risk taxonomy and introduce AudioGuardBench, the first benchmark that jointly evaluates audio-input and audio-output safety across diverse threat models, with sliceable coverage over languages, voices (including celebrity/impersonation and child voice), and non-speech sound events. To defend against these threats efficiently, we propose AudioGuard, a unified guardrail that decomposes safety reasoning into complementary components: SoundGuard for waveform-level audio-native cue detection and ContentGuard (ASR + TextGuard) for policy-grounded semantic moderation, composed via an interpretable integration module for scenario-specific decisions. Extensive experiments on AudioGuardBench and five complementary benchmarks show that AudioGuard consistently improves guardrail accuracy over strong audio-LLM-based baselines while substantially reducing latency.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Guardrail Model, Audio Language Model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_jQ5QOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents AUDIO SAFETY BENCH, a policy-grounded benchmark for evaluating audio safety across diverse threat models, and AUDIO GUARD, a modular audio safety framework. The benchmark covers multilingual speech, non-speech harmful events, speaker attributes, voice impersonation, and voice–content compositional risks. AUDIO GUARD combines SoundGuard, which detects waveform-level acoustic cues using an ECAPA-TDNN encoder, with ContentGuard, which performs ASR-based semantic safety classification. A rule-based integration module combines their predictions into scenario-specific safety decisions. Experiments on the proposed benchmark and four external benchmarks demonstrate competitive safety detection performance compared with several audio large language models.

Critical strengths
  1. The paper addresses an important and timely problem in audio safety, extending beyond traditional transcript-based content moderation to include sound events, speaker attributes, and voice–content compositional risks.
  2. AUDIO SAFETY BENCH provides a relatively comprehensive evaluation setting, covering 17 languages, diverse speaker conditions, and multiple audio-related threat models. The inclusion of hard-benign examples is particularly useful for evaluating false positives.
  3. The modular design of AUDIO GUARD is practical and interpretable. Compared with monolithic audio-LLM judges, the proposed framework demonstrates strong detection performance and potentially more efficient deployment. The evaluation also includes external benchmarks, component-wise analyses, and statistical uncertainty estimates.
Critical weaknesses
  1. The threshold selection and calibration protocol are insufficiently specified. The compositional integration relies on category-specific thresholds, but their numerical values and selection procedures are not clearly described. It is unclear whether these thresholds are calibrated on independent validation data or adjusted across benchmarks. Since threshold selection directly affects safety accuracy, unsafe recall, and false-positive rates, the authors should clarify the calibration protocol and report the operating thresholds.
  2. The benchmark annotations and evaluation protocols require greater transparency. AUDIO SAFETY BENCH is a major contribution, but the complete fine-grained label inventory, per-category sample distributions, and training/validation/test partitioning criteria are insufficiently documented. The mapping from sound and content annotations to final safety labels also requires clarification. In particular, the authors should explicitly define whether Content-only Accuracy in Table 5 measures exact risk-category matching or binary Safe/Unsafe accuracy, and explain how annotations from the four external benchmarks are converted into the unified evaluation space.
  3. The distinction between speaker recognition and voice impersonation detection is unclear. SoundGuard uses a pretrained ECAPA-TDNN encoder to recognize speaker-related attributes, including public-figure voices. However, recognizing a public figure's voice does not necessarily distinguish genuine recordings from AI-generated voice clones. The authors should clarify whether SoundGuard explicitly detects synthesized or impersonated speech, whether genuine and cloned recordings of the same speaker receive different labels, and how voice-cloning misuse is evaluated in the ElevenLabs subset. Otherwise, the framework appears closer to speaker-aware content moderation than explicit impersonation detection.
  4. The choice of the SoundGuard backbone lacks sufficient justification. ECAPA-TDNN is primarily designed for speaker representation learning, whereas SoundGuard must also recognize diverse non-speech events. Comparisons with general-purpose audio encoders such as EAT under matched training conditions would help justify this design. Moreover, reporting the total parameters and computational cost of the complete AUDIO GUARD pipeline, including ASR and TextGuard, would better substantiate its claimed efficiency relative to large audio-language models.
  5. The current ablation study does not directly quantify the benefit of sound–content integration. Tables 5 and 6 report content-only and sound-only accuracy, respectively, while Table 2 evaluates joint sound–content accuracy. Since these metrics correspond to different prediction targets, their values cannot directly establish the benefit of fusion. A more informative ablation would evaluate SoundGuard-only, ContentGuard-only, and full AUDIO GUARD on identical test samples using the same binary Safe/Unsafe labels and metrics, including Accuracy, F1, unsafe recall, and FPR. This is particularly important for voice–content compositional scenarios.
Questions for authors

How are the integration thresholds calibrated, and are they fixed across benchmarks within each threat model?

What constitutes a correct prediction in Tables 2 and 5, and how are external-benchmark annotations mapped to these evaluation targets?

Does the ElevenLabs evaluation measure speaker-identity screening, voice-clone detection, or both? How are genuine and cloned recordings of the same speaker labeled?

Could you compare SoundGuard-only, ContentGuard-only, and full AudioGuard on identical samples using the same binary Safe/Unsafe labels, reporting unsafe recall and FPR, particularly for compositional risks and hard-benign cases?

4 comments

We sincerely thank the reviewer for the detailed and technically insightful feedback. We appreciate the recognition of the practical and interpretable design of AudioGuard. Below, we clarify the calibration procedure, evaluation metrics, architectural choices, and benefits of compositional integration.

Q1. How are integration thresholds selected, and are they fixed across benchmarks?

Thank you for raising this important methodological question.

AudioGuard uses a policy-grounded integration mechanism in which each rule is expressed as a conjunction of threshold conditions over sound/voice cues and semantic content risks. The thresholds are configured according to the relevant threat model and remain fixed across benchmarks evaluated under the same threat-model configuration. The separate 10% calibration partition is used for threshold selection; evaluation examples are not used to adjust thresholds on a per-benchmark basis.

Q2. What exactly do the reported accuracy metrics measure, and how are benchmark labels mapped into the evaluation space?

We appreciate this request and agree that the distinctions between the different evaluation metrics should be more explicit.

Our evaluation considers three related but distinct prediction targets:

  1. Content-only accuracy (Table 5): Evaluates the correctness of semantic content risk predictions, independently of acoustic cue recognition.
  2. Sound-only accuracy (Table 6): Evaluates the correctness of waveform-level audio-native cue predictions, such as speaker-related attributes and non-speech events.
  3. End-to-end joint sound+content accuracy (Table 2): Evaluates the combined sound and semantic predictions under the unified evaluation protocol.

These metrics serve different purposes. In particular, the component-wise metrics quantify the recognition capabilities of individual modules, whereas the end-to-end metric evaluates their combined behavior. We agree that the paper should not rely on comparisons between different prediction targets as direct evidence of fusion improvements.

For external benchmarks, the available unsafe categories are collapsed into a single unsafe label, and results are reported at the binary safe/unsafe level rather than as category-wise classification accuracy. Consequently, the external scores should be interpreted as binary safety detection results, not fine-grained taxonomy-matching accuracy.

Q3. Why is ECAPA-TDNN appropriate for general sound-event detection, and how efficient is the complete pipeline?

Thank you for the suggestion. Our selection of ECAPA-TDNN is primarily motivated by its efficient architecture and strong pretrained acoustic representations.

SoundGuard is designed as a lightweight detector of safety-relevant acoustic cues rather than a general-purpose audio understanding model. We initialize it with the pretrained SpeechBrain ECAPA-TDNN encoder and attach a multi-label classification head trained on our audio-native risk categories. This design enables reuse of pretrained acoustic features while keeping the task-specific training component relatively small.

Although ECAPA-TDNN was originally developed for speaker representation learning, the relevant question for our application is whether its learned representations support accurate recognition of the audio-native cues needed by the safety taxonomy. Our experiments demonstrate sound-only accuracy of 0.887 on Speech, 0.876 on Non-Speech, and 0.918 on ElevenLabs, supporting its effectiveness within the evaluated risk categories.

Regarding efficiency, Table 2 reports the observed end-to-end deployment latency: 1.423 seconds for AudioGuard, compared with 3.245 seconds for Gemini 3 and 2.542 seconds for GPT-Audio. These measurements include the AudioGuard pipeline rather than SoundGuard alone.

We emphasize that these measurements are not compute-matched: AudioGuard is evaluated locally, whereas the proprietary models are accessed through APIs with additional network and service overhead. We therefore interpret these results as observed deployment latency under the available interfaces, not as proof of universally lower intrinsic computational cost.

The modular architecture also enables SoundGuard and ContentGuard to operate in parallel and allows deployments to enable only the components required for the relevant threat model.

Q4. Does sound–content integration improve final safety decisions when evaluated under identical labels and metrics?

We appreciate this suggestion and agree that the benefit of sound–content integration should be distinguished from the individual recognition accuracy of the two components.

The key motivation for AudioGuard is that audio safety cannot always be decomposed into independent sound and content moderation decisions. Many real-world policies are inherently compositional: the same semantic content or acoustic cue can lead to different safety decisions depending on their combination and deployment context.

For example, detecting sexual content without recognizing that the speaker has a child-like voice may miss the relevant compositional violation. Conversely, blocking all child-like voices or all sensitive acoustic events would produce unnecessary false positives. Similarly, public-figure voice impersonation combined with misinformation can create a policy-relevant risk that independent moderation systems may not adequately represent.

Our integration module explicitly models these relationships through policy-grounded rules over SoundGuard and ContentGuard predictions. This provides two key advantages:

  1. Effectiveness: The system can identify risks requiring joint acoustic and semantic evidence, which transcript-only moderation cannot capture.
  2. Flexibility: The same detectors can support different audio safety policies and deployment scenarios through configurable integration rules without retraining the underlying models.

Our existing component-wise evaluations establish that SoundGuard and ContentGuard recognize complementary signals, but they use different prediction targets and therefore do not by themselves quantify the gain from fusion under a common binary label. The current evidence supports the design motivation and overall end-to-end effectiveness.

Dear Reviewer,

Thank you again for your constructive feedback. We have addressed your questions in our rebuttal and provided additional clarifications.

We would greatly appreciate it if you could take another look at our responses when you have a chance. Please let us know if any concerns remain or if further clarification would be helpful.

Thank you for your time and consideration!

Reviewer_jQ5QOpenReview ↗

Official Comment by Reviewer jQ5Q

Thank you for the clarifications. However, two central concerns remain unresolved. First, the current evaluation does not establish whether SoundGuard can distinguish genuine voices from cloned voices of the same speaker, which is essential to the impersonation-detection claim. Second, as the authors also acknowledge, the existing component-wise results do not demonstrate the actual benefit of sound–content integration. The requested same-sample, same-label comparison of SoundGuard-only, ContentGuard-only, and the full AudioGuard is still missing. These are important empirical gaps in the central claims of the paper, so I maintain my original rating.

Reviewer_KoE1OpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper points out that audio safety cannot be judged from spoken words alone. Speaker identity, non-speech sounds, and the combination of voice and content can also create safety risks. The authors introduce AudioSafetyBench, a new benchmark covering these cases, and AudioGuard, which separately checks audio signals and spoken content before combining the results. Experiments show better accuracy than several audio language models used as safety judges.

Critical strengths
  • Safety of Large Audio Language Models is important as the voice agent and audio interactive systems are more and more broadly used.
  • AudioSafetyBench covers many useful settings, including 17 languages, different speakers, non-speech sounds, and difficult safe examples.
  • The evaluation includes four external benchmarks, and the authors distinguish these results from evaluations on their own benchmark.
Critical weaknesses

The benchmark construction needs more detail. Much of the audio is synthesized, and the safety labels are based on policies collected from different platforms. It is unclear how the labels were verified, how different policies were combined, and how overlap between training and evaluation examples was prevented.

The proposed method is relatively straightforward. It combines an audio classifier, speech recognition, a text safety model, and policy rules. The paper does not clearly show what is new compared with existing safety pipelines.

Questions for authors

What does AudioGuard add beyond existing audio detection and speech recognition plus text safety pipelines?

How much does SoundGuard improve the final safety decisions over ContentGuard alone?

How were benchmark labels verified, different platform policies combined, and training or evaluation overlap prevented?

3 comments

We sincerely thank the reviewer for the positive assessment and constructive suggestions. We appreciate the recognition of AudioSafetyBench's coverage and the importance of audio-specific safety evaluation. We address the concerns regarding novelty, component contributions, and benchmark reliability below.

Q1. What does AudioGuard contribute beyond a conventional audio classifier + ASR + text safety pipeline?

We appreciate this question and would like to clarify that our contribution is not the introduction of an individual audio encoder or ASR model, but rather a unified, policy-grounded framework for audio safety across diverse threat models.

Our work makes three closely connected contributions.

First, we introduce an audio-specific safety taxonomy and benchmark. Existing text-centric safety formulations are insufficient for risks involving non-speech sound events, speaker attributes, impersonation, and combinations of acoustic and semantic information. AudioSafetyBench systematically captures these dimensions across input and output safety scenarios.

Second, we propose a modular audio safety architecture that explicitly separates acoustic evidence from semantic content. SoundGuard detects waveform-level cues that cannot be recovered reliably from transcripts, while ContentGuard identifies semantic policy violations.

Third, we introduce policy-grounded compositional integration. Rather than independently flagging all sensitive acoustic or semantic cues as unsafe, AudioGuard evaluates their combinations under deployment-specific policies. For example, a child-like voice is not inherently unsafe, but its combination with sexual content may constitute a serious safety violation. Similarly, a public-figure voice combined with misinformation can introduce risks that neither speaker recognition nor text moderation captures independently.

This formulation also allows the same acoustic and semantic detectors to be reused across voice assistants, TTS services, and voice-cloning applications by changing the applicable policy rules without retraining the underlying components.

Thus, the central distinction from a conventional ASR-plus-text pipeline is the explicit use of waveform-level cues in context-dependent, compositional policy decisions.

Q2. Does SoundGuard meaningfully improve end-to-end safety decisions beyond ContentGuard alone?

Yes. SoundGuard addresses a fundamental limitation of transcript-only safety systems: some safety-relevant information is inherently acoustic and cannot be recovered from the transcript.

For example, a transcript-based guardrail may correctly recognize the semantic content of an utterance while remaining unaware that the speech uses a child-like voice or impersonates a public figure. Similarly, non-speech events such as distress sounds, gunfire, or sexual sounds may contain important policy-relevant information without meaningful transcribed text.

The two components are therefore complementary rather than redundant. Our current results support this decomposition: ContentGuard achieves 0.936 content-only accuracy on the Speech split, while SoundGuard achieves 0.887 sound-only accuracy on the same split. However, these metrics evaluate different prediction targets and should not be directly compared to quantify the contribution of fusion.

The central advantage of AudioGuard is the ability to compose these signals for safety decisions, particularly when risk depends jointly on sound and content.

Q3. How are benchmark labels verified, platform policies unified, and training–test overlap prevented?

Thank you for highlighting the importance of benchmark reliability. Our benchmark construction follows a policy-grounded, multi-stage process.

Policy consolidation. We collect publicly available safety policies from over 20 audio-related platforms, including messaging services, conferencing applications, live-streaming platforms, and audio-sharing services. We use an LLM-assisted pipeline to extract relevant safety requirements and organize them into a unified hierarchical taxonomy covering semantic harms, audio-native risks, and compositional risks.

Instance generation and verification. We generate scenario-grounded audio examples conditioned on these policy categories, including both unsafe cases and hard-benign counterfactuals. Our construction pipeline applies multiple filtering criteria, including audio intelligibility, generation quality, and consistency with the target safety policy. The construction procedure retains unsafe examples based on intended policy-violation criteria checked by human review. This describes example verification; no inter-annotator agreement statistic is reported in the current manuscript.

Data partitioning. We adopt a 70% / 10% / 20% split for training, calibration, and testing, respectively. The training set is used to optimize SoundGuard and TextGuard, while the independent calibration set is used to select integration thresholds without accessing test labels. The held-out test set is reserved for final evaluation. To prevent leakage, examples derived from the same source audio or underlying content are assigned to the same partition, including their synthesized and augmented variants. Importantly, the four external benchmarks are independently constructed and are not used for model training or threshold calibration.

This partitioning description concerns source-level overlap; it does not imply that every public-figure identity is held out across splits.

Thanks for your clarifying, your responses have addressed my concerns. I will maintain my score as 3.

Reviewer_PnHYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies audio safety guardrails. The authors build AudioSafetyBench, a policy-grounded benchmark intended to cover audio-input and audio-output safety, including public-figure and child-like voices, voice–content compositional risks, non-speech sound events, and multiple languages. The authors propose AudioGuard, which combines SoundGuard, ContentGuard, and a rule-based layer that produces scenario-specific decisions. On AudioSafetyBench and four external benchmarks, AudioGuard is reported to outperform audio-LLM judges.

Critical strengths
  1. Rather than equating audio safety with harmful text rendered as speech, the paper considers when the audio signal itself is safety-relevant. It identifies risks that are not recoverable from a transcript, namely non-speech harmful events, speaker identity and attributes, and voice–content combinations, and treats such acoustic cues as contextual signals whose safety relevance depends on content and deployment setting rather than as intrinsically unsafe.
  2. The benchmark’s hard-benign counterfactuals keep the speaker and acoustic cues and change only the semantics, so a system that reacts to a sensitive voice, sound, or keyword alone will fail on them. The authors also acknowledge that AudioGuard is trained on the same policy space as AudioSafetyBench, and therefore evaluate it additionally on four independently constructed benchmarks instead of relying on in-domain results alone.
Critical weaknesses
  1. The external evaluation does not establish the claimed generalization advantage. On the external benchmarks, AudioGuard and Gemini 3 are relatively close, in contrast to the large margins on AudioSafetyBench. Several of the external benchmarks appear to contain only harmful queries, so a stricter content policy naturally scores higher. In addition, the “joint sound+content accuracy” metric needs a formal definition.
  2. The paper does not compare against any trained audio guardrail, such as SALMONN-Guard (ICML 2026), and related benchmarks such as AudioTrust (ICLR 2026) are not discussed. AudioSafetyBench also lacks real human speech, yet input-side moderation mostly faces real speech, and screening user audio before cloning concerns real recordings typically. Large-scale red teaming is listed as a contribution, but the paper reports no red-teaming findings (e.g., systems tested, scale, bypass rates).
Questions for authors
  1. Is the training data of SoundGuard and TextGuard disjoint from all AudioSafetyBench test splits in terms of speakers and sound clips? How does SoundGuard handle public figures not seen in training?
  2. For public-figure voices re-synthesized in other languages, was it verified that the cloned identity is preserved?
  3. How are baseline refusals (e.g., declining speaker identification) scored under the joint metric?
  4. What is the safe/unsafe ratio of each external benchmark? Please report, for all methods, the block rate on harmful samples and the pass rate on benign samples.
4 comments

We sincerely thank the reviewer for the thoughtful feedback and constructive suggestions. We appreciate the recognition of our audio-specific safety formulation and hard-benign counterfactual evaluation. Below, we address each concern and clarify the contributions and evaluation protocols.

Q1. Does AudioGuard genuinely generalize to external benchmarks, or do its improvements primarily reflect stricter blocking policies?

We appreciate this important question. We explicitly distinguish two evaluation settings:

  1. Matched-policy evaluation: AudioSafetyBench evaluates how effectively AudioGuard implements the target audio safety taxonomy. Since AudioGuard is trained under the same policy space, we do not interpret these results as architecture-independent generalization.
  2. Out-of-distribution evaluation: We evaluate on four independently constructed external benchmarks (Nemotron-Audio, Jailbreak-AudioBench, Omni-SafetyBench, and AdvWave), which are not constructed using our benchmark generation pipeline.

As reported in Table 2, AudioGuard achieves an external-only average accuracy of 0.876, compared with 0.836 for Gemini 3 and 0.823 for GPT-Audio. AudioGuard also achieves the highest point estimate on each of the four external benchmarks.

We agree that aggregate accuracy alone does not fully characterize safety–utility trade-offs, especially on datasets with imbalanced safe/unsafe distributions. Our Appendix Table 4 already reports unsafe recall and hard-benign false-positive rates. AudioGuard achieves unsafe recall of 0.900–0.940, with FPRs of 0.160–0.236 on deliberately challenging hard-benign subsets. On standard benign Nemotron-Audio examples, the observed FPR is 0.032.

Q2. How does AudioGuard compare with trained audio guardrails and related audio safety benchmarks?

Thank you for highlighting these related works. Following the reviewer's suggestion, we have added SALMONN-Guard (ICML 2026), a specialized trained audio safety guardrail, as an additional baseline. The results are summarized below, alongside our method.

MethodASB SpeechASB Non-SpeechASB ElevenLabsNemotronJailbreakOmniAdvWaveExternal Avg.
SALMONN-Guard0.2340.3520.1830.7930.7280.4230.7720.679
AudioGuard (Ours)0.8320.8760.8900.9130.8930.7430.9530.876

AudioGuard consistently outperforms SALMONN-Guard across all seven evaluation settings, achieving an absolute improvement of 19.7 percentage points on the four external benchmarks (0.876 vs. 0.679). The particularly large improvements on AudioSafetyBench highlight the importance of explicitly modeling audio-native and compositional safety risks, although these matched-policy results should not be interpreted as architecture-independent generalization.

Regarding AudioTrust, the relevant distinction is the scope of evaluation: our benchmark explicitly covers audio-native events, speaker attributes, compositional risks, and both input- and output-side deployment scenarios. This comparison concerns threat-model coverage, not a claim that AudioTrust lacks all of these capabilities. Unlike approaches primarily focused on speech-content safety, AudioSafetyBench provides unified evaluation across (1) non-speech harmful sound events, (2) speaker identities and attributes, (3) voice–content compositional risks, and (4) both audio-input and audio-output safety across diverse deployment scenarios.

This additional trained-guardrail baseline complements the prompted audio-LLM comparisons, while the larger in-domain gains should be understood in light of the matched-policy setting.

Q3. How representative is AudioSafetyBench of real-world audio safety scenarios, given its reliance on synthesized speech?

We agree that real-world applicability is an important consideration. Our use of synthesized speech is a deliberate benchmark design choice rather than an assumption that synthetic audio fully represents real-world speech.

Controlled synthesis allows us to systematically vary speaker identities, acoustic attributes, linguistic content, and risk combinations while preserving the relevant safety labels. In particular, our hard-benign counterfactuals preserve potentially sensitive acoustic cues while changing the policy-relevant semantics, allowing us to determine whether models perform genuine contextual safety reasoning rather than simply matching acoustic or lexical triggers.

Beyond the controlled synthesis setting, we evaluate on the ElevenLabs red-teaming subset, which involves a separate commercial voice-generation pipeline and thus introduces different synthesis artifacts and speaker characteristics. AudioGuard achieves 0.890 accuracy on this subset. We additionally evaluate robustness under noise and overlapping audio, where accuracy changes from 0.832 to 0.793, remaining above the evaluated Gemini 3 and GPT-Audio baselines under the same perturbation setting.

We acknowledge that naturally occurring spontaneous speech, diverse recording devices, and unconstrained multi-speaker environments remain important directions for further evaluation.

Q4. What concrete evidence supports the claimed large-scale audio red teaming?

Thank you for this suggestion. Our red-teaming process directly motivates both the audio-specific risk taxonomy and the benchmark construction.

We investigate multiple deployment-relevant threat models, including:

  1. Audio-input moderation for audio-language models and voice assistants.
  2. Audio screening in voice-cloning pipelines.
  3. Audio-output safety for speech generation and TTS systems.
  4. Interactive voice-agent scenarios involving both input and output audio.

Our instance-generation pipeline combines policy-grounded unsafe content generation, controlled voice synthesis, speaker conditioning, audio-native event construction, and multi-stage filtering. Candidate instances are filtered based on generation quality, transcript-only moderation behavior, and consistency with the intended safety policy.

The resulting taxonomy captures non-speech harmful events, speaker-dependent risks, and voice–content compositional violations that cannot be adequately characterized through text alone. However, the current manuscript does not report per-target attack counts or bypass rates, so the red-teaming claim is best understood as a description of the benchmark-construction process rather than a quantified attack-success comparison.

Q5. How is data leakage prevented, and can SoundGuard recognize previously unseen public-figure voices?

We appreciate the opportunity to clarify the training and evaluation setup.

SoundGuard is trained to predict audio-native cues, including speaker-related attributes and selected public-figure identities, whereas TextGuard is trained for policy-grounded semantic moderation. The benchmark uses training, calibration, and test partitions, with the stated 70% / 10% / 20% allocation detailed in our response to KoE1. Grouping source recordings and their derivatives within a partition is intended to prevent audio-variant and transcript leakage; this does not itself establish speaker-identity-disjoint evaluation.

Importantly, recognition of an unseen recording and recognition of a previously unseen speaker identity are different tasks. The current SoundGuard formulation uses predefined speaker-related labels and does not explicitly solve open-set identification of arbitrary public figures. Accordingly, we do not claim open-set identification of arbitrary previously unseen public figures.

Nevertheless, SoundGuard demonstrates strong performance across different voice and synthesis conditions, achieving sound-only accuracy of 0.887 on Speech and 0.918 on ElevenLabs. These results indicate performance on the evaluated recording and synthesis conditions, but do not establish open-set identity recognition or unseen-identity generalization.

Q6. Does multilingual voice synthesis preserve the intended speaker identity?

Our multilingual construction is designed to preserve the target speaker condition while changing the language of the spoken content; we distinguish this from independently verified identity preservation.

As described in Section 3.2, we first translate the original transcript using the Google Translate API and then re-synthesize the translated content under the same target voice condition. This process allows us to construct multilingual counterparts while preserving the intended risk category, deployment scenario, and speaker condition.

The benchmark covers 17 languages, enabling evaluation of safety behavior across linguistic variations under controlled voice conditions.

This protocol preserves the specified conditioning identity, but conditioning alone is not proof of perfect perceptual or biometric identity preservation in every language. We therefore interpret the multilingual experiments as tests under consistent target-voice conditioning, not as a separate verification study of cross-lingual speaker identity.

Q7. How is joint sound–content accuracy defined, and how are baseline refusals scored?

We thank the reviewer for requesting clarification of the evaluation protocol.

AudioGuard produces two complementary groups of predictions: (1) sound/voice cues extracted directly from the waveform and (2) semantic content risk labels extracted from the recognized speech. These predictions are combined through a policy-grounded integration layer to support scenario-specific safety decisions.

For the end-to-end comparison, audio-language-model baselines receive the same evaluation taxonomy and are prompted to produce structured sound and content predictions. As described in Section 5.1, all methods are evaluated using the same joint sound+content evaluation protocol. For tasks with both sound and content labels, a joint prediction is counted as correct only when both required components are correct; where a dataset provides only binary safety labels, evaluation is performed in that available label space.

A baseline refusal to produce the required structured risk prediction cannot count as a correct joint prediction. The joint metric evaluates the requested prediction rather than treating refusal as evidence of correct safety classification.

Dear Reviewer,

Thank you again for your constructive feedback. We have addressed your questions in our rebuttal and provided additional clarifications.

We would greatly appreciate it if you could take another look at our responses when you have a chance. Please let us know if any concerns remain or if further clarification would be helpful.

Thank you for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.