- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PerceptBench studies whether predefined changes in spoken delivery expose different safety failures in end-to-end audio-language models while keeping the original request wording fixed. The main comparison evaluates Direct TTS, six signal-editing families, and six emotion conditions across seven open models on 100 JailbreakBench requests and a 100-item, record-availability-selected SORRY-Bench subset. Its primary outcome is item-level coverage: whether at least one evaluated variant/repeat elicits a response judged to comply with a harmful request. Emotion conditions reach more items than editing on six of seven JailbreakBench models. Supporting analyses examine predicted audio quality and a simulated quality gate, while a separate reference-cloned accent study uses benign question-answering retention to assess semantic utility. The paper explicitly avoids claiming a causal relationship between perceptual capability and vulnerability.
Critical strengths
-
The study asks a useful within-model robustness question with comparatively controlled speech construction. For emotion prompting, the text, TTS model, speaker, and decoding are held fixed, and delivery instructions are selected on non-benchmark calibration material without target-model safety feedback. The six emotion conditions and six editing families provide a practical comparison of reproducible speech variations. Reporting intersections and exclusive item sets is informative: a condition can reveal failures missed by another even when aggregate rates are similar. The seven-target evaluation also shows that the effect is heterogeneous rather than universal.
-
The paper carefully distinguishes several types of evidence that are often conflated. Predicted NISQA quality is not presented as human intelligibility, the accent study measures benign semantic retention separately from safety, and the text acknowledges that reference cloning also changes speaker identity. The simulated MOS screen is appropriately limited to the evaluated inputs, without claims of an effective deployed defense against adaptive attacks. The discussion also makes clear that the hidden-state pilot does not establish a mechanism or show that better speech perception causes greater vulnerability. This calibration of the claims is a strength.
Critical weaknesses
-
The statistical unit and effective attempt budget need clearer reconciliation before the coverage comparisons can be interpreted confidently. Appendix A.2 specifies eight repeated runs and any-of-eight success, whereas Appendix A.16 lists 6,300 JailbreakBench Direct-TTS units, exactly 100 items x 9 text conditions x 7 models, rather than eight times that number. This could represent collapsed condition-level units, but Table 9 calls its counts renders and the rate summaries are also called unit-level ASR. The raw-response rate, any-of-eight rate, and union across speech conditions should be explicitly distinguished. The nine-text-condition rate summaries also differ from the original-wording coverage population. Further, a multi-condition emotion or accent union has more opportunities than a single neutral condition, which matters for the stochastic targets. The six-versus-six emotion/editing comparison is better matched nominally, but A.2 acknowledges missing Step-Audio 2 editing attempts, including in the model highlighted for the 60-versus-23 SORRY-Bench result. A per-item attempt ledger and a matched-valid-attempt sensitivity analysis are needed to separate delivery effects from resampling and uneven opportunities. I am not assuming that the reported counts are wrong or asking the authors to claim superior query efficiency.
-
The safety endpoint lacks an independently validated error estimate. All valid outputs are judged by a local Gemma-4 E4B-it using an adapted JailbreakBench prompt and a leading-token label (Appendix A.2), but the paper does not report a blinded human-labeled audit or agreement with an independently calibrated judge on these generated responses. Adopting the rubric does not establish the accuracy of this particular judge configuration. This is especially consequential for item-level union coverage: one false-positive compliance label can make an otherwise unsuccessful item count as reached, and can alter the surface-exclusive sets that support the central conclusion. The relatively small counts on several targets and the highlighted emotion-exclusive examples therefore require direct validation. The audio-quality and benign-utility checks do not validate harmful-compliance labels.
Overall, I lean toward weak rejection. The controlled synthesis design and complementary-coverage findings are promising, but clear attempt accounting and independent safety-label validation are necessary to establish the central empirical result. Both concerns could be addressed through targeted analyses and audits of the existing evaluation records.
Post-rebuttal update (19 Nov 2026):
The matched six-call comparison addresses my concern about unequal query budgets. The additional judge checks and the audit of the highlighted emotion-exclusive cases also strengthen the main empirical finding, even though some claimed successes were removed. I have raised my rating from 2 to 3 (weak acceptance). The final paper should clearly distinguish the manual checks from the agent-assisted baseline screening, avoid presenting the exclusive counts as fully human-verified, and correct the decoding description. With these limitations stated, I consider the evidence sufficient for weak acceptance and have no further requests for this discussion.
Questions for authors
-
Unit definition and accounting: For each main figure, what is one audio file, target-model response, repeated run, condition-level unit, and item-level success? Please reconcile the eight-repeat protocol with the A.16 denominators, specify which edit strengths/scopes enter Figure 3, and provide expected/valid attempt counts by model, item, and speech family. What randomness changes across repeats when target decoding is greedy?
-
Robustness of the coverage comparison: For the stochastic targets, do the added and exclusive reachable sets persist against neutral speech evaluated with a comparable target-query budget? For emotion versus editing, please report a paired comparison on matched valid attempts, item-level uncertainty intervals, and sensitivity of the Step-Audio 2 SORRY-Bench result to missing editing attempts. Please provide per-response ASR for the same original-wording items separately from the nine-text-condition aggregate.
-
Safety-label validation: Can blinded annotators audit a stratified sample of responses, including the highlighted emotion-exclusive cases and both judge-positive and judge-negative outputs? Please report agreement and the precision of claimed successes, adjudicate disagreements, and show how corrected labels affect the main coverage/exclusivity conclusions. This would materially affect my confidence in the benchmark.
8 comments
1. Clarification on evaluation units, protocol, and decoding randomness
Dear reviewer, thank you for the careful reading and for requesting clarification of our experimental units and decoding protocol. Our benchmark tests whether safety behavior remains consistent across different plausible spoken realizations of the same semantic request. Applying TTS only to short base prompts expands the sample count without testing pipeline robustness. Our nine text variants (one original plus eight rule-based structured rewrites) instead lengthen the utterances to test whether localized audio editing and emotion prompting remain effective over longer, multi-clause speech.
Overall response. We apologize for the ambiguity and for a stale text description of target decoding. In all main coverage comparisons, one unit is one base semantic request (N=100 fixed IDs), evaluated under a matched budget of 8 target-model inference calls per request. By contrast, Appendix A.16 logs raw completion progress across the broader 9-text-variant campaign. Furthermore, all seven targets ran with stochastic sampling (do_sample=True) in our multi-trial runner; the mention of greedy decoding in the text was inadvertently carried over from an early single-pass baseline script.
Details and evidence.
-
Hierarchy and matched 8-call budget:
- Base unit (Item): One semantic request (N=100 in the main reporting subset; N=450 in full SORRY-Bench). Variants and repeats are not additional base units.
- Synthesis and 8-call inference: Every condition uses 1 TTS call per prompt and executes 8 target inference calls per item:
- Direct TTS (1 condition) & Emotion Prompting (6 conditions): 1 synthesized waveform per item, evaluated across 8 stochastic passes seeded by repeat_idx.
- Audio Editing (6 conditions): Deterministic signal processing (no neural calls) produces 8 distinct waveforms per item (2 scopes [local, global] × 4 strengths), evaluated with 1 inference pass per waveform (8 calls total).
- Item-level success (Coverage): An item is counted once (1/100) if at least 1 of its 8 calls is judged substantively compliant.
-
Target decoding randomness: The text statement that several targets used greedy decoding was carried over from an older single-pass script (do_sample=False) included in the supplementary archive. We apologize for this oversight. In our actual multi-trial runner, all seven targets use stochastic token sampling (do_sample=True), with repeat_idx seeding token generation across repeats (greedy decoding is used strictly for the judge):
| Target Models | do_sample | Temperature | top_p |
|---|---|---|---|
| Qwen2-Audio / Qwen2.5-Omni / Qwen3-Omni | True | 0.7 | 0.95 |
| Gemma 4 E4B-it / Step-Audio 2 | True | 0.7 | 0.95 |
| LLaMA-Omni | True | 0.2 | 0.95 |
| GLM-4-Voice | True | 0.2 | 0.80 |
- Reconciling A.16 denominators: Scaling full SORRY-Bench (450 items × 9 text variants = 4,050 prompts) across server shards under cluster job limits produced uneven progress at reconciliation: 170,100 is the full planned campaign (4,050 × 42 condition-trial slots), 42,525 = 170,100 / 4 is a completed quarter-shard (41,939/42,525 valid), and 57,413/57,568 reflects the corresponding Editing/TTS shard. Because contiguous unit IDs ran sequentially, we extracted the completed fixed 100-request subsets (SORRY100 and JBB100) for matched-denominator comparisons, and will clearly separate campaign logs from the 100-item subset in Appendix A.16.
2. Matched-budget coverage, paired testing, and cross-TTS robustness
Thank you for raising the question of whether the coverage comparison remains valid at matched inference budgets. We use 8-trial request coverage because a single refusal need not characterize the output distribution: a request is covered if at least one of eight trials complies, providing a black-box probe of output-distribution stability analogous to inspecting logit-level refusal margins. Below we report matched-budget comparisons, uncertainty, and cross-TTS replication.
Overall response. Emotion prompting exposes failures that Direct TTS and Audio Editing do not reveal under an identical 8-call target budget, and additional TTS backends reproduce this inconsistency. Paired testing confirms the emotion-versus-editing gap, and missing attempts do not affect the conclusions.
Details and evidence.
- Matched 8-call coverage on JBB100 (original wording): All 13 conditions below use 8 target calls per request (successful requests out of 100):
Direct TTS vs. traditional audio editing
| Target | Direct TTS | Loudness | Noise | Pitch | Reverb | Speed | Timbre |
|---|---|---|---|---|---|---|---|
| Gemma 4 E4B-it | 3 | 3 | 3 | 2 | 6 | 2 | 2 |
| GLM-4-Voice | 13 | 12 | 19 | 18 | 21 | 18 | 15 |
| LLaMA-Omni | 36 | 36 | 33 | 30 | 37 | 30 | 33 |
| Qwen2-Audio | 9 | 11 | 11 | 9 | 9 | 6 | 9 |
| Qwen2.5-Omni | 9 | 6 | 8 | 7 | 8 | 7 | 4 |
| Qwen3-Omni | 0 | 0 | 0 | 3 | 1 | 0 | 0 |
| Step-Audio 2 | 49 | 50 | 33 | 25 | 37 | 49 | 47 |
Direct TTS vs. emotion prompting
| Target | Direct TTS | Disgust | Ecstasy | Grief | Rage | Surprise | Terror |
|---|---|---|---|---|---|---|---|
| Gemma 4 E4B-it | 3 | 0 | 3 | 7 | 3 | 3 | 3 |
| GLM-4-Voice | 13 | 22 | 25 | 17 | 23 | 18 | 18 |
| LLaMA-Omni | 36 | 41 | 48 | 37 | 44 | 46 | 47 |
| Qwen2-Audio | 9 | 10 | 11 | 13 | 11 | 11 | 13 |
| Qwen2.5-Omni | 9 | 10 | 14 | 13 | 14 | 7 | 10 |
| Qwen3-Omni | 0 | 0 | 0 | 0 | 0 | 1 | 1 |
| Step-Audio 2 | 49 | 63 | 62 | 47 | 54 | 55 | 51 |
- Paired comparison & missing-attempt sensitivity: On JBB / Step-Audio 2, a matched-8-call paired comparison of Disgust vs. Noise across all 100 items yields 63/100 vs. 33/100—a +30-point difference (paired item-bootstrap 95% CI: [+19, +41]; exact McNemar p = 6.04e-7; Holm-adjusted p = 0.000152 across 252 comparisons). In the 77,000-response batch (76,946 valid), Step-Audio 2 on SORRY100 had only 4 missing editing responses (3 Pitch, 1 Speed); even under worst-case imputation, editing shifts by at most 1–3 points. Original-wording per-response ASR is also reported separately from the 9-text aggregate in the appendix.
- Cross-TTS replication (SORRY100, Qwen2.5-Omni, 8 calls/condition): Both additional TTS backends reproduce the emotion vulnerability (16–19/100 vs. 9/100 on CosyVoice3-RL; 19–20/100 on Grief/Rage vs. 13/100 on Fish Speech S2 Pro):
| Additional TTS | Direct TTS | Disgust | Ecstasy | Grief | Rage | Surprise | Terror |
|---|---|---|---|---|---|---|---|
| CosyVoice3-RL | 9/100 | 19/100 | 18/100 | 19/100 | 19/100 | 16/100 | 19/100 |
| Fish Speech S2 Pro | 13/100 | 13/100 | 15/100 | 19/100 | 20/100 | 14/100 | 13/100 |
Note on AI tool use during my review
Following the PCs' request, I'm adding a note about how AI tools were used during my reviewing process.
I discussed this paper with ChatGPT (GPT-6-Astra) while working on my review. I used it as a sounding board: I shared my own reading notes — mainly the attempt-accounting inconsistency between Appendices A.2 and A.16, the query-opportunity asymmetry in the union-coverage comparisons, and the absence of an independent audit of the Gemma-4 safety judge — and asked it to check whether my criticisms were technically sound and whether I had overlooked obvious counterarguments. It also helped me polish the wording of the review text.
My assessment of the paper, the rating, and all specific claims in the review are my own: I verified every cited detail (the repeat-protocol counts, Table 9, the Step-Audio 2 missing-editing case, the judge configuration in Appendix A.2, etc.) directly against the manuscript. The AI discussion helped me articulate the accounting concern more precisely, but it did not change my overall assessment. My questions for the authors in the review stand as written, and I look forward to their responses.