- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper focuses on a safety challenge in multimodal models, where harmful evidence may be obscured by benign context when multiple input modalities are combined. This can cause safety models to fail to detect harmful content. To address this problem, the authors introduce Multi2AV-Safety, a benchmark designed to evaluate model safety across different multimodal input combinations. The benchmark covers 11 input combinations, multiple attack types, and harm structures. The authors also conduct controlled experiments to examine how different modality combinations and additional benign context affect safety detection. Furthermore, the paper proposes PerceptGuard as a targeted solution. It introduces a Risk Relation Loss that explicitly constrains the risk relationships between individual modalities and their combined inputs. It helps the model better detect harmful content that emerges from multimodal combinations.
Critical strengths
-
The proposed Multi2AV-Safety benchmark covers diverse input modality combinations, attack types, and harm structures, effectively addressing the limited multimodal input coverage of existing safety benchmarks. This benchmark has considerable practical value. Furthermore, the authors conduct controlled experiments to analyze how different modality combinations and the addition of benign context affect safety detection, providing direct empirical evidence for the identified safety challenges. The evaluation of 32,592 generated outputs by three independent experts, with high inter-annotator agreement, further strengthens the reliability of the experimental results.
-
Beyond introducing the benchmark, the paper also proposes a targeted solution, PerceptGuard. In particular, the design of the Risk Relation Loss is thoughtful and well-motivated. By explicitly constraining the risk relationships between individual modalities and their combined inputs, it helps the model better recognize harm emerging from multimodal composition and prevents harmful evidence from being obscured by benign context. The subsequent ablation studies further demonstrate the effectiveness of PerceptGuard in improving safety detection.
Critical weaknesses
-
Limited evaluation of false positive rates. The paper does not fully evaluate PerceptGuard's false positive rates across different multimodal input configurations. A practical safety guard should achieve both high detection rates and low false positive rates. Although the authors provide additional FPR experiments in the appendix, the evaluation is limited in scale and modality coverage, especially lacking four-modality benign inputs. Moreover, PerceptGuard still has a relatively high FPR of 42.52% on MMSafeAware Over-safety. I suggest adding more experiments to examine the trade-off between FPR and recall.
-
Imbalanced benchmark distribution. The benchmark has an uneven data distribution, with few samples in some categories. This may affect the representativeness of the overall results. For example, bimodal inputs account for 80.3% of the benchmark, while four-modality inputs account for only 6.1%. The four-modality Temporal category contains only 25 samples, which may lead to less reliable statistical results. As a result, the overall metrics may not fully reflect model performance across complex multimodal combinations and attack types. Although the authors report results for individual groups, I suggest increasing the number of samples and making the sample sizes more consistent across categories to improve the statistical reliability of the results.
Questions for authors
-
Have the authors evaluated the trade-off between recall and false positive rates under different decision thresholds? In particular, does PerceptGuard maintain its detection performance when the FPR is reduced?
-
How does the uneven sample distribution across modality combinations and attack types affect the overall evaluation results? Would the main conclusions remain consistent if different categories were given similar weights?
5 comments
Response to Weakness1&Question1 [2/2]
MMSafeAware Over-safety further tests recognition of benign cross-modal context, where an individual modality may appear unsafe while the complete image–text request is benign. To contextualize performance on this challenging subset, we additionally evaluate three general-purpose multimodal models on the same 428 pairs.
Table R4. False positives on the 428 benign image–text pairs from MMSafeAware Over-safety.
| Model | Predicted Safe | Predicted Harmful | FPR ↓ |
|---|---|---|---|
| Claude-Opus-5 | 210 | 218 | 50.93% |
| gemini-4-argon | 251 | 177 | 41.36% |
| GPT-6-Sol | 192 | 236 | 55.14% |
| Shieldstral-1.0-3B | 188 | 240 | 56.07% |
| GR-Omni | 143 | 285 | 66.59% |
| SingGuard-4B | 136 | 292 | 68.22% |
| PerceptGuard (Ours) | 246 | 182 | 42.52% |
PerceptGuard achieves 42.52% FPR, below the evaluated Claude and GPT models and close to Gemini’s 41.36%. It also outperforms all evaluated specialized guards, reducing FPR by 13.55 percentage points relative to the strongest specialized baseline, Shieldstral-1.0-3B. These comparisons highlight the difficulty of recognizing benign cross-modal context and PerceptGuard’s competitive performance on this subset.
We have incorporated the expanded benign evaluation and recall–FPR analysis, including ROC curves, into Section 4.2 (p. 9, Fig. 5). Detailed benign-input comparisons are provided in Appendix E.5 (p. 38, Tables E.5–E.6), with threshold-specific results in Appendix E.10 (p. 41, Tables E.13–E.14).
Response to Weakness2
To our knowledge, Multi2AV-Safety is the first safety benchmark for audio–video generation covering all 11 multimodal input combinations. Within each three- and four-modality configuration, we allocate 25 requests per feasible combination of attack intent and harmful-evidence assignment, balanced across the five harm categories. Direct attacks admit 15 such assignments with four modalities versus three with two, requiring five times as many requests for equal per-assignment coverage, alongside coordinated media generation and verification of individual and joint safety. In the future work, we will expand these subsets with additional independent, expert-validated samples to improve coverage and balance across modality orders.
To assess the effect of modality imbalance, we construct a 2,025-request evaluation set with 675 requests per modality order. We retain all 675 four-modality requests and select 675 bimodal and 675 trimodal requests through stratified sampling within input configurations, covering harm structures, attack intents, and harm categories. Aggregate scores assign equal weight to each evaluated modality order, preventing the larger bimodal group from dominating the comparison.
Table R5. Performance by modality order on the balanced evaluation set and comparison with the original full benchmark (%).
| Model | Metric | Bimodal | Trimodal | Four-modal | Order-balanced | Original Full |
|---|---|---|---|---|---|---|
| MiniMax-H3 | ASR ↓ | 62.48 | 75.56 | 79.56 | 72.53 | 73.20 |
| Seedance2 | ASR ↓ | 25.63 | 40.15 | 36.00 | 33.93 | 34.02 |
| LTX-2 | ASR ↓ | 94.22 | 91.56 | 89.33 | 91.70 | 91.65 |
| daVinci-MagiHuman | ASR ↓ | 78.98 | 78.46 | — | 78.72 | 78.76 |
| Qwen3-Omni | Recall ↑ | 65.33 | 66.22 | 52.59 | 61.38 | 61.35 |
| GuardReasoner-Omni | Recall ↑ | 81.19 | 72.15 | 61.19 | 71.51 | 71.50 |
| PerceptGuard | Recall ↑ | 74.81 | 87.70 | 95.85 | 86.12 | 86.06 |
Table R6. Performance by attack intent on the balanced evaluation set.
Parentheses report the original full-benchmark results (%).
| Model | Metric | Direct | Jailbreak | Temporal | Adversarial |
|---|---|---|---|---|---|
| MiniMax-H3 | ASR ↓ | 83.30 (82.06) | 64.97 (62.19) | 43.66 (59.52) | 50.78 (52.59) |
| Seedance2 | ASR ↓ | 27.73 (31.18) | 41.83 (34.75) | 49.33 (42.07) | 38.67 (44.72) |
| LTX-2 | ASR ↓ | 95.64 (96.22) | 87.83 (83.47) | 74.67 (88.00) | 88.00 (88.00) |
| daVinci-MagiHuman | ASR ↓ | 92.25 (88.49) | 61.93 (64.27) | 72.73 (70.77) | 90.00 (78.67) |
| Qwen3-Omni | Recall ↑ | 74.93 (73.82) | 39.33 (39.28) | 54.67 (48.83) | 54.67 (52.31) |
| GuardReasoner-Omni | Recall ↑ | 82.49 (85.93) | 59.33 (51.00) | 60.00 (52.14) | 52.89 (49.95) |
| PerceptGuard | Recall ↑ | 83.64 (85.39) | 87.00 (83.12) | 88.00 (91.72) | 95.56 (94.87) |
The balanced evaluation preserves the main patterns underlying our four findings. Generators remain vulnerable under complex multimodal conditioning, with four-modality ASRs ranging from 36.00% to 89.33%. Existing safeguards also continue to struggle, with four-modality recall of 52.59% for Qwen3-Omni and 61.19% for GuardReasoner-Omni. Temporal attacks still elicit harmful outputs from individually benign inputs and evade both baseline guards, supporting the compositional-risk finding. Obscured intent likewise remains challenging. On Seedance2, Jailbreak, Temporal, and Adversarial attacks all achieve higher ASRs than Direct attacks, while both baseline guards detect these categories less reliably than Direct attacks.
PerceptGuard retains the highest order-balanced recall at 86.12%, closely matching its 86.06% full-benchmark result and exceeding GuardReasoner-Omni’s 71.51%. It also leads across all four attack categories in the balanced evaluation. These results support the consistency of the main conclusions under equal representation of modality orders.
We have summarized the balanced-subset findings in Section 4.2 (p. 10), clarified the coverage limitations and planned expansion in Section 5 (p. 10), and added the detailed evaluation in Appendix G.1 (p. 46, Tables G.1–G.2).
Response to Weakness1&Question1 [1/2]
We appreciate the reviewer’s positive assessment of the benchmark’s broad coverage, controlled experiments, and PerceptGuard’s risk-relation design. To address your concerns about false positives and sample imbalance, we added six targeted evaluations and analyses, reported in eight tables (R1–R8), together with ROC curves across eight tasks (Fig. 5) and revised the corresponding manuscript sections. Supporting results and brief analyses are provided in the replies below.
To expand benign coverage, we add 60 four-modality requests whose individual inputs and complete combinations are verified as benign. This brings the evaluation to 610 benign requests across all 11 multimodal input configurations. PerceptGuard achieves the lowest FPR among the compared models at 8.36% (Table R1).
Table R1. False-positive rates on 610 benign multimodal requests.
To examine the recall–FPR trade-off, we select five request-level and three response-level tasks with relatively balanced benign and harmful classes from the 32-task evaluation suite. Keeping the trained model fixed, we vary the decision threshold from 0.1 to 0.9 and report recall, FPR, and F1 in Tables R2 and R3. We also provide the corresponding ROC curves in the revised main text (Section 4.2, p. 9, Fig. 5), with AUROC above 0.95 on all five request-level tasks.
Table R2. Request-level performance across decision thresholds.
Entries report recall / FPR / F1 (%).
Table R3. Response-level performance across decision thresholds.
Entries report recall / FPR / F1 (%).
PerceptGuard retains strong detection as the false-positive rate is reduced. At a threshold of 0.7, recall exceeds 80% on all eight tasks, with XD-Violence achieving 93.80% recall at 5.00% FPR. At a threshold of 0.9, AEGIS 2.0 request detection retains 83.06% recall, while FPR falls from 16.42% to 7.20%, more than halving the default FPR with a 1.98-percentage-point decrease in F1.