acceptodds
Under review as a conference paper at ICLR 2027

SEFIR: Safety Detection in Frozen Multimodal Large Language Models via Internal Evidence Fusion

Abstract

Multimodal safety detection is difficult because harmfulness may depend on the interaction between an image and a text prompt. A common solution is to align each multimodal backbone with safety-specific data, but this is costly to repeat across model families, scales, and safety taxonomies. To address this issue, we propose SEFIR (Safety Evidence Fusion from Internal Representations), a lightweight plug-in detector for frozen MLLMs. SEFIR does not rely on the final representation alone, nor does it concatenate all hidden states. Instead, it builds a compact safety evidence space from internal representations in the visual encoder, multimodal projector, and language model. It summarizes each selected component, keeps safety-discriminative coordinates through sparse diagnostic selection, and fuses the retained evidence using held-out component utility. A small prediction head is then trained on the fused representation for global safety classification. Across multiple MLLM backbones and multimodal safety benchmarks, SEFIR improves over matched safety-oriented baselines without backbone-level realignment. Further analyses show that safety evidence is unevenly distributed across internal components, and that sparse selection with held-out utility-guided fusion helps organize this evidence into an effective lightweight detector.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Multimodal safety detection, Frozen multimodal large language models, Sparse probing, Internal representation fusion

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_SGpAOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SEFIR, a safety detector for frozen MLLMs. It selects sparse features from the visual encoder, projector, and language model, then combines them using held-out F1 scores. Experiments report gains across several backbones and datasets.

Critical strengths
  1. Good motivation: using internal representations avoids updating the full backbone for safety detection.
  2. The experiments cover several models and datasets. SEFIR improves over the supervised probes in Table 6, and the ablations test the main design choices.
Critical weaknesses
  1. Unclear safety labels: SarcNet (Yue et al., 2024) has sarcasm labels, but Appendix A describes its labels as harmfulness. The paper does not explain how safety labels were obtained, including image and text labels. This makes the SarcNet safety results hard to interpret.
  2. Unclear gain over prior work: SIREN (Jiao et al., 2026) already uses sparse feature selection and F1-weighted fusion. SEFIR changes pooling and supervision and adds visual and projector features, but there is no direct comparison with SIREN. In Table 7, adding visual and projector features gives only a 1.31-point average gain over language-only detection.
  3. Limited transfer evidence: Table 3 only tests transfer from Falcon. Without simpler probes trained on the same source data, it is unclear whether SEFIR transfers better than those methods.
  4. Incomplete efficiency evaluation: Figure 6 reports detector-head FLOPs, excluding pooling and hidden-state access. The paper does not report measured latency or peak memory, so the actual inference cost is unclear.
Questions for authors
  1. Can you provide the label mappings and explain whether SarcNet (Yue et al., 2024) received new safety annotations?
  2. Can you compare with SIREN (Jiao et al., 2026) using the same MLLM language states, training data, and prediction head?
  3. What are the latency and peak memory costs, including hidden-state collection and pooling? The FLOPs in Figure 6 cover only the detector head.
3 comments

Thank you for your feedback.

Q1. ... label mappings ... safety annotations ...

Answer: Sarcasm can convey derogation or abuse, making such uses relevant to safety detection. For example, sample 2506.jpg in the released SARCNet dataset pairs a news screenshot with commentary including "真该死的媒体" ("the media really deserve to die"), an explicit abusive statement within ironic commentary. We include SARCNet as a complementary test of context-dependent image-text understanding. Its separate modality and pair annotations also support SEFIR's modality-specific diagnostic supervision. We retain the original sarcasm labels, mapping image_label, text_label, and multi_label to yv,yt,yy^v,y^t,y, respectively; no new safety annotations were added. We have clarified the SARCNet labels in Appendix A.1 and A.4. Across the other six datasets in Table 1, the mean Macro-F1 scores are 90.25, 90.06, 90.57, and 87.05 for Qwen3-VL-4B, Qwen2-VL-7B, Qwen3-VL-8B, and Llama-3.2-11B, respectively: 14.18, 30.44, 21.21, and 25.54 points above their corresponding Table 1 baselines. SEFIR scores higher than the corresponding Table 1 baseline in each of these 24 backbone-dataset combinations.

Q2. ... SIREN ... language states ...

Answer: SEFIR builds an internal safety-evidence bank from the visual, projector, and language components of frozen MLLMs, combining sparse diagnostic selection with held-out-utility fusion for lightweight detection without backbone realignment. SIREN uses mean pooling, global-label probes, and min-max F1 weighting across language layers. SEFIR instead uses activation-magnitude pooling, image-side targets for visual probes, and text-side targets for projector/language probes; it fuses component evidence with softmax weights derived from truncated probes' held-out Macro-F1 against global labels.

SIREN (Jiao et al., 2026) was developed for text-only LLMs. Its original text-input pipeline omits the image; using image-conditioned MLLM states requires adapting representation extraction and retraining. It was therefore not included as an off-the-shelf multimodal baseline.

We adapted SIREN to frozen Qwen3-VL-4B and Qwen3-VL-8B, retaining its pooling, probing, and weighting choices and using SEFIR's training data and MLP architecture. Training uses fixed SEFIR settings rather than SIREN's hyperparameter search, with validation-based early stopping and calibration. SEFIR scores higher in all eight settings below, averaging a 15.14-point advantage. Entries are Macro-F1 (%); differences are SEFIR minus adaptation.

DatasetQwen3-VL-4B SIREN AdaptationQwen3-VL-4B SEFIRDifferenceQwen3-VL-8B SIREN AdaptationQwen3-VL-8B SEFIRDifference
Falcon80.1588.31+8.1682.1088.42+6.32
Hateful Memes75.1879.44+4.2676.1479.88+3.74
HoliSafe57.6296.75+39.1360.7496.18+35.44
VLGuard85.0999.72+14.6390.3799.79+9.42

Q3. ... latency ... peak memory ...

Answer: Figure 6 reports detector-head FLOPs. Appendix A.3, Efficiency Accounting, includes pooling in the total additional arithmetic cost and explains its dependence on image resolution and sequence length. It also identifies hidden-state access and coordinate gathering as memory operations. The distinction between arithmetic cost and runtime efficiency is discussed by Dehghani et al. (2021), cited in the appendix. The additional experiment directly measures the complete detection pipeline on Falcon, comparing latency and peak memory with and without SEFIR under identical conditions (Qwen3-VL-4B, RTX 5090, BF16, batch size 1):

MetricWithout SEFIRWith SEFIRIncrease
Mean latency (ms/sample)65.4670.575.11 (7.81%)
Peak allocated memory (MiB)10,847.7510,988.84141.09 (1.30%)

Synchronized timing covers image decoding and preprocessing, backbone prefill, hidden-state collection, pooling, coordinate gathering, fusion, and classification. No representation disk cache is used. Model loading and autoregressive generation are outside the timed path. We have added these measurements to Table 6.

Q4. ... transfer ... source data ...

Answer: We evaluated Falcon-trained SEFIR, Final-Layer Linear Probe, and Naive Hidden-State MLP on four target datasets using Qwen3-VL-4B. All three methods use the same source training data. Target datasets are reserved for evaluation and do not enter model selection or tuning:

TargetLinear ProbeHidden-State MLPSEFIR
Hateful Memes43.9342.8963.88
HoliSafe50.9252.6456.23
SafeTag70.9271.0179.66
VLGuard65.3870.7175.67
Mean Macro-F1 (%)57.7959.3168.86

With Falcon as the training source, SEFIR's average Macro-F1 exceeds the linear probe by 11.07 points and the MLP by 9.55 points. We have added this comparison to Table 5.

Thanks for the rebuttal and response, which addresses most of my concerns. I will update my score accordingly.

Thank you for your response.

Reviewer_LmNDOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes SEFIR, a lightweight safety detector for frozen multimodal large language models. It extracts intermediate representations from visual, projector, and language components, applies sparse coordinate selection, and fuses features using held-out utility scores before global safety classification. Experiments across multiple backbones and seven datasets show substantial gains over safety-oriented baselines, alongside ablation and transfer studies.

Critical strengths
  1. SEFIR avoids expensive full‑backbone safety realignment. Instead it harvests already‑computed intermediate representations inside frozen multimodal models. This is practically meaningful: it can be plugged into off‑the‑shelf MLLMs without full retraining, with tiny parameter and computation overhead. The design carefully handles heterogeneity across vision, projector and language components, rather than simply concatenating all hidden states or only using final‑layer output.
  2. Extensive experiments span multiple model families, seven diverse multimodal safety benchmarks, rich ablations, robustness tests under corruptions and adversarial inputs, cross‑backbone transfer. Two formal theorems give bounds for sparse coordinate truncation and representation perturbation behavior, which strengthens technical rigor beyond purely empirical engineering work.
Critical weaknesses
  1. When trained only on Falcon dataset and directly tested on unseen safety benchmarks without target‑domain fine‑tuning, Macro‑F1 drops to ~62‑70%. This shows the detector still strongly fits the risk taxonomy of training data. While authors note this limitation, there is little analysis about what types of safety concepts transfer well and what fail, which limits real‑world zero‑shot applicability.
  2. Adversarial tests are restricted to OCR prompt injection, text jailbreak and standard image corruptions. It lacks evaluations against classic visual adversarial examples (pixel‑level adversarial perturbation on images), which is a key threat vector for multimodal safety detectors built on internal representations. We do not know how much visual adversarial noise can fool SEFIR.
Questions for authors
  1. How are the global, image-side, and text-side labels defined for each dataset, particularly SARCNet and Hateful Memes? Could the authors provide explicit mappings and clarify the split-count discrepancies?
  2. Were all baselines given comparable access to target-dataset supervision? Could the authors provide broader matched-supervision comparisons?
  3. What are the end-to-end inference latency and peak memory costs, including intermediate-state extraction and pooling?
3 comments

Thank you for your feedback.

Q1. ... label definitions ... SARCNet and Hateful Memes ... split counts ...

Answer: Hateful Memes evaluates hate conveyed by image-text interaction, including cases where each modality is benign alone. Its hateful/non-hateful labels define the global target yy.

SARCNet tests context-dependent expression, where sarcasm can convey derogation or abuse. We retain its original image-side, text-side, and pair-level sarcasm labels, mapping image_label, text_label, and multi_label to yvy^v, yty^t, and yy, respectively. No new safety annotations were added. We have clarified the SARCNet labels in Appendix A.1 and A.4.

We directly use the official Hateful Memes training and evaluation splits [1, 2] and have clarified this protocol in Appendix A.1.

Q2. ... target-dataset supervision ... matched-supervision comparisons ...

Answer: Table 1 compares systems with different access to target-dataset supervision. We now report matched-supervision results across four backbones and seven datasets.

Final-Layer Linear Probe (Linear) and Naive Hidden-State MLP (MLP) use the same frozen backbones, target training splits, and evaluation protocol as SEFIR. We also trained Guard + LoRA for three epochs using the same dataset partition protocol. We have added the Guard + LoRA comparison with Llama-3.2-11B SEFIR to Table 2 and its training settings to Appendix A.2. Each backbone cell lists SEFIR / Linear / MLP; Guard + LoRA reports one score per dataset. All values are Macro-F1 (%).

DatasetQwen3-VL-4BQwen3-VL-8BQwen2-VL-7BLlama-3.2-11BGuard + LoRA
Falcon88.31/84.88/83.7588.42/85.96/84.9585.96/80.55/83.0985.97/80.26/81.8442.30
Hateful Memes79.44/76.13/78.0079.88/73.42/69.8584.19/81.68/80.9168.20/62.64/64.8371.08
HoliSafe96.75/91.64/90.3896.18/90.61/82.4195.50/90.43/89.3195.23/91.64/91.8560.49
SafeTag90.44/88.16/89.5391.80/89.38/87.8689.68/83.76/86.3288.47/84.97/84.9380.84
SARCNet85.39/81.36/80.7286.71/81.42/82.5884.58/80.28/83.3373.50/71.25/70.6671.08
VLGuard99.72/90.88/96.2799.79/92.35/92.4799.51/87.71/96.6598.12/90.23/95.1695.85
VLSU86.86/81.12/83.2187.34/78.98/82.8285.51/76.53/81.8986.30/76.12/82.3373.58

SEFIR exceeds both probes in all 28 settings. Equally weighted mean gains are 5.48 points over Linear (range: 2.25-11.80) and 4.28 over MLP (range: 0.91-13.77). Among 56 comparisons, 22 gains are at least 5 points and four exceed 10: +10.18 on Llama-3.2-11B/VLSU and +11.80 on Qwen2-VL-7B/VLGuard over Linear; +10.03 on Qwen3-VL-8B/Hateful Memes and +13.77 on Qwen3-VL-8B/HoliSafe over MLP. SEFIR exceeds Guard + LoRA in 27 of 28 comparisons.

Q3. ... end-to-end latency ... peak memory ... hidden-state extraction and pooling ...

Answer: Figure 6 reports detector-head FLOPs. Appendix A.3 includes pooling in the added arithmetic cost and identifies hidden-state access and coordinate gathering as memory operations. We measured end-to-end detection latency and peak allocated memory on Falcon with and without SEFIR using Qwen3-VL-4B, RTX 5090, BF16, and batch size 1.

MetricWithout SEFIRWith SEFIRIncrease
Mean latency (ms/sample)65.4670.575.11 (7.81%)
Peak allocated memory (MiB)10,847.7510,988.84141.09 (1.30%)

Synchronized timing includes image decoding and preprocessing, backbone prefill, hidden-state collection, pooling, coordinate gathering, fusion, and classification. No representation disk cache is used. Model loading and autoregressive generation are excluded from timing. We have added these measurements to Table 6.

Q4. ... pixel-level visual adversarial perturbations ...

Answer: We evaluated SEFIR on VLGuard and SafeTag with Qwen3-VL-4B using five-step projected gradient descent (PGD-5) [3] at perturbation budgets of 2/255, 4/255, and 8/255. Clean scores are from Table 1; all values are Macro-F1 (%).

DatasetCleanPGD-5 2/255PGD-5 4/255PGD-5 8/255
VLGuard99.7297.7497.7495.54
SafeTag90.4489.9787.7885.85

At 8/255, SEFIR achieves 95.54% on VLGuard and 85.85% on SafeTag, respectively 4.18 and 4.59 points below clean.

References

[1] Kiela, D., Firooz, H., Mohan, A., Goswami, V., Singh, A., Ringshia, P., and Testuggine, D. (2020). The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes. NeurIPS.

[2] DrivenData. Hateful Memes: Phase 1. Official dataset page.

[3] Madry, A., Makelov, A., Schmidt, L., Tsipras, D., and Vladu, A. (2018). Towards Deep Learning Models Resistant to Adversarial Attacks. ICLR.

Reviewer_LmNDOpenReview ↗

Response

Thank you for your detailed response. I acknowledge that your rebuttal has partially addressed my concerns. My score is already positive, and I stand by it.

Thank you for your response.

Reviewer_F7Q8OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a safety classifier specifically for multimodal foundation models. They propose a fusion architecture to combine vision-encoder, projector and language-model layers. They report classification metrics on safety-tuned and guardrail models on seven safety datasets and four base models, showing interesting gains across datasets, and could lead to a new multimodal guardrail architecture in practice.

Critical strengths
  • Extensive evaluations across 7 datasets, 4 backbones and related baselines
  • Consistent gains across datasets compared to respect fair comparisons
Critical weaknesses

The proposed approach in the main results actually trains on each benchmark’s in-distribution train split, whereas their respective baselines are not. This comparison mixes the benefits of the proposed approach with in-domain training. The paper does report a fairer comparison via supervised probes but the margin of improvement then shrinks significantly. The fairer comparison should be reported as the main result, with in-domain training vs. zero-shot eval clearly labeled as such

Questions for authors
  1. Can you extend the supervised-probe comparison (final-layer linear probe and hidden-state MLP) to all four backbones and seven datasets, and add a supervised guard fine-tuned on the same training splits?
  2. Can you expand on the compositional risk harm in particular? If you subset eval sets to just the compositional harm data, how does the proposed approach perform?
3 comments

Thank you for your feedback.

Q1. ... matched supervision ... four backbones and seven datasets ... supervised probes ... fine-tuned guard ...

Answer: SEFIR targets safety detection on frozen MLLMs without repeating backbone-level safety alignment. To evaluate SEFIR's representation design, we compare it with supervised probes trained on the same target splits. Table 1 separately compares SEFIR with safety-trained systems.

We expanded Final-Layer Linear Probe (Linear) and Naive Hidden-State MLP (MLP) to all four backbones and seven datasets, using the same frozen backbones, target training splits, and evaluation protocol as SEFIR. These controls test final-layer and direct intermediate-state readout, respectively; individual design choices are analyzed in Table 3 and Appendix B.2. We also trained Guard + LoRA for three epochs using the same dataset partition protocol. We have added the Guard + LoRA comparison with Llama-3.2-11B SEFIR to Table 2 and its training settings to Appendix A.2. Each backbone cell reports SEFIR / Linear / MLP; Guard + LoRA reports one score per dataset. All values are Macro-F1 (%).

DatasetQwen3-VL-4BQwen3-VL-8BQwen2-VL-7BLlama-3.2-11BGuard + LoRA
Falcon88.31 / 84.88 / 83.7588.42 / 85.96 / 84.9585.96 / 80.55 / 83.0985.97 / 80.26 / 81.8442.30
Hateful Memes79.44 / 76.13 / 78.0079.88 / 73.42 / 69.8584.19 / 81.68 / 80.9168.20 / 62.64 / 64.8371.08
HoliSafe96.75 / 91.64 / 90.3896.18 / 90.61 / 82.4195.50 / 90.43 / 89.3195.23 / 91.64 / 91.8560.49
SafeTag90.44 / 88.16 / 89.5391.80 / 89.38 / 87.8689.68 / 83.76 / 86.3288.47 / 84.97 / 84.9380.84
SARCNet85.39 / 81.36 / 80.7286.71 / 81.42 / 82.5884.58 / 80.28 / 83.3373.50 / 71.25 / 70.6671.08
VLGuard99.72 / 90.88 / 96.2799.79 / 92.35 / 92.4799.51 / 87.71 / 96.6598.12 / 90.23 / 95.1695.85
VLSU86.86 / 81.12 / 83.2187.34 / 78.98 / 82.8285.51 / 76.53 / 81.8986.30 / 76.12 / 82.3373.58

Across all 28 backbone-dataset settings, SEFIR exceeds both supervised probes. Giving each setting equal weight, its mean Macro-F1 gains are 5.48 points over Linear and 4.28 over MLP. Taking the higher of the two reported probe scores in each setting as the reference yields a mean gain of 3.57 points, ranging from 0.91 to 7.32 points. SEFIR also scores higher than Guard + LoRA in 27 of 28 comparisons.

Q2. ... compositional harm ... subset evaluation ...

Answer: We evaluate HoliSafe's annotation-defined SSU category, where image and text are individually safe but unsafe in combination, alongside SSS cases, where both modalities and their combination are safe. The probes also receive target-dataset training. On two frozen backbones, we report SSU recall and false-negative rate (FNR), SSS specificity, and Macro-F1 on the SSU+SSS subset. All values are percentages.

BackboneMethodSSU Recall ↑SSU FNR ↓SSS Specificity ↑SSU+SSS Macro-F1 ↑
Qwen3-VL-4BSEFIR98.581.4293.8096.28
Qwen3-VL-4BLinear91.498.5194.5792.96
Qwen3-VL-4BMLP95.044.9695.3595.18
Qwen3-VL-8BSEFIR99.290.7196.9098.14
Qwen3-VL-8BLinear95.044.9679.8487.61
Qwen3-VL-8BMLP81.5618.4499.2289.98

SEFIR achieves the highest SSU recall and SSU+SSS Macro-F1 on both backbones. MLP is the stronger probe by SSU+SSS Macro-F1. Relative to MLP, SEFIR reduces SSU FNR from 4.96% to 1.42% on 4B and from 18.44% to 0.71% on 8B. SSS specificity decreases from 95.35% to 93.80% and from 99.22% to 96.90%, respectively.

The 1.10-point Macro-F1 gain on 4B accompanies a 3.54-percentage-point reduction in the SSU miss rate; on 8B, the corresponding changes are +8.16 and -17.73 points. On both backbones, SEFIR detects more compositional harms and achieves higher Macro-F1 across the unsafe and safe subsets.

Thank you for the rebuttal. I keep my score due to the main weakness from my review that the proposed approach actually trains on every benchmark's self-created train split. Ideally, public evaluation benchmarks should be held constant and training should only happen on relevant training data.

Thank you for your feedback.

Q1. ... benchmark splits ... target-benchmark training ...

Answer: For datasets with official splits, we retain the original splits, as documented in Appendix A.1. Tables 4–5 address the evaluation setting raised in your comment: we train the detectors only on Falcon, then evaluate them on fixed splits from the other benchmarks without training or tuning on those benchmarks. In Table 5, SEFIR outperforms both Falcon-trained probes on all four target benchmarks. Its mean Macro-F1 of 68.86% is 11.07 percentage points higher than that of the final-layer linear probe and 9.55 points higher than that of the hidden-state MLP. In the separate Guard + LoRA comparison reported in our previous response (Table 2), SEFIR keeps the MLLM backbone frozen and scores higher on six of seven datasets, including gains of 2.27 points on VLGuard, 7.63 on SafeTag, and 12.72 on VLSU.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.