- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a posterior reweighting framework to explain in-context learning (ICL) jailbreaks in multimodal large language models (MLLMs). It models safe and harmful generation as two latent behavioral modes and interprets in-context demonstrations as evidence that shifts the posterior preference between these modes. Based on this formulation, the authors derive scaling predictions with respect to the number, ratio, adversarial strength, and semantic diversity of harmful demonstrations. They also propose a risk-gated inference-time defense that selectively injects benign demonstrations for inputs in an intermediate-risk regime while bypassing benign inputs and directly refusing highly harmful ones.
The authors evaluate the proposed framework across multiple MLLMs, attack modalities, and datasets. On Qwen3-VL-8B, the proposed defense reduces the attack success rate (ASR) on MM-SafetyBench from 57.65% to 25.61% for image attacks, from 53.08% to 24.39% for text attacks, and from 58.53% to 28.00% for mixed-modal attacks, while reducing the average utility score only from 72.1 to 71.5. The paper also reports cross-dataset prediction results for its fitted scaling law, achieving an overall RMSE of 3.54% when transferring from AdvBench to MM-SafetyBench.
Critical strengths
- Relevant problem and broad empirical evaluation.
The paper addresses an important safety problem in MLLMs and evaluates attacks across multiple models, including open-weight and proprietary systems, as well as image, text, and mixed-modal settings. The experiments cover multiple benchmarks and examine several factors affecting jailbreak success. Evaluating both ASR and standard multimodal utility is particularly valuable because it avoids assessing defense effectiveness solely through attack mitigation.
- Promising robustness–utility trade-off.
The risk-gated defense is simple and does not require modifying the base model’s parameters. On Qwen3-VL-8B, it substantially reduces ASR across all three attack modalities while retaining most of the original utility. The comparisons with In-Context Defense, unconditional benign injection, AdaShield, and ECSO, together with additional evaluations under FigStep and detector-aware adaptive attacks, provide useful evidence of its practical potential.
Critical weaknesses
- The theoretical framework relies on strong assumptions that limit its mechanistic claims.
The posterior reweighting formulation is mathematically plausible, but its key assumptions are not sufficiently justified. In particular, Assumption (19) in Appendix A.1 assumes fixed harmful-output probabilities (p_h) and (p_s) for any target input under the two latent modes. This is a strong simplification because harmfulness can depend substantially on the target query, even when the underlying behavioral mode is unchanged. Similarly, the semantic-diversity result assumes that the redundancy term decreases as the number of semantic categories increases, which effectively builds the desired monotonic relationship into the formulation.
The empirical validation does not fully resolve these concerns. Selecting a two-component Gaussian mixture model using BIC on hidden-state representations does not establish that the two components correspond specifically to safe and harmful generation modes. Moreover, the paper does not directly estimate the posterior probability of the harmful mode or the per-demonstration evidence contributions. The cross-dataset logistic regression results are encouraging, but they establish predictive association more directly than they establish the proposed internal mechanism. The authors should distinguish more carefully between a phenomenological model that organizes empirical observations and a validated mechanistic account of MLLM behavior.
- The defense mechanism and its evaluation require clearer specification.
The defense uses a harmfulness detector described as estimating risk from the target input and its in-context demonstrations. However, Appendix A.4 describes a CLIP-ViT-Base-Patch32 classifier trained on GPT-5-generated harmful inputs and benign OK-VQA examples, without sufficiently clarifying how the detector represents text-only inputs, mixed-modal inputs, and the user-provided demonstration context. It is therefore difficult to determine whether the detector captures the contextual jailbreak risk that motivates the proposed method or primarily recognizes harmfulness in individual inputs.
The paper also states that the gating thresholds are optimized on the detector’s training set. Although the evaluation datasets are reported to be disjoint from training data, a held-out calibration protocol and modality-specific generalization results would strengthen confidence in the learned gating policy. Finally, because benign demonstration injection is already an established defense strategy, the paper should more clearly isolate the contribution of risk gating from the benefit of adding benign demonstrations, ideally with controlled comparisons under matched intervention and token budgets.
Questions for authors
-
Validation of the latent modes: How do the authors establish that the two Gaussian mixture components identified by BIC correspond to safe and harmful generation modes, rather than other sources of variation in hidden-state representations? Can the authors provide quantitative evidence linking cluster assignments to behavioral safety labels or directly estimate the context-dependent harmful-mode posterior?
-
Harmfulness detector and calibration: How exactly does the CLIP-ViT-Base-Patch32 detector process text-only and mixed-modal inputs, and does it explicitly encode the user-provided in-context demonstrations and their responses? How are the lower and upper gating thresholds learned, and can the authors report results using a separate validation set, with performance broken down by attack modality?
-
Contribution of risk gating: Can the authors provide a more controlled comparison that isolates the benefit of the learned gating policy from unconditional benign demonstration injection, including an oracle-gating upper bound or an equivalent intervention-budget comparison? This would help establish whether the proposed gating mechanism itself accounts for the observed robustness–utility improvement.
5 comments
Q3: On isolating the contribution of risk gating
We agree that this comparison is important and provide the intervention-budget comparison together with two further controls on Qwen3-VL-8B. In our framework, the effect of a fixed injection budget depends on the current posterior: the reduction is largest at log-odds and vanishes at both ends. The gate should therefore matter through which inputs it selects.
- Budget-matched unconditional injection. We inject into every input, with the number of demonstrations reduced to match our average budget (300 injected tokens per input). ASR is 34.73% at 71.7 utility, versus 25.61% and 71.5 for ours, so the gain does not come from a smaller budget.
- Random gating. We inject into the same fraction of inputs as our gate but select them at random. ASR is 45.85%, versus 25.61% for ours, so selecting the right inputs matters.
- Component decomposition. Table 15 separates the two branches of the gate: refusal only yields 46.79% ASR and injection only yields 34.89%, while the full gate reaches 25.61%. The improvement therefore does not come from the refusal branch alone.
W1 (theoretical assumptions): On Assumption (19) and the redundancy term
Query-dependent . Prop. 1 is stated for a fixed target input, so Assumption (19) can be relaxed to query-dependent without changing the result. Since depends only on the context, averaging over queries gives , which remains monotone in the posterior. All four scaling predictions therefore hold under this relaxation.
Redundancy term. That redundancy decreases with more categories is an explicit modeling choice, reflecting that same-category demonstrations carry correlated evidence. It is not guaranteed by construction: its prediction could have failed empirically, yet it is confirmed across seven MLLMs and three modalities (Table 5).
Dear Reviewer DCP1,
Thank you again for your careful reading and constructive suggestions. As the discussion period ends soon, we would like to kindly check whether our response has addressed your concerns, in particular the new analyses of the latent modes, the detector and calibration details, and the controlled comparisons isolating the contribution of the gate. The revised manuscript is uploaded with all changes highlighted in blue. If our response resolves your concerns, we would be grateful if you could reconsider your evaluation. We are happy to address any remaining questions.
Best regards,
The Authors
We thank the reviewer for the careful reading and constructive suggestions.
W1 (latent modes) & Q1: On validation of the latent modes
Whether an MLLM internally executes an explicit posterior update cannot be directly verified, which equally holds for prior latent-variable theories of ICL (Xie et al.; Wies et al.). Our framework is a predictive model built on a latent-variable abstraction, and we validate it through its observable consequences. As suggested, we add analyses that link the clusters to safety labels and estimate the context-dependent posterior.
Existing experiment. In Sec. 3.4, we collect hidden states of Qwen3-VL-8B under different in-context configurations and fit Gaussian mixture models; BIC selects two components (Table 1). We agree that BIC alone does not tell what the two components represent.
New analyses. (i) Linking clusters to safety labels. We compare each sample's component assignment with the harmful/safe label of its response given by the GPT-5 judge. The assignment is strongly associated with the output (purity 91.34%): samples in one component mostly yield harmful responses, and samples in the other mostly yield safe ones. The association is strong but not exact, which matches Prop. 1, where each mode produces harmful outputs with its own probability (ph>ps) and does not determine the output. (ii) Ruling out other sources of variation. We fix the target query and vary only the context from k=0 to k=8 harmful demonstrations. 79.22% of samples move from one component to the other. Since the query, image, and modality are held fixed, this shift cannot be explained by input content.
Conclusion. The two components are thus latent states that are shifted by the context and that govern the tendency toward harmful or safe outputs, which is how our framework defines the two generation modes. (iii) Estimating the posterior. The probability that a hidden state belongs to the harmful component then gives an estimate of Pr(Z=h∣C), which rises from 3.57% to 65.57% as k increases from 0 to 8, consistent with the posterior shift predicted by Eq. 9. We add these analyses to Sec. 3.4.
W2 & Q2: On detector design and threshold calibration
Detector. CLIP-ViT-B/32 serves as a multimodal feature extractor: text is encoded by the CLIP text encoder and images by the image encoder, separately for mixed-modal inputs. To handle CLIP's 77-token limit, each demonstration's request and response, as well as the target query, are scored segment-wise, with longer texts chunked. The segment scores si are aggregated additively into u(xt,C)=σ(∑ilogit(si)), mirroring the evidence accumulation in Eq. 7. The detector therefore encodes the demonstrations and their responses explicitly, and its score reflects risk accumulated over the context, not the target input alone.
Threshold learning. The thresholds (τℓ,τu) are produced by a two-layer network trained with Lgate (Eqs. 16–18, Sec. 4.2), with the detector and the target MLLM frozen. For differentiability, the hard gate is relaxed during training as σ((u−τ)/Tg) with Tg=0.1; hard thresholds are used at inference.
Held-out calibration. We re-learn the thresholds on a held-out split disjoint from both detector training and evaluation data. On Qwen3-VL-8B, this yields 25.13 / 25.51 / 26.99% ASR (image / text / mixed) with utility 71.5, comparable to the original 25.61 / 24.39 / 28.00% and 71.5. The learned gate thus generalizes across modalities. We will add these details to App. A.4.