acceptodds
Withdrawn from ICLR 2027

Amplify to Detect: Gradient-Guided Weight Interventions for Multimodal Jailbreak Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision–language models (VLMs) can be jailbroken by malicious intent conveyed through text, images, or their interaction. Detecting such inputs before harmful generation is therefore important, but detectors must also avoid falsely flagging benign requests. Existing hidden-state detectors use VLM representations as fixed features, leaving unclear which internal computations produce harmful–benign separation and whether these signals can be strengthened. We introduce Amplify to Detect, a frozen-model intervention for multimodal jailbreak detection. Our method places multiplicative scalers on output channels of VLM linear projections and uses first-order sensitivity of a paired harmful–benign representation-separation objective to identify influential channels. We amplify channels predicted to increase separation, then apply hidden-state detectors to the resulting representations. Across three open-weight VLMs and multiple detectors, targeted amplification generally improves malicious-intent detection under a stringent 5% false-positive-rate constraint. For HiddenDetect, TPR at this operating point increases by 18.8–57.2 percentage points across the three models. Direction ablations further show that reversing the predicted intervention direction reverses its effect, while random would decrease the performance. These findings suggest that safety-relevant hidden-state signals arise partly from identifiable internal computations and can be selectively strengthened for more reliable jailbreak detection.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Vision Language Model, Jailbreak Detection

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_mySuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes Amplify to Detect, a frozen-model intervention for multimodal jailbreak detection. It places multiplicative scalers on the output channels of a VLM’s linear projections and scores each channel by the first-order sensitivity of a paired harmful–benign refusal-margin objective to that scaler. The top positively scored channels are amplified by a factor 1+ε, and the resulting hidden states are fed to existing detectors without retraining the VLM. Across LLaVA-1.6, Qwen2.5-VL, and InternVL3, amplification improves AUROC in all cases and TPR@5% in nearly all; the largest gains are for the weak HiddenDetect baseline. Direction and selection ablations show that reversing the sign reverses the effect and that random or weight-magnitude selection does not help, while the refusal-margin score does.

Critical strengths

S1. A clean, well-motivated formulation. Casting channel selection as a signed first-order sensitivity of a harmful–benign refusal margin is simple, principled, and connects naturally to the SNIP/pruning literature. Scoring whole projection-output channels rather than individual weights is a sensible unit of attribution.

S2. The intervention is cheap and modular. The method keeps the VLM frozen, adds no trainable parameters to the backbone, and plugs into existing hidden-state detectors without changing their training or scoring. This makes it easy to adopt and a reasonable complement to detector design rather than a replacement.

S3. Informative and honest ablations. The signed-direction control, the matched-random and magnitude baselines, and the layer/strength sweeps isolate where the effect comes from and show it is not a generic perturbation. The Limitations section is unusually candid about what the method does and does not establish.

Critical weaknesses

W1. The headline gains may be an artifact of a weak base detector. The large improvements are concentrated on HiddenDetect, whose original TPR@5% is very low (18.9–40.4). A plain logistic-regression probe on the original representations already reaches 84.6–92.2 TPR@5%, higher than amplified HiddenDetect, and barely moves after amplification. So the intervention mostly lifts a weak detector toward the level of a trivial strong baseline, rather than advancing the state of the art; the paper should argue what amplification buys over simply using a better detector.

W2. The causal framing is overstated. The paper repeatedly claims safety signals can be “causally strengthened,” but all evidence is that amplifying a refusal-margin proxy raises a downstream detection score. The signed-direction result (reversing ε reverses the effect) is a mathematical consequence of the first-order expansion, not independent evidence of causality, and the paper itself concedes that improving the margin does not establish safer generation. The claims should be scoped to detection separability.

W3. The effect is model- and layer-dependent, undermining generality. On InternVL3 the intervention can reduce both AUROC and TPR@5%, and the gains depend strongly on the (ℓa, ℓd) pair, with the AUROC-optimal pair differing from the TPR-optimal one. Without a model-agnostic rule for choosing these, “consistent improvement” is hard to sustain.

W4. Narrow evaluation and no adaptive attack. Evidence comes from three open 7–8B models, one detector split, and 355 attribution pairs, with the 65 held-out pairs testing transfer only within the same source. There is no adaptive-attack evaluation and no measurement of benign task quality, false refusals, or inference cost. For a safety-detection method, the absence of an adaptive attacker that knows which channels are amplified is a significant gap.

W5. Formatting and reporting issues that should be fixed. In Table 1, the KCD entries overrun the column boxes. These same KCD TPR@5% figures have standard deviations larger than their means (9.67 ± 15.50, 22.56 ± 38.93, 33.07 ± 38.22), meaning the detector is close to random across the five seeds. Separately, the Figure 7(a) caption labels the amplification factor “g” while the text and equations use ε; the symbol should be made consistent.

Questions for authors

Please refer to weaknesses.

Reviewer_FrK6OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work presents Amplify to Detect, a method for improving jailbreak detection in vision-language models. Instead of directly using hidden states for detection, it identifies important channels through gradient-based scores and amplifies them to better separate harmful and benign inputs. Experiments on three VLMs and four detectors show improvements, especially under a low false positive rate of 5%.

Critical strengths
  1. Improving jailbreak detection with fewer false alarms is an important problem, especially for real-world applications.

  2. A simple and practical approach is proposed. It does not require retraining the original VLM and can work with different existing detectors.

  3. Results are promising, with large improvements for HiddenDetect across all three models.

Critical weaknesses
  1. Safety-related neurons, gradient-based importance scores, and internal model interventions have already been explored in previous studies. Combining these ideas through channel amplification may not be a strong enough technical contribution.

  2. Table 4 shows clear gains on text-only inputs, while performance on text-image inputs slightly decreases. This raises concerns about how well the approach addresses multimodal jailbreaks.

  3. Amplifying internal channels could affect general VLM performance or lead to more false refusals, but these effects are not evaluated.

  4. Already strong detectors receive only small improvements, while KCD shows large variance across runs. MCD even performs slightly worse on InternVL3.

Questions for authors
  1. How is this approach different from earlier work on safety neurons and gradient-based interventions? What is the main new contribution?

  2. Why are the improvements much larger for text-only inputs than for text-image inputs in Table 4? Could the authors provide more analysis?

  3. Does channel amplification affect general VLM performance or increase false refusals on harmless requests?

  4. Would this approach remain effective against adaptive jailbreak attacks designed to bypass the detector?

Reviewer_Az2sOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

The paper aims to identify and amplify channels that can help existing methods better detect jailbreaks. For each model, they use HiddenDetect’s refusal signal to find channels that help separate harmful and benign inputs, then amplify the top chosen channels for the detectors.

Critical strengths

S1. Their work emphasizes that the model layers themselves have the capability to flag jailbreaks, and we can find which channels help and use them to boost detection. However, the results show improvements mainly on weaker detectors like HiddenDetect. On stronger detectors, the method does not show substantial AUROC improvement.

S2. The ablations also support their channel selection method. Random channel selection has little impact, and reversing the scaling direction worsens detection.

Critical weaknesses

W1. The performance depends heavily on detector strength. Furthermore, the amplified HiddenDetect also remains below the unmodified LR probe on all three models, so we would gain more by switching detectors than by applying the channel scaling intervention.

W2. There should also be more analysis of what the selected layers and channels capture, and how or why these signals help flag jailbreaks.

Questions for authors

HiddenDetect itself performs poorly in the results on its own, so how reliable is its refusal signal for identifying these channels? and how can its own flagged layers improve it so much?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.