- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper proposes a training-free method for detecting jailbreak attacks against large language models (LLMs). The authors observe that jailbreak attacks alter attention patterns, particularly at the assistant-header position. Based on this observation, they introduce the Routing Deviation Index (RDI), a metric that quantifies deviations in attention patterns from a reference distribution. They then use this metric to develop two methods: RISA-D for jailbreak detection and RISA-S for steering model behavior.
Critical strengths
-
Understanding the role of attention in LLM jailbreaking is an interesting research direction, as demonstrated by several prior works, particularly those studying attack mechanisms. Focusing on the assistant-header position is also an intuitive and potentially useful design choice. The paper offers a potentially useful perspective on detecting jailbreak attacks through internal model signals rather than relying solely on input-level features or output-level behavior.
-
The proposed approach is practical in that it operates during the prefill stage, potentially enabling jailbreak detection without additional model training or substantial inference-time overhead.
Critical weaknesses
-
Attention statistics show allocation, measure attention allocation rather than semantic information flow. Furthermore, relying on a specific harmful prompt distribution () raises concerns about robustness against unseen topics, prompt templates, and attack families. The whole approach depends on the distribution of ; would the same selected heads and thresholds work across distributions, or whether the detected routing pattern generalizes to unseen harmful topics, attack families, and templates...etc
-
Evaluation across only three models is insufficient to establish broader applicability across diverse model architectures, instruction-tuning strategies, and attack variants. Also, the paper does not sufficiently address whether an adaptive adversary could construct jailbreak attacks that evade detection by specifically targeting the RDI metric. Since the proposed method relies on measurable deviations in attention patterns, an adversary aware of the detection mechanism might attempt to preserve or manipulate these patterns while still achieving a successful jailbreak.
-
In Equation 2, the mean term appears to cancel out when deriving the quantities that ultimately matter. It is therefore unclear why this term is introduced in the first place. The authors should clarify the motivation for this formulation and explain whether the mean contributes any meaningful information to the resulting metric.
Questions for authors
-
How sensitive the reference distribution and detection performance is to variations in model size, instruction-tuning strategies, and attack types?
-
Have the authors evaluated the system against an adaptive adversary explicitly optimizing to minimize RDI deviation while maintaining jailbreak efficacy?
4 comments
Response (2/3)
Weakness 3: In Equation 2, the mean term appears to cancel out when deriving the quantities that ultimately matter. It is therefore unclear why this term is introduced in the first place. The authors should clarify the motivation for this formulation and explain whether the mean contributes any meaningful information to the resulting metric.
Weakness Response 3: We thank the reviewer for carefully pointing out this detail. The mean term is introduced so that attention heads with different magnitudes are comparable.
As stated in Section 4.2, we standardize each head so that different heads become comparable. Even on ordinary prompts, heads route very different magnitudes of attention, so a head's raw value mostly reflects its own normal operating level rather than the effect of the jailbreak.
The mean is introduced to remove exactly this head-specific level.
Eq. (2) then quantifies the routing deviation caused by a jailbreak by comparing, within the same head, the routing of a jailbreak prompt with that of its corresponding harmful prompt. The cancellation of the mean here is therefore intentional: a head's baseline level appears on both sides of the comparison and drops out, so Eq. (2) measures only what the jailbreak adds on top of the harmful prompt, with heads with different baseline levels remaining comparable.
Response (3/3)
Questions For Authors:
Question 1: How sensitive the reference distribution and detection performance is to variations in model size, instruction-tuning strategies, and attack types?
Response 1: In summary, RISA is insensitive to model size and the fine-tuning strategy in both its reference distribution and its detection. Attack type is the only factor that can matter, and only by shifting the selection of safety heads, an effect the leave-one-attack-family-out results show to be negligible.
Reference distribution. RISA relies on two types of reference: one from benign prompts and one from attack (harmful/jailbreak) prompts.
The benign reference sets the detection threshold and is unrelated to all three factors (model size, tuning strategy, attack type), since it uses only benign prompts on the model and follows the same procedure regardless of model size, tuning strategy or attack type (Weakness Response 2).
The attack reference is the only step that sees attacks and is used solely to select the routing heads.
Model size and tuning do not enter this step, as RISA itself involves no tuning.
Attack type can shift this selection and hence detection, but the leave-one-attack-family-out test shows this effect is negligible (Weakness Response 1 to VB6u, Appendix D.3).
Detection performance. Detection stays high across all three factors.
We evaluated model families of different sizes from different developers, which therefore differ in their tuning strategies, spanning 7B to 24B (Weakness Response 2).
On the two larger models, the TPR is 0.93-1.00 on all seven attack families.
Attack type affects detection only through head selection, so we test it directly with leave-one-attack-family-out.
The left-out family is still detected with AUROC in 20 of 21 settings (Appendix D.3, Table 13). Finally, an adaptive adversary that lowers RDI also strips the prompt of its attack strength, so detection performance is not compromised. (Response to Q2).
Question 2: Have the authors evaluated the system against an adaptive adversary explicitly optimizing to minimize RDI deviation while maintaining jailbreak efficacy?
Response 2:
We evaluate this adversary in Appendix D.4, where the attacker explicitly minimizes the RDI deviation against a frozen detector, and we now also measure whether the adapted prompts keep their jailbreak efficacy.
The results show that even when the attacker lowers RDI to evade detection, this adaptation also weakens their attack, so RISA still enforces safety while keeping the benign false positive rate unchanged.
Specifically, we have added three adaptive attacks to the paper.
Adaptive-Cipher keeps the encoding with the lowest , Adaptive-PAIR rewrites the goal for up to four rounds using as feedback, and Adaptive-GCG minimizes with gradients flowing through attention (Eq. (8)).
We report three quantities for the first 50 behaviors per family and model.
Harmful is the share of prompts whose response from the undefended model is judged harmful, which shows whether the adaptation preserves jailbreak efficacy.
Bypass & harmful is the share of prompts that are not detected and still elicit a harmful response, i.e., prompts that reach the model unsteered with a harmful outcome.
Each cell shows the original prompts → the adaptive prompts.
Table 3: Evaluation of RISA against adaptive adversaries.
| Model | Attack | RISA-D (%) | Harmful (%) | Bypass & harmful (%) | DSR with RISA-S (%) |
|---|---|---|---|---|---|
| Qwen-2.5-7B | Cipher | 100 → 100 | 4 → 0 | 0 → 0 | 98 → 100 |
| PAIR | 94 → 82 | 22 → 8 | 0 → 2 | 70 → 88 | |
| GCG | 100 → 100 | 18 → 4 | 0 → 0 | 96 → 96 | |
| Llama-3.1-8B | Cipher | 100 → 100 | 8 → 72 | 0 → 0 | 58 → 88 |
| PAIR | 66 → 64 | 22 → 16 | 0 → 0 | 96 → 96 | |
| GCG | 100 → 96 | 0 → 0 | 0 → 0 | 100 → 100 | |
| Gemma-2-9B | Cipher | 100 → 100 | 38 → 2 | 0 → 0 | 72 → 98 |
| PAIR | 84 → 24 | 22 → 6 | 0 → 2 | 100 → 98 | |
| GCG | 98 → 100 | 8 → 12 | 2 → 0 | 98 → 100 | |
| Mean | 93.6 → 85.1 | 15.8 → 13.3 | 0.2 → 0.4 | 87.6 → 96.0 |
Lower detection alone did not translate into a successful attack. Where adaptation lowers detection most (Gemma/PAIR, 84% → 24%), it also removes most of the jailbreak itself (harmful 22% → 6%), and the DSR stays at 98%. This means that the rewritten request carries neither the detection signals, nor jailbreak signals.
Dear Reviewer Saga,
Thank you again for your valuable feedback and for taking the time to review our submission. As the discussion deadline approaches, please let us know if you have further concerns for clarification.
We would greatly appreciate your continued engagement in the discussion.
Best regards,
Authors.
Response (1/3)
Thank you for the careful reading and valuable feedback.
Weaknesses:
Weakness Response 1: Firstly, we agree that attention allocation is not semantic information flow [c1, c2], while DeTAM [c3] demonstrates that it is a measurable signal tied to safety behavior. Therefore, RDI can measure how far the routing is displaced from normal refusal toward jailbroken behavior. Reviewer
B9AWalso recognizes that RDI is explicitly defined, uses only a small number of heads, and needs no additional classifier or backward pass.Secondly, the same heads and thresholds, selected on a specific prompt distribution, can generalize to unseen harmful topics, unseen attack families. Specially, we examine the detection on three unseen harmful topics in Table 1. With the heads and the threshold unchanged, RISA achieves perfective detection in all 27 combinations of model, source and attack family. Furthermore, we examine the detection on unseen attack families with different templates in
Appendix D.3. RISA also generalizes well to these settings. Reviewervb6ualso lists the leave-one-attack-out evaluation as a strength.Table 1: Detection on Unseen Harmful Topics.
[c1] Attention is not Explanation,NAACL,2019.
[c2] Attention is Not Only a Weight: Analyzing Transformers with Vector Norms,EMNLP,2020.
[c3] DeTAM:Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification,Findings of ACL,2025.
Weakness Response 2: We address the two parts in turn: model coverage, with new results on two additional model families, and the adaptive adversary. In summary, the evaluation shows that RISA transfers easily to more recent and larger models and stays robustness to an adaptive adversary.
(1) Model coverage. We evaluated RISA on two more recent and larger instruction-tuned models from families not covered in the paper,
Phi-4 (14B)andMistral-Small-3 (24B). Together with Qwen, Llama and Gemma, the evaluation now covers five model families from 7B to 24B, with all seven attack families evaluated on each.Table 2: Evaluation on Phi-4 and Mistral.
The same detection protocol transfers to both models, reaching a TPR of 0.93 to 1.00 across all seven families. RISA also improves DSR on both models, with a smaller gain on Phi-4 because its baseline is already strong. This follows a scaling law of the steering [c4]: steering efficacy decays with scale, so the norm-bounded default is undersized at 14B and 24B. Raising α to 6.0 can lift the weakest family on Mistral (ReNeLLM) from 0.41 to 0.61 without degrading benign inputs.
(2) Adaptive adversary. In our
response to Question 2below, we show that even when the attacker lowers RDI to evade detection, this adaptation also weakens their attack, so RISA still enforces safety while keeping the benign FPR unchanged.[c4] Scaling Laws for Activation Steering with Llama 2 Models and Refusal Mechanisms, arXiv 2507.11771, 2025.