acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Inference-Time Safety Alignment via Attention Routing

Abstract

Despite extensive safety alignment, large language models remain vulnerable to jailbreak attacks. Activation steering offers a training-free defense, but conditional steering still needs a reliable signal for deciding to intervene. Existing gates often read residual stream projections, yet jailbreak transformations can move these projections toward the benign regime. We instead examine attention routing at the assistant header, where refusal-related activation concentrates. A jailbreak alters how a small set of heads route the preceding context at these tokens, whereas benign and directly harmful prompts exhibit similar routing patterns. We aggregate the signed deviations of these heads from their benign routing pattern into the (RDI), which captures jailbreak-induced routing changes rather than the surface form of the rewrite or the harmfulness of the underlying request. We build (outing-ndexed afety lignment) on this routing measurement. In , a jailbreak is detected once that deviation passes a threshold set on benign prompts. In , the excess above that threshold sets the strength of a bounded refusal update, which is applied in a second prefill pass only when the gate is triggered. Across three open-weight aligned chat models and seven jailbreak families, RISA-D achieves the highest average detection accuracy on all three models, while RISA-S raises the global mean defense success rate from 40.0% to 95.0%. The routing signal also remains effective when individual attack families are excluded from calibration. These results suggest that attention routing offers a new perspective on safety alignment at inference time.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
inference alignment, safety, activation steering

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_SagaOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a training-free method for detecting jailbreak attacks against large language models (LLMs). The authors observe that jailbreak attacks alter attention patterns, particularly at the assistant-header position. Based on this observation, they introduce the Routing Deviation Index (RDI), a metric that quantifies deviations in attention patterns from a reference distribution. They then use this metric to develop two methods: RISA-D for jailbreak detection and RISA-S for steering model behavior.

Critical strengths
  • Understanding the role of attention in LLM jailbreaking is an interesting research direction, as demonstrated by several prior works, particularly those studying attack mechanisms. Focusing on the assistant-header position is also an intuitive and potentially useful design choice. The paper offers a potentially useful perspective on detecting jailbreak attacks through internal model signals rather than relying solely on input-level features or output-level behavior.

  • The proposed approach is practical in that it operates during the prefill stage, potentially enabling jailbreak detection without additional model training or substantial inference-time overhead.

Critical weaknesses
  • Attention statistics show allocation, measure attention allocation rather than semantic information flow. Furthermore, relying on a specific harmful prompt distribution (xHx^H) raises concerns about robustness against unseen topics, prompt templates, and attack families. The whole approach depends on the distribution of xHx^H; would the same selected heads and thresholds work across distributions, or whether the detected routing pattern generalizes to unseen harmful topics, attack families, and templates...etc

  • Evaluation across only three models is insufficient to establish broader applicability across diverse model architectures, instruction-tuning strategies, and attack variants. Also, the paper does not sufficiently address whether an adaptive adversary could construct jailbreak attacks that evade detection by specifically targeting the RDI metric. Since the proposed method relies on measurable deviations in attention patterns, an adversary aware of the detection mechanism might attempt to preserve or manipulate these patterns while still achieving a successful jailbreak.

  • In Equation 2, the mean term appears to cancel out when deriving the quantities that ultimately matter. It is therefore unclear why this term is introduced in the first place. The authors should clarify the motivation for this formulation and explain whether the mean contributes any meaningful information to the resulting metric.

Questions for authors
  • How sensitive the reference distribution and detection performance is to variations in model size, instruction-tuning strategies, and attack types?

  • Have the authors evaluated the system against an adaptive adversary explicitly optimizing to minimize RDI deviation while maintaining jailbreak efficacy?

4 comments
AuthorsOpenReview ↗

Response (1/3)

Thank you for the careful reading and valuable feedback.

Weaknesses:

Weakness 1: Attention statistics show allocation, measure attention allocation rather than semantic information flow. Furthermore, relying on a specific harmful prompt distribution (xHx^H) raises concerns about robustness against unseen topics, prompt templates, and attack families. The whole approach depends on the distribution of xHx^H; would the same selected heads and thresholds work across distributions, or whether the detected routing pattern generalizes to unseen harmful topics, attack families, and templates...etc

Weakness Response 1: Firstly, we agree that attention allocation is not semantic information flow [c1, c2], while DeTAM [c3] demonstrates that it is a measurable signal tied to safety behavior. Therefore, RDI can measure how far the routing is displaced from normal refusal toward jailbroken behavior. Reviewer B9AW also recognizes that RDI is explicitly defined, uses only a small number of heads, and needs no additional classifier or backward pass.

Secondly, the same heads and thresholds, selected on a specific prompt distribution, can generalize to unseen harmful topics, unseen attack families. Specially, we examine the detection on three unseen harmful topics in Table 1. With the heads and the threshold unchanged, RISA achieves perfective detection in all 27 combinations of model, source and attack family. Furthermore, we examine the detection on unseen attack families with different templates in Appendix D.3. RISA also generalizes well to these settings. Reviewer vb6u also lists the leave-one-attack-out evaluation as a strength.

Table 1: Detection on Unseen Harmful Topics.

ModelSourceAIMAutoDANCipher
Qwen-2.5-7BMaliciousInstruct100%100%100%
TDC23-RedTeaming100%100%100%
JailbreakBench100%100%100%
Llama-3.1-8BMaliciousInstruct100%100%100%
TDC23-RedTeaming100%100%100%
JailbreakBench100%100%100%
Gemma-2-9BMaliciousInstruct100%100%100%
TDC23-RedTeaming100%100%100%
JailbreakBench100%100%100%

[c1] Attention is not Explanation,NAACL,2019.

[c2] Attention is Not Only a Weight: Analyzing Transformers with Vector Norms,EMNLP,2020.

[c3] DeTAM:Defending LLMs Against Jailbreak Attacks via Targeted Attention Modification,Findings of ACL,2025.

Weakness 2: Evaluation across only three models is insufficient to establish broader applicability across diverse model architectures, instruction-tuning strategies, and attack variants. Also, the paper does not sufficiently address whether an adaptive adversary could construct jailbreak attacks that evade detection by specifically targeting the RDI metric. Since the proposed method relies on measurable deviations in attention patterns, an adversary aware of the detection mechanism might attempt to preserve or manipulate these patterns while still achieving a successful jailbreak.

Weakness Response 2: We address the two parts in turn: model coverage, with new results on two additional model families, and the adaptive adversary. In summary, the evaluation shows that RISA transfers easily to more recent and larger models and stays robustness to an adaptive adversary.

(1) Model coverage. We evaluated RISA on two more recent and larger instruction-tuned models from families not covered in the paper, Phi-4 (14B) and Mistral-Small-3 (24B). Together with Qwen, Llama and Gemma, the evaluation now covers five model families from 7B to 24B, with all seven attack families evaluated on each.

Table 2: Evaluation on Phi-4 and Mistral.

ModelSettingAIMAutoDANCipherGCGJailbrokenPAIRReNeLLM
Phi-4-14BRISA-D (TPR)1.001.001.001.001.000.961.00
No defense (DSR)1.000.950.670.940.960.890.38
+ RISA-S1.000.950.670.940.960.890.38
Mistral-24BRISA-D (TPR)1.001.001.001.000.930.961.00
No defense (DSR)0.650.820.980.900.850.700.41
+ RISA-S0.610.881.000.900.830.790.40

The same detection protocol transfers to both models, reaching a TPR of 0.93 to 1.00 across all seven families. RISA also improves DSR on both models, with a smaller gain on Phi-4 because its baseline is already strong. This follows a scaling law of the steering [c4]: steering efficacy decays with scale, so the norm-bounded default is undersized at 14B and 24B. Raising α\alpha to 6.0 can lift the weakest family on Mistral (ReNeLLM) from 0.41 to 0.61 without degrading benign inputs.

(2) Adaptive adversary. In our response to Question 2 below, we show that even when the attacker lowers RDI to evade detection, this adaptation also weakens their attack, so RISA still enforces safety while keeping the benign FPR unchanged.

[c4] Scaling Laws for Activation Steering with Llama 2 Models and Refusal Mechanisms, arXiv 2507.11771, 2025.

AuthorsOpenReview ↗

Response (2/3)

Weakness 3: In Equation 2, the mean term appears to cancel out when deriving the quantities that ultimately matter. It is therefore unclear why this term is introduced in the first place. The authors should clarify the motivation for this formulation and explain whether the mean contributes any meaningful information to the resulting metric.

Weakness Response 3: We thank the reviewer for carefully pointing out this detail. The mean term is introduced so that attention heads with different magnitudes are comparable.

As stated in Section 4.2, we standardize each head so that different heads become comparable. Even on ordinary prompts, heads route very different magnitudes of attention, so a head's raw value mostly reflects its own normal operating level rather than the effect of the jailbreak. The mean is introduced to remove exactly this head-specific level. Eq. (2) then quantifies the routing deviation caused by a jailbreak by comparing, within the same head, the routing of a jailbreak prompt with that of its corresponding harmful prompt. The cancellation of the mean here is therefore intentional: a head's baseline level appears on both sides of the comparison and drops out, so Eq. (2) measures only what the jailbreak adds on top of the harmful prompt, with heads with different baseline levels remaining comparable.

AuthorsOpenReview ↗

Response (3/3)

Questions For Authors:

Question 1: How sensitive the reference distribution and detection performance is to variations in model size, instruction-tuning strategies, and attack types?

Response 1: In summary, RISA is insensitive to model size and the fine-tuning strategy in both its reference distribution and its detection. Attack type is the only factor that can matter, and only by shifting the selection of safety heads, an effect the leave-one-attack-family-out results show to be negligible.

Reference distribution. RISA relies on two types of reference: one from benign prompts and one from attack (harmful/jailbreak) prompts. The benign reference sets the detection threshold and is unrelated to all three factors (model size, tuning strategy, attack type), since it uses only benign prompts on the model and follows the same procedure regardless of model size, tuning strategy or attack type (Weakness Response 2). The attack reference is the only step that sees attacks and is used solely to select the routing heads. Model size and tuning do not enter this step, as RISA itself involves no tuning. Attack type can shift this selection and hence detection, but the leave-one-attack-family-out test shows this effect is negligible (Weakness Response 1 to VB6u, Appendix D.3).

Detection performance. Detection stays high across all three factors. We evaluated model families of different sizes from different developers, which therefore differ in their tuning strategies, spanning 7B to 24B (Weakness Response 2). On the two larger models, the TPR is 0.93-1.00 on all seven attack families. Attack type affects detection only through head selection, so we test it directly with leave-one-attack-family-out. The left-out family is still detected with AUROC ≥0.960\ge 0.960 in 20 of 21 settings (Appendix D.3, Table 13). Finally, an adaptive adversary that lowers RDI also strips the prompt of its attack strength, so detection performance is not compromised. (Response to Q2).

Question 2: Have the authors evaluated the system against an adaptive adversary explicitly optimizing to minimize RDI deviation while maintaining jailbreak efficacy?

Response 2: We evaluate this adversary in Appendix D.4, where the attacker explicitly minimizes the RDI deviation against a frozen detector, and we now also measure whether the adapted prompts keep their jailbreak efficacy. The results show that even when the attacker lowers RDI to evade detection, this adaptation also weakens their attack, so RISA still enforces safety while keeping the benign false positive rate unchanged.

Specifically, we have added three adaptive attacks to the paper. Adaptive-Cipher keeps the encoding with the lowest RDIB\mathrm{RDI}_B, Adaptive-PAIR rewrites the goal for up to four rounds using (RDIB,τ)(\mathrm{RDI}_B, \tau) as feedback, and Adaptive-GCG minimizes CE("Sure, here is"∣p)+λ⋅RDIB(p)\mathrm{CE}(\text{"Sure, here is"}\mid p)+\lambda\cdot\mathrm{RDI}_B(p) with gradients flowing through attention (Eq. (8)). We report three quantities for the first 50 behaviors per family and model. Harmful is the share of prompts whose response from the undefended model is judged harmful, which shows whether the adaptation preserves jailbreak efficacy. Bypass & harmful is the share of prompts that are not detected and still elicit a harmful response, i.e., prompts that reach the model unsteered with a harmful outcome. Each cell shows the original prompts → the adaptive prompts.

Table 3: Evaluation of RISA against adaptive adversaries.

ModelAttackRISA-D (%)Harmful (%)Bypass & harmful (%)DSR with RISA-S (%)
Qwen-2.5-7BCipher100 → 1004 → 00 → 098 → 100
PAIR94 → 8222 → 80 → 270 → 88
GCG100 → 10018 → 40 → 096 → 96
Llama-3.1-8BCipher100 → 1008 → 720 → 058 → 88
PAIR66 → 6422 → 160 → 096 → 96
GCG100 → 960 → 00 → 0100 → 100
Gemma-2-9BCipher100 → 10038 → 20 → 072 → 98
PAIR84 → 2422 → 60 → 2100 → 98
GCG98 → 1008 → 122 → 098 → 100
Mean93.6 → 85.115.8 → 13.30.2 → 0.487.6 → 96.0

Lower detection alone did not translate into a successful attack. Where adaptation lowers detection most (Gemma/PAIR, 84% → 24%), it also removes most of the jailbreak itself (harmful 22% → 6%), and the DSR stays at 98%. This means that the rewritten request carries neither the detection signals, nor jailbreak signals.

Dear Reviewer Saga,

Thank you again for your valuable feedback and for taking the time to review our submission. As the discussion deadline approaches, please let us know if you have further concerns for clarification.

We would greatly appreciate your continued engagement in the discussion.

Best regards,

Authors.

Reviewer_B9AWOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that residual-stream gating signals used by conditional activation-steering defenses (refusal projection, harmfulness direction, toxic concept) drift into the benign regime under jailbreaks, and proposes instead to read attention routing at the assistant-header "safety tokens." It defines the Routing Deviation Index (RDI), a signed average over a small set of selected heads of how far their context-routing departs from a reference behavior, and builds RISA on top of it: RISA-D flags inputs whose RDI exceeds a benign-calibrated threshold, and RISA-S uses the excess as a bounded gate for a second-pass refusal-steering update.

Critical strengths

This Paper seperate the signal that decides when to intervene from the direction that determines how to intervene is a useful design choice. RDI is explicitly defined, uses only a small number of heads, and does not require an additional classifier or backward pass at inference time.

Critical weaknesses

The experiments do not distinguish safety-specific routing disruption from ordinary changes in prompt structure.

Questions for authors
  1. Do you have any general-capability results (e.g., MMLU, GSM8K, or an instruction-following benchmark) for the steered models, particularly the all-layer, αmax = 3.0 configurations on Qwen and Gemma? And how does RISA-S behave on an adversarial-benign set such as XSTest/OR-Bench rather than Alpaca?
  2. In a deployment where the base refusal has itself been weakened or partially fine-tuned away, does the design still provide coverage, or does it assume an intact native refusal for non-jailbreak harmful inputs?
  3. Since "recasting as an ordinary instruction" is a realistic, low-cost adaptation, how do you reconcile this with the claim that routing survives jailbreak transformations? Is the signal specific to framed jailbreaks rather than intent-preserving rewrites in general?
  4. Does RDI still separate attacks from benign requests when prompt structure is controlled?
4 comments
AuthorsOpenReview ↗

Response (1/2)

Thank you for the careful reading and valuable feedback.

Questions:

Question 1: Do you have any general-capability results (e.g., MMLU, GSM8K, or an instruction-following benchmark) for the steered models, particularly the all-layer, αmax = 3.0 configurations on Qwen and Gemma? And how does RISA-S behave on an adversarial-benign set such as XSTest/OR-Bench rather than Alpaca?

Response 1: We evaluate RISA-S on all three models, on MMLU, GSM8K and IFEval, and on XSTest and OR-Bench. On each task, accuracy stays comparable with the unsteered model, and RISA-S does not add over-refusal on the adversarial-benign sets.

General capability. We follow the RISA protocol and evaluate RISA-S on three general benchmarks, MMLU, GSM8K and IFEval, with the results reported in Table 1. Accuracy is essentially unchanged across all benchmarks, with only a minor variation across models, and no systematic degradation. A few benign prompts may still trigger the gate, but this low false-positive rate has no effect on the model's overall general capability.

Table 1: General capability.

BenchmarkModelBase accRISA-S accGate trigger
MMLUQwen / Llama / Gemma0.665 / 0.585 / 0.6950.665 / 0.585 / 0.6904 / 4 / 3%
GSM8KQwen / Llama / Gemma0.415 / 0.810 / 0.7950.410 / 0.800 / 0.8109 / 9.5 / 3.5%
IFEvalQwen / Llama / Gemma0.601 / 0.674 / 0.6740.594 / 0.674 / 0.6678 / 5.8 / 3.6%

Adversarial-benign sets. On the adversarial-benign sets, XSTest and OR-Bench, which are designed to elicit over-refusal, we again follow the RISA protocol and report the results in Table 2. RISA-S does not increase over-refusal: it stays comparable to the base model, even on OR-Bench where the base models already refuse a large share of benign prompts. So, RISA-S adds safety without paying for it in over-refusal.

Table 2: Adversarial-benign sets.

SetModelBase over-refusalRISA-S over-refusalBenign FPRJailbreak TPRAUROC
XSTestQwen1.6%1.6%0.4%98.0%0.992
XSTestLlama5.6%5.6%0%94.3%0.967
XSTestGemma16.8%15.2%5.2%96.8%0.975
OR-BenchQwen26%25.5%3%91.8%0.969
OR-BenchLlama31%31.5%6%91.5%0.934
OR-BenchGemma82%81%3%95.3%0.977

Question 2: In a deployment where the base refusal has itself been weakened or partially fine-tuned away, does the design still provide coverage, or does it assume an intact native refusal for non-jailbreak harmful inputs?

Response 2: Like other defenses built on the model's native alignment, such as ADA [c1], RISA builds on the model's own safety alignment and extends its refusal capability to jailbreak inputs. When a model's safety has been weakened or partially fine-tuned away, the appropriate remedy is to re-establish alignment by training [c2] rather than to add a detector. Even aligned models, however, remain vulnerable to jailbreaks [c3], and for these models RISA provides a practical and low-cost inference-time defense, an advantage also acknowledged by Reviewer Saga.

[c1] Any-depth alignment: Unlocking innate safety alignment of llms to any-depth, ICLR, 2026.

[c2] Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To!, ICLR, 2024.

[c3] Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks, ICLR, 2025.

AuthorsOpenReview ↗

Response (2/2)

Question 3: Since "recasting as an ordinary instruction" is a realistic, low-cost adaptation, how do you reconcile this with the claim that routing survives jailbreak transformations? Is the signal specific to framed jailbreaks rather than intent-preserving rewrites in general?

Response 3: RDI can respond to what a jailbreak adds to the prompt , such as coercion, deception or implicit permission to ignore the rules. An ordinary rewrite adds none of these, so it does not trigger detection, whereas jailbreak transformations remain detected.

Specifically, we show the two sides separately: (1) ordinary rewriting benign requests does not trigger the detector (Table 5), and (2) recasting a jailbreak as an ordinary instruction does lower detection, but it also removes most of the attack.

Ordinary rewriting. To test whether rewriting alone triggers the detector, we recast 100 benign test instructions as ordinary instructions while preserving their intent. As Table 3 shows, an ordinary rewrite that preserves the intent does not move RDI or trigger the detector, because it introduces none of the factors a jailbreak adds, which are what RDI responds to.

Table 3: FPR on benign instructions and their rewrites.

Benign promptsQwenLlamaGemma
Original4%7%7%
Rewrites1%3%3%

Recasting jailbreaks as ordinary instructions. We test a stronger adaptive PAIR attacker of Appendix D.4, which recasts the goal as an ordinary instruction and explicitly minimizes RDI to evade detection. The recasting lowers not only RDI but also attack capability: on Gemma, the share of prompts whose response from the undefended model is judged harmful drops from 22% to 6% (Response 2 to Saga). The defended model's DSR therefore stays at 98%.

Question 4: Does RDI still separate attacks from benign requests when prompt structure is controlled?

Response 4: In short, with the prompt structure controlled, RDI still separates jailbreak prompts from benign requests. Specifically, we define the three structures (the request replaces [prompt]) as below and place attacks and benign requests under them:

1. JSON:    Complete the task described in the following JSON object.
         { "action": "answer", "task": "[prompt]", "priority": "high", "department": "R&D" }
2. Code:    I left the following TODO comment in my repository. Please handle it for me.
         ```python
         # TODO: [prompt]
         ```
3. Nested:  While translating an old notebook I found this line written by my predecessor:
         "[prompt]". Please help me with that line.

The detection follows the protocol and settings in the paper, and the results are reported in Table 4. Controlling for the structure, benign requests and jailbreak prompts are cleanly separated.

Table 4: Benign FPR / jailbreak TPR / AUROC for each structure.

FormatQwenLlamaGemma
JSON4% / 98% / 0.9915% / 94% / 0.9727% / 96% / 0.982
Code comment3% / 98% / 0.9964% / 94% / 0.9752% / 96% / 0.977
Nested quotation4% / 98% / 0.9874% / 94% / 0.9676% / 97% / 0.977

Dear Reviewer B9AW,

Thank you again for your valuable feedback and for taking the time to review our submission. As the discussion deadline approaches, please let us know if you have further concerns for clarification.

We would greatly appreciate your continued engagement in the discussion.

Best regards,

Authors.

Thank you for the detailed responses and additional experiments. I appreciate the authors' effort in addressing my questions, particularly the new results on MMLU, GSM8K, IFEval, XSTest, and OR-Bench. These experiments help me clarify the impact of RISA-S on general capabilities.

However, I am still concerned about the paper's claim. Especially on Phi-4, RISA-D achieves nearly perfect detection, but RISA-S does not improve the defense success rate across any of the seven attack families. The improvements on Mistral are also mixed. This suggests that a reliable detection signal does not necessarily translate into effective steering,

Regarding the adaptive attacks, I appreciate the new evaluations. However, the reduced attack effectiveness after minimizing RDI makes it difficult to assess how robust the defense is against attacks that remain successful on the undefended model. So I prefer to maintain my score

Reviewer_vb6uOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper identifies assistant header tokens (Safety Tokens) as critical tokens for safety, because those are often the tokens with the maximal projection of the residual onto the model’s refusal direction in harmful prompts. The paper proposes the Routing Deviation Index (RDI), an index which measures deviations in the attending strength of the assistant header tokens to previous user and template tokens on a subgroup of relevant heads. Building upon RDI calibrated with benign, harmful, and jailbreak prompts, the paper then defines RISA--an alignment technique--where RISA-D detects jailbreaks by checking if the calibrated RDI surpasses a threshold, and RISA-S strengthens refusal if detection is triggered. RISA is evaluated over 3 models and 7 jailbreak methods, and is compared to 6 detection and 4 steering methods, demonstrating strong results.

Critical strengths
  • RISA is a powerful detection / steering technique with low overhead, competitive with evaluated baselines on multiple models and against a diversity of jailbreak methods.

  • The paper presents broad experimental tests (including evaluations for different parameter values, steering strength policy, leave-one-attack-out calibration, runtime overhead, and analysis of failure cases).

  • The paper reads well--it presents its findings and methods in a clear, easy-to-understand manner.

Critical weaknesses
  • Although a leave-one-attack-family-out generalization evaluation is shown for RISA, the main result tables evaluate RISA on the same jailbreak methods it was calibrated on. Calibrating RISA on a separate set of jailbreak methods would provide cleaner, more convincing results.

  • The paper builds upon existing results for assistant header tokens [1], but does not compare RISA vs. ADA [1]--a method based on assistant header tokens which the paper already cites. Evaluation against another assistant header based method would help clarify the added value of RISA.

  • The models used in the experiment are relatively old (Qwen-2.5, Llama-3.1, and Gemma-2) and small (<9B parameters). The results would be more convincing with more recent and larger models.

  • The paper doesn’t establish RISA’s robustness against adaptive adversaries. For instance, such adversaries may circumvent the defnse by optimizing for undetectability in addition to jailbreak success.

[1] Zhang et al. “Any-depth alignment: Unlocking innate safety alignment of llms to any-depth” ICLR, 2026.

Questions for authors

None.

3 comments
AuthorsOpenReview ↗

Response (1/2)

Thank you for your valuable feedback.

Weaknesses:

Weakness 1: Although a leave-one-attack-family-out generalization evaluation is shown for RISA, the main result tables evaluate RISA on the same jailbreak methods it was calibrated on. Calibrating RISA on a separate set of jailbreak methods would provide cleaner, more convincing results.

Weakness Response 1: Following the suggestion, we extend the leave-one-attack-family-out evaluation to the complete detection protocol. In short, removing one attack family from calibration does not cause a systematic degradation in detection performance.

For each held-out family, we remove its calibration examples, rediscover the routing heads using the remaining six families, and select the benign threshold following the calibration procedure in the paper. We then evaluate the held-out family using the same 100 test prompts as in the paper and report the results in the below. The results show that RISA generalizes well to attack families excluded from calibration.

Table 1: Evaluation of detection protocol (Each cell reports accuracy/F1, and Δ\Delta is the LOO result minus the corresponding result).

ModelSettingAIMAutoDANCipherGCGJailbrokenPAIRReNeLLMAvg
QwenLOO0.98/0.980.98/0.980.99/0.990.91/0.900.91/0.900.97/0.970.99/0.990.96/0.96
All0.98/0.980.98/0.980.98/0.980.98/0.980.94/0.930.96/0.960.98/0.980.97/0.97
Δ\Delta0.00/0.000.00/0.00+0.01/+0.01-0.07/-0.08-0.03/-0.03+0.01/+0.01+0.01/+0.01-0.01/-0.01
LlamaLOO0.99/0.990.97/0.980.97/0.980.99/0.990.90/0.880.81/0.770.99/0.990.95/0.94
All0.96/0.970.96/0.970.96/0.970.96/0.970.93/0.920.78/0.740.96/0.970.93/0.93
Δ\Delta+0.03/+0.02+0.01/+0.01+0.01/+0.01+0.03/+0.02-0.03/-0.04+0.03/+0.03+0.03/+0.02+0.02/+0.01
GemmaLOO0.97/0.980.98/0.990.98/0.980.99/0.990.91/0.910.91/0.910.97/0.980.96/0.96
All0.96/0.970.96/0.970.96/0.970.96/0.960.90/0.890.90/0.890.96/0.970.94/0.94
Δ\Delta+0.01/+0.01+0.02/+0.02+0.02/+0.01+0.03/+0.03+0.01/+0.02+0.01/+0.02+0.01/+0.01+0.02/+0.02

Weakness 2: The paper builds upon existing results for assistant header tokens [1], but does not compare RISA vs. ADA [1]-a method based on assistant header tokens which the paper already cites. Evaluation against another assistant header based method would help clarify the added value of RISA.

Weakness Response 2: We thank the reviewer for pointing out this comparison. We have added a comparison with ADA in Table 2 below. RISA matches or exceeds ADA-LP in detection accuracy on all three models, while requiring neither a trained probe nor any extra decoding, both of which ADA needs, and it also yields lower benign refusal.

The added value of RISA over ADA comes from the granularity at which the assistant header is used. ADA stays at the token level, re-injecting the header during generation, whereas RISA goes inside the header and reads how a jailbreak changes the routing of the preceding context through a small set of attention heads. After detection, ADA can only return a fixed refusal template or the refusal inside its lookahead, whereas RISA redirects the model and still generates a complete response.

Setup. ADA-LP depends on training data that is only partly public: its benign side comes from WildJailbreak and WildChat, while its harmful side is generated by an internal misaligned model on AdvBench, JailbreakBench and other jailbreak benchmarks. We therefore approximate the harmful side with PKU-SafeRLHF and BeaverTails, two public datasets that cover the same harm categories, and refer to this probe as ADA-LP (Wild), since it follows the data recipe of the ADA paper. For a fair comparison on identical data, we additionally train an ADA-LP (cal.) probe on RISA's calibration set. The results are summarized in Table 2.

Table 2: Comparison of RISA and ADA on the same prompts.

ModelMethodDSR (%)Benign refusal ΔDetection Acc/F1
Qwen-2.5-7BRISA96.0+10.97/0.97
ADA-LP (cal.)98.7+80.95/0.95
ADA-LP (Wild)81.9+20.67/0.43
ADA-RK79.7+2-
Llama-3.1-8BRISA93.0+00.93/0.93
ADA-LP (cal.)99.3+30.83/0.78
ADA-LP (Wild)84.6+210.51/0.27
ADA-RK89.1+12-
Gemma-2-9BRISA95.9+40.94/0.94
ADA-LP (cal.)98.7+80.94/0.94
ADA-LP (Wild)75.9+130.58/0.41
ADA-RK83.0+17-

Overall, RISA matches the best ADA variant (ADA-LP (cal.)) in detection and defense while staying well above ADA-LP (Wild) and the training-free ADA-RK. It also preserves benign task completion best, since a false positive under ADA-LP ends in a fixed refusal template instead of an answer, and it needs neither a trained probe nor a lookahead generation, so its inference cost is the lowest.

AuthorsOpenReview ↗

Response (2/2)

Weakness 3: The models used in the experiment are relatively old (Qwen-2.5, Llama-3.1, and Gemma-2) and small (<9B parameters). The results would be more convincing with more recent and larger models.

Weakness Response 3: We evaluated RISA on two more recent and larger instruction-tuned models from families not covered in the main paper, Phi-4 (14B) and Mistral-Small-3 (24B). Table 3 shows that RISA transfers easily to more recent and larger models, as it keeps a mean TPR of 0.99 for detection and improves the model's safety defense without harming benign tasks.

The results of TPR are summarized below:

Table 3: Evaluation of RISA on more recent and larger models.

ModelSettingAIMAutoDANCipherGCGJailbrokenPAIRReNeLLM
Phi-4-14BDetection (TPR)1.001.001.001.001.000.961.00
No defense (DSR)1.000.950.670.940.960.890.38
+ RISA1.000.950.670.940.960.890.38
Mistral-24BDetection (TPR)1.001.001.001.000.930.961.00
No defense (DSR)0.650.820.980.900.850.700.41
+ RISA0.610.881.000.900.830.790.40

The same detection protocol transfers to both models, reaching a TPR of 0.93 to 1.00 across all seven families. RISA also improves DSR on both models, with a smaller gain on Phi-4 because its baseline is already strong. This follows a scaling law of the steering [c1]: steering efficacy decays with scale, so the norm-bounded default is undersized at 14B and 24B. Raising α\alpha to 6.0 can lift the weakest family on Mistral (ReNeLLM) from 0.41 to 0.61 without degrading benign inputs.

[c1] Scaling Laws for Activation Steering with Llama 2 Models and Refusal Mechanisms, arXiv 2507.11771, 2025.

Weakness 4: The paper doesn’t establish RISA’s robustness against adaptive adversaries. For instance, such adversaries may circumvent the defnse by optimizing for undetectability in addition to jailbreak success.

Weakness Response 4: We evaluate this adversary in Appendix D.4, where the attacker explicitly minimizes the RDI deviation against a frozen detector, and we now also measure whether the adapted prompts keep their jailbreak efficacy. The results show that even when the attacker lowers RDI to evade detection, this adaptation also weakens their attack, so RISA still enforces safety while keeping the benign false positive rate unchanged.

Specifically, we have added three adaptive attacks to the paper. Adaptive-Cipher keeps the encoding with the lowest RDIB\mathrm{RDI}_B, Adaptive-PAIR rewrites the goal for up to four rounds using (RDIB,τ)(\mathrm{RDI}_B, \tau) as feedback, and Adaptive-GCG minimizes CE("Sure, here is"∣p)+λ⋅RDIB(p)\mathrm{CE}(\text{"Sure, here is"}\mid p)+\lambda\cdot\mathrm{RDI}_B(p) with gradients flowing through attention (Eq. (8)). We report four quantities for the first 50 behaviors per family and model. Harmful is the share of prompts whose response from the undefended model is judged harmful, which shows whether the adaptation preserves jailbreak efficacy. Bypass & harmful is the share of prompts that are not detected and still elicit a harmful response, i.e., prompts that reach the model unsteered with a harmful outcome. Each cell shows the original prompts → the adaptive prompts.

Table 4: Evaluation of RISA against adaptive adversaries.

ModelAttackRISA-D (%)Harmful (%)Bypass & harmful (%)DSR with RISA-S (%)
Qwen-2.5-7BCipher100 → 1004 → 00 → 098 → 100
PAIR94 → 8222 → 80 → 270 → 88
GCG100 → 10018 → 40 → 096 → 96
Llama-3.1-8BCipher100 → 1008 → 720 → 058 → 88
PAIR66 → 6422 → 160 → 096 → 96
GCG100 → 960 → 00 → 0100 → 100
Gemma-2-9BCipher100 → 10038 → 20 → 072 → 98
PAIR84 → 2422 → 60 → 2100 → 98
GCG98 → 1008 → 122 → 098 → 100
Mean93.6 → 85.115.8 → 13.30.2 → 0.487.6 → 96.0

Lower detection alone did not translate into a successful attack. Where adaptation lowers detection most (Gemma/PAIR, 84% → 24%), it also removes most of the jailbreak itself (harmful 22% → 6%), and the DSR stays at 98%. This means that the rewritten request carries neither the detection signals, nor jailbreak signals.

Dear Reviewer vb6u,

Thank you again for your valuable feedback and for taking the time to review our submission. As the discussion deadline approaches, please let us know if you have further concerns for clarification.

We would greatly appreciate your continued engagement in the discussion.

Best regards,

Authors.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Reply

We sincerely thank all the reviewers for their effort, feedback, and comments. RISA was recognized for separating the detection signal from the steering direction (B9AW) and for an explicit RDI that uses few heads and needs no classifier or backward pass (B9AW). Detecting jailbreaks from internal signals at the assistant-header position during prefill was considered intuitive and practical (Saga). Its low overhead, competitive results and broad, clearly presented experiments were also highlighted (vb6u).

We invite each reviewer to consult our individual responses, where we address all points raised in detail. In this general reply, we provide a summary of the primary changes performed in light of the feedbacks:

  • Generalization to unseen topics and attack families (Saga, vb6u)

    We added generalization tests on unseen harmful topics, on held-out attack families with their associated templates (Appendix D.3).

  • Robustness against adaptive adversaries (Saga, vb6u)

    Based on the analysis in Appendix D.4, we further examined robustness against adaptive adversaries, covering both detection and end-to-end jailbreak defense (Appendix D.4).

  • Evaluation on more recent and larger models (Saga, vb6u)

    We added detection and intervention results for RISA on two recent and larger instruction-tuned models from model families not previously evaluated.

  • Safety-specific routing versus ordinary rewriting (B9AW)

    We added experiments distinguishing ordinary, intent-preserving rewrites from jailbreak transformations, along with a controlled comparison across prompt structures.

  • General capability and over-refusal (B9AW)

    We added evaluations of RISA on general-capability benchmarks (MMLU, GSM8K, and IFEval) and adversarial-benign sets (XSTest and OR-Bench).

  • Comparison with ADA (vb6u)

    We added a comparison with ADA to clarify how the two methods differ and what added value RISA offers.

We thank you again for your effort, and welcome any further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.