acceptodds
Under review as a conference paper at ICLR 2027

AttentionSwitch: Jailbreak Defense for LLMs via Joint Learning of Sparse Attention-Head Selection and Scaling

Abstract

Large language models have achieved remarkable performance across a wide range of tasks, yet they remain susceptible to jailbreak attacks that circumvent safety mechanisms and elicit harmful or unintended outputs. Although safety can be improved through model fine-tuning, such approaches can be computationally expensive and may degrade previously acquired capabilities through catastrophic forgetting. Recent jailbreak defenses have therefore explored targeted interventions on specific model components associated with unsafe behavior. However, existing approaches typically decouple two inherently related decisions: where to intervene and how to modify the selected components. Such sequential optimization may be suboptimal because the optimal intervention location can depend on the intervention applied there, and vice versa. In this work, we introduce AttentionSwitch, the first jailbreak defense to jointly optimize intervention sites and strengths within a frozen LLM. Specifically, AttentionSwitch learns a sparse set of attention heads together with query-conditioned multiplicative scaling factors, allowing the outputs of the selected heads to be either attenuated or amplified without altering the directions of their representations. Across popular language models and jailbreak attacks, AttentionSwitch outperforms existing defenses, with particularly strong results when training data is scarce.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI Security, LLM Safety, Jailbreak Attacks

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_1rg8OpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AttentionSwitch, a jailbreak defense that jointly learns which attention heads to intervene on and how strongly to scale their outputs in a frozen LLM. The design is motivated by the claim that existing defenses select intervention sites before optimizing the intervention, even though the optimal site and strength are coupled. Across five models and nine attack types, the authors report higher attack rejection than SFT, LoRA-SFT, ASGuard, and DDPO while better preserving benign behavior, although adaptive attacks reduce the margin.

Critical strengths
  • Jointly formulates head selection and scaling, addressing the coupling problem caused by diagnose-then-intervene pipelines.
  • Whole-head scaling is simple, direction-preserving, and compatible with a frozen LLM.
  • Comprehensive evaluation: five models, nine attacks, multiple baselines, and separate utility and over-refusal metrics.
Critical weaknesses

The defense relies on jailbreak prompts during training, leaving robustness to truly unseen attack families unclear. The harmful training set is drawn equally from Plain Harmful, Human-Crafted, and GCG examples, so the model is exposed to representative jailbreak prompts even though six attack types are held out for evaluation. The paper does not isolate how much of the gain comes from learning a general refusal mechanism versus fitting the jailbreak distribution, for example by training only on Plain Harmful prompts or on benign data alone. This concern is reinforced by the adaptive-attack results: average Attack Rejection Rate drops from 97.62% under non-adaptive GCG to 86.80% under Adaptive GCG, and from 92.26% under non-adaptive PAIR to 77.60% under Adaptive PAIR. These substantial drops suggest that the learned head selection and scaling are partly attack-specific and may not transfer to attackers that explicitly optimize against the defense or to entirely unknown attack categories.

Questions for authors
  1. How much of AttentionSwitch's robustness comes from the jailbreak prompts used in training, and would it remain effective on entirely unseen attack families?
7 comments
AuthorsOpenReview ↗

Generalization to Unseen Attack Families

We thank the reviewer for the constructive feedback. To directly address the concern about generalization to unseen attack families, we conducted additional experiments training AttentionSwitch separately on Plain Harmful (PH)-only, Human-Crafted (HC)-only, and GCG-only harmful examples.

Each setting uses 30 training samples (15 harmful + 15 benign), with identical training settings and a 20% head budget. We evaluate all five models across nine attack categories: Plain Harmful (PH), Human-Crafted (HC), GCG, PEZ, UAT, AutoPrompt, GBDA, GCG-M, and PAIR, with 235 test prompts per attack category (2,115 harmful prompts in total per model), alongside 500 benign prompts for BPR and 250 deceptive-benign prompts to evaluate over-refusal (ORR).

Importantly, PH-only training contains neither optimization-based jailbreak suffixes/prefixes nor human-crafted jailbreak templates in its training examples. Nevertheless, it achieves 96.44% average ARR, 98.52% BPR, and 10.32% ORR, closely matching the original mixed PH+HC+GCG training (96.58% ARR, 97.68% BPR, and 10.08% ORR). HC-only and GCG-only training also achieve strong performance, with 98.32% ARR, 97.00% BPR, and 11.12% ORR for HC-only, and 97.18% ARR, 97.96% BPR, and 11.44% ORR for GCG-only.

Table R1: Attack Rejection Rate (ARR), Benign Pass Rate (BPR), and Over-Refusal Rate (ORR) across nine attack families under different training settings (ntrain=30n_{\mathrm{train}}=30). PH-only, HC-only, and GCG-only indicate that the harmful training examples are drawn only from Plain Harmful, Human-Crafted, and GCG attacks, respectively. All values are percentages. Arrows indicate the preferred direction of each metric.

ModelTrainingARR ↑ (%)BPR ↑ (%)ORR ↓ (%)
Llama-3.1-8BPH-only98.7299.0010.80
HC-only99.9598.6015.20
GCG-only98.8798.8014.80
DeepSeek-7BPH-only93.4898.6016.00
HC-only97.4995.4016.80
GCG-only95.9397.2016.40
OpenChat-3.5PH-only93.1099.207.20
HC-only94.1496.405.20
GCG-only92.9699.007.60
Vicuna-13BPH-only98.7796.2012.00
HC-only100.0095.6011.20
GCG-only99.2995.2011.60
Gemma-3-27BPH-only98.1199.605.60
HC-only100.0099.007.20
GCG-only98.8799.606.80

For clarity, we provide the complete per-attack results with standard errors in the following comment.

AuthorsOpenReview ↗

Generalization to Unseen Attack Families (Detailed Results)

Table R2: Detailed per-attack results corresponding to Table R1, including standard errors.

ModelTrainPHHCGCGPEZUATAutoPromptGBDAGCG-MPAIR
Llama-3.1-8BPH-only100 ± 097.87 ± 0.9497.87 ± 0.9498.72 ± 0.7397.87 ± 0.9499.15 ± 0.60100 ± 098.72 ± 0.7398.30 ± 0.84
HC-only100 ± 0100 ± 0100 ± 0100 ± 0100 ± 099.57 ± 0.42100 ± 0100 ± 0100 ± 0
GCG-only100 ± 094.04 ± 1.54100 ± 0100 ± 0100 ± 099.57 ± 0.42100 ± 099.57 ± 0.4296.60 ± 1.18
DeepSeek-7BPH-only100 ± 093.19 ± 1.6492.77 ± 1.6992.77 ± 1.6994.04 ± 1.5494.47 ± 1.4995.32 ± 1.3894.04 ± 1.5484.68 ± 2.35
HC-only100 ± 098.72 ± 0.7396.17 ± 1.2597.45 ± 1.0399.15 ± 0.6094.04 ± 1.5497.45 ± 1.0396.17 ± 1.2598.30 ± 0.84
GCG-only100 ± 089.79 ± 1.9897.02 ± 1.1199.15 ± 0.6099.57 ± 0.4297.87 ± 0.9499.57 ± 0.4297.02 ± 1.1183.40 ± 2.43
OpenChat-3.5PH-only97.87 ± 0.9485.11 ± 2.3294.47 ± 1.4996.60 ± 1.1894.89 ± 1.4494.47 ± 1.4993.62 ± 1.5991.49 ± 1.8289.36 ± 2.01
HC-only99.15 ± 0.6095.74 ± 1.3291.49 ± 1.8294.47 ± 1.4994.47 ± 1.4993.19 ± 1.6491.49 ± 1.8293.19 ± 1.6494.04 ± 1.54
GCG-only95.74 ± 1.3287.66 ± 2.1593.62 ± 1.5996.60 ± 1.1895.32 ± 1.3894.89 ± 1.4493.19 ± 1.6492.34 ± 1.7387.23 ± 2.18
Vicuna-13BPH-only100 ± 097.02 ± 1.1195.32 ± 1.3899.57 ± 0.42100 ± 098.72 ± 0.73100 ± 099.15 ± 0.6099.15 ± 0.60
HC-only100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0
GCG-only100 ± 097.45 ± 1.03100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0100 ± 096.17 ± 1.25
Gemma-3-27BPH-only98.72 ± 0.7398.72 ± 0.7397.45 ± 1.0399.15 ± 0.6098.72 ± 0.7396.17 ± 1.2598.72 ± 0.7397.87 ± 0.9497.45 ± 1.03
HC-only100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0100 ± 0
GCG-only99.57 ± 0.4299.57 ± 0.42100 ± 098.72 ± 0.73100 ± 098.30 ± 0.84100 ± 095.74 ± 1.3297.87 ± 0.94
AuthorsOpenReview ↗

Adaptive Robustness to Unseen Attack Families (Plain Harmful-Trained Defenses)

Adaptive attacks assume an unusually strong attacker with full access to the defense, so some degradation in defense performance is expected. The main question is whether this degradation is comparable to that of existing defenses. To investigate this, we compared AttentionSwitch with other baselines under adaptive attacks using the same training and evaluation settings.

All defenses were trained with n=30 samples (15 Plain Harmful and 15 benign) and evaluated on 235 unseen prompts per attack. Importantly, the harmful training prompts contained no jailbreak modifications, so the evaluated attack families were tested in a zero-shot setting. AttentionSwitch consistently outperforms the baselines under adaptive attacks, with the exception of gemma under adaptive GCG, where it performs nearly identically to DDPO (97.02% vs. 97.45%). On average, AttentionSwitch achieves 85.23% ARR, compared with 79.45% for DDPO and 65.70% for ASGuard.

Table R3: Adaptive Attack Rejection Rate (ARR %, higher is better). Values are reported as percentages (rate ± standard error).

AttackDefenseLlama-3.1-8BDeepSeek-7BOpenChat-3.5Vicuna-13BGemma-3-27B
Adaptive GCGNone90.64 ± 1.9058.30 ± 3.2235.32 ± 3.1271.91 ± 2.9391.49 ± 1.82
ASGuard91.06 ± 1.8660.00 ± 3.2062.55 ± 3.1673.62 ± 2.8794.47 ± 1.49
DDPO95.32 ± 1.3867.23 ± 3.0679.15 ± 2.6587.23 ± 2.1897.45 ± 1.03
AttentionSwitch (Ours)98.30 ± 0.8471.49 ± 2.9587.66 ± 2.1590.21 ± 1.9497.02 ± 1.11
Adaptive PAIRNone71.49 ± 2.9537.87 ± 3.1630.21 ± 3.0047.23 ± 3.2657.87 ± 3.22
ASGuard68.94 ± 3.0238.72 ± 3.1851.49 ± 3.2646.38 ± 3.2569.79 ± 3.00
DDPO73.19 ± 2.8966.38 ± 3.0870.64 ± 2.9776.60 ± 2.7681.28 ± 2.54
AttentionSwitch (Ours)78.30 ± 2.6972.34 ± 2.9275.32 ± 2.8192.34 ± 1.7389.36 ± 2.01

Benign Pass Rate (BPR) and Over-Refusal Rate (ORR) are evaluated on benign datasets and therefore do not change across attack types. The BPR and ORR values are the same as those reported in Table R1.

Reviewer_kSFnOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses the problem that existing attention-head-level jailbreak defenses typically first determine which attention heads to intervene on and then optimize the intervention, thereby ignoring the interdependence between intervention location and intervention strength and potentially limiting defense effectiveness. The authors propose AttentionSwitch, which selects a small number of attention heads via Gumbel-Softmax and Top-K mechanisms and dynamically generates scaling coefficients conditioned on each head's Query representation to amplify or suppress attention head outputs. The method jointly optimizes head selection and scaling parameters while keeping the pretrained LLM weights frozen, unifying attention head selection and dynamic scaling into a joint optimization framework for parameter-efficient jailbreak defense. Experiments on 5 LLMs and 9 attack types show that the method is particularly suitable for few-shot training scenarios, improving attack refusal rates while largely preserving normal task capability.

Critical strengths
  1. Strong experimental results with broad validation. The paper uses 5 LLMs and 9 attack types and tests few-shot training settings.
  2. Low parameter and runtime overhead. The model remains frozen and the additional controller has very few parameters. The authors report that average generation latency increases from 196.93 ms without defense to 202.07 ms, an increase of only about 5.14 ms, which is a valuable advantage for practical deployment.
  3. The problem is clearly defined, and the structure and method description are relatively clear.
Critical weaknesses
  1. The core methodological novelty is limited and mainly a combination of existing techniques. Applying joint optimization to jailbreak defense and designing a Query-conditioned positive scalar attention head scaling scheme constitutes reasonable incremental innovation.
  2. The ablation studies are insufficient to fully demonstrate the independent contributions of joint optimization and dynamic scaling, especially lacking comparisons with fairer non-joint optimization strategies.
  3. Although the overall experimental results are good, robustness under adaptive attacks remains insufficient, and direct comparisons with other defense methods under identical conditions are missing.
  4. Insufficient justification of dynamic scaling. The paper lacks controlled comparisons against fixed scalar scaling and other conditioning strategies, and the claimed advantage of direction-preserving scaling is not adequately supported by empirical or mechanistic analysis.
Questions for authors
  1. Is joint optimization truly superior to the two-stage "select first, intervene later" approach? Comparisons with random selection, gradient-importance selection, and two-stage optimization under the same objective are recommended.
  2. What advantages does Query-conditioned dynamic scaling offer over fixed scaling? Does the direction-preserving constraint actually matter? What are the respective contributions of the three training stages?
  3. How does the method perform in direct comparison with ASGuard, DDPO, and other methods under identical attack conditions? What causes the notable performance degradation of some models under adaptive attacks?
5 comments
AuthorsOpenReview ↗

Weaknesses 2 and 4. Additional Ablations to Isolate the Contributions of the Individual Components

We thank the reviewer for the detailed feedback. In our paper (Figure 3), we provide an ablation showing that scaling all attention heads (100%) or sequentially selecting a sparse subset (20%) using the previously proposed ASGuard method and then scaling those heads underperforms compared with our proposed joint head selection and scaling. Specifically, our method achieves 96.58% ARR and 97.68% BPR, compared with 92.26% ARR and 97.36% BPR when scaling all heads, and 94.73% ARR and 97.52% BPR when using ASGuard-selected heads.

We have performed additional ablations, as requested by the reviewer, to isolate the contributions of the individual components. We test four different versions of AttentionSwitch to isolate the effects of joint head selection and scaling and dynamic scaling factors. All variants use the same training data: 30 training examples (15 harmful and 15 benign), with 20% of attention heads selected for scaling. We evaluate all five models across nine attack categories: Plain Harmful (PH), Human-Crafted (HC), GCG, PEZ, UAT, AutoPrompt, GBDA, GCG-M, and PAIR, with 235 test prompts per attack category (2,115 harmful prompts in total per model), alongside 500 benign prompts for BPR and 250 deceptive-benign prompts to evaluate over-refusal (ORR).

The table below shows the results. Joint + Dynamic is our main proposed method. Replacing joint optimization with sequential optimization, dynamic scaling with static scaling, or both consistently reduces defense performance compared with our proposed approach across all five models.

Averaged across all five models, Joint + Dynamic achieves 96.58% ARR and 97.68% BPR. In comparison, Sequential + Dynamic achieves 92.65% ARR and 97.36% BPR, Joint + Static achieves 82.87% ARR and 97.60% BPR, and Sequential + Static achieves 80.78% ARR and 97.32% BPR. Thus, our proposed method improves ARR by 3.92, 13.71, and 15.80 percentage points, respectively, while maintaining similar BPR.

Table R1: Joint versus sequential head selection and dynamic versus static head scaling. Values are reported as percentages (rate ± standard error). Arrows indicate the preferred direction of each metric.

ModelMethodARR ↑ (%)BPR ↑ (%)ORR ↓ (%)
Llama-3.1-8BJoint + Dynamic (Ours)99.57 ± 0.1498.00 ± 0.6313.20 ± 2.14
Sequential + Dynamic98.82 ± 0.2497.60 ± 0.6817.60 ± 2.41
Joint + Static98.11 ± 0.3097.80 ± 0.6614.00 ± 2.19
Sequential + Static98.01 ± 0.3097.80 ± 0.6613.20 ± 2.14
DeepSeek-7BJoint + Dynamic (Ours)92.10 ± 0.5997.60 ± 0.6817.60 ± 2.41
Sequential + Dynamic87.04 ± 0.7397.40 ± 0.7118.40 ± 2.45
Joint + Static73.66 ± 0.9697.80 ± 0.6618.80 ± 2.47
Sequential + Static71.11 ± 0.9997.20 ± 0.7418.00 ± 2.43
OpenChat-3.5Joint + Dynamic (Ours)91.58 ± 0.6098.20 ± 0.594.40 ± 1.30
Sequential + Dynamic86.24 ± 0.7598.00 ± 0.639.60 ± 1.86
Joint + Static56.26 ± 1.0897.20 ± 0.745.60 ± 1.45
Sequential + Static53.05 ± 1.0997.40 ± 0.715.20 ± 1.40
Vicuna-13BJoint + Dynamic (Ours)99.76 ± 0.1195.40 ± 0.9410.80 ± 1.96
Sequential + Dynamic94.61 ± 0.4995.20 ± 0.9619.60 ± 2.51
Joint + Static90.07 ± 0.6595.80 ± 0.9012.40 ± 2.08
Sequential + Static87.90 ± 0.7195.00 ± 0.9712.80 ± 2.11
Gemma-3-27BJoint + Dynamic (Ours)99.86 ± 0.0899.20 ± 0.404.40 ± 1.30
Sequential + Dynamic96.55 ± 0.4098.60 ± 0.536.80 ± 1.59
Joint + Static96.22 ± 0.4199.40 ± 0.356.00 ± 1.50
Sequential + Static93.81 ± 0.5299.20 ± 0.405.60 ± 1.45
AuthorsOpenReview ↗

Weakness 4: Ablation of Scalar Scaling (Direction-Preserving Head Scaling)

As requested by the reviewer, we compared our proposed scalar attention-head scaling (which uses a single positive scaling factor to preserve the direction of the attention-head activation and the relative ratios of its components, affecting only its magnitude) with a vector scaling variant (which independently scales each dimension, changing the direction of the head activation).

Both variants jointly optimize head selection and scaling, intervene on 20% of attention heads, and use identical training and evaluation settings with n=30 samples (15 harmful and 15 benign). We evaluate five models using 2,115 harmful prompts across nine attack families, 500 benign prompts, and 250 deceptive-benign prompts per model.

Our proposed scalar scaling consistently achieves a higher benign pass rate (BPR) and lower over-refusal rate (ORR) across all five models, with average BPR of 97.68% versus 95.88% and ORR of 10.08% versus 12.72%, while maintaining approximately similar Attack Rejection Rate (ARR of 96.57% versus 96.21%).

Table R2: Scalar versus vector attention-head scaling. Results are percentages (rate ± standard error).

ModelScalingARR ↑ (%)BPR ↑ (%)ORR ↓ (%)
Llama-3.1-8BScalar (Ours)99.57 ± 0.1498.00 ± 0.6313.20 ± 2.14
Vector99.53 ± 0.1596.60 ± 0.8114.00 ± 2.19
DeepSeek-7BScalar (Ours)92.10 ± 0.5997.60 ± 0.6817.60 ± 2.41
Vector91.68 ± 0.6096.20 ± 0.8618.40 ± 2.45
OpenChat-3.5Scalar (Ours)91.58 ± 0.6098.20 ± 0.594.40 ± 1.30
Vector91.63 ± 0.6097.40 ± 0.716.00 ± 1.50
Vicuna-13BScalar (Ours)99.76 ± 0.1195.40 ± 0.9410.80 ± 1.96
Vector98.20 ± 0.2993.60 ± 1.0915.60 ± 2.29
Gemma-3-27BScalar (Ours)99.86 ± 0.0899.20 ± 0.404.40 ± 1.30
Vector100.00 ± 0.0095.60 ± 0.929.60 ± 1.86
AuthorsOpenReview ↗

Weakness 3. Additional Adaptive Robustness

Adaptive attacks assume an unusually strong attacker with full access to the defense, so some degradation in defense performance is expected. The main question is whether this degradation is comparable to that of existing defenses. As requested by the reviewer, we compared AttentionSwitch with other baselines under adaptive attacks using the same training and evaluation settings.

All defenses were trained with n=30 samples (15 Plain Harmful and 15 benign) and evaluated on 235 unseen prompts per attack. Importantly, the harmful training prompts contained no jailbreak modifications, so the evaluated attack families were tested in a zero-shot setting. AttentionSwitch consistently outperforms the baselines under adaptive attacks, with the exception of gemma under adaptive GCG, where it performs nearly identically to DDPO (97.02% vs. 97.45%). On average, AttentionSwitch achieves 85.23% ARR, compared with 79.45% for DDPO and 65.70% for ASGuard.

Table R3: Adaptive Attack Rejection Rate (ARR %, higher is better). Values are reported as percentages (rate ± standard error).

AttackDefenseLlama-3.1-8BDeepSeek-7BOpenChat-3.5Vicuna-13BGemma-3-27B
Adaptive GCGNone90.64 ± 1.9058.30 ± 3.2235.32 ± 3.1271.91 ± 2.9391.49 ± 1.82
ASGuard91.06 ± 1.8660.00 ± 3.2062.55 ± 3.1673.62 ± 2.8794.47 ± 1.49
DDPO95.32 ± 1.3867.23 ± 3.0679.15 ± 2.6587.23 ± 2.1897.45 ± 1.03
AttentionSwitch (Ours)98.30 ± 0.8471.49 ± 2.9587.66 ± 2.1590.21 ± 1.9497.02 ± 1.11
Adaptive PAIRNone71.49 ± 2.9537.87 ± 3.1630.21 ± 3.0047.23 ± 3.2657.87 ± 3.22
ASGuard68.94 ± 3.0238.72 ± 3.1851.49 ± 3.2646.38 ± 3.2569.79 ± 3.00
DDPO73.19 ± 2.8966.38 ± 3.0870.64 ± 2.9776.60 ± 2.7681.28 ± 2.54
AttentionSwitch (Ours)78.30 ± 2.6972.34 ± 2.9275.32 ± 2.8192.34 ± 1.7389.36 ± 2.01
Reviewer_y1uLOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AttentionSwitch, an inference-time jailbreak defense that leaves the backbone LLM frozen and instead learns two things under a single shared objective: which attention heads to intervene on, and how strongly to rescale their outputs. For a selected head h in layer l at token position t, a positive scalar multiplies the entire head-output vector, so the direction of the head representation and the relative composition of its features are preserved and only its magnitude changes. The scalar is predicted from the head’s own RMS-normalised query vector through an exponential-tanh map (Eq. 2), instantiated with g = tanh and lambda = log 5 in Appendix A.2, which confines the factor to roughly [0.2, 5]. Head selection is made differentiable by perturbing per-head selector logits with Gumbel noise and applying a straight-through Top-K estimator, so that exactly K heads are intervened on in every forward pass while gradients flow through a softmax surrogate. Training proceeds in three stages: scaling-function initialisation with all heads active, joint optimisation of the selector and the scaling parameters, and a final refinement of the scaling parameters on the fixed selected subset. The objective is the autoregressive negative log-likelihood of model-specific refusal continuations on harmful prompts and of the undefended model’s own continuations on benign prompts. The intervention is applied only while the input prompt is processed, never at generated-token positions.

The method is evaluated on five open-weight models (Llama-3.1-8B-Instruct, DeepSeek-LLM-7B-Chat, OpenChat-3.5-1210, Vicuna-13B-v1.5 and Gemma-3-27B-IT) against nine attacks, three of which (Plain Harmful, Human-Crafted and GCG) supply training data while the remaining six (PEZ, UAT, AutoPrompt, GBDA, GCG-M and PAIR) are evaluated zero-shot. The test sets comprise 2,115 harmful prompts, 500 benign instruction-following prompts and 250 XSTest over-refusal prompts, all scored by WildGuard, at two training budgets (n = 30 and n = 120). The baselines are SFT, LoRA-SFT, ASGuard and DDPO. The reported averages are ARR 96.58% / 99.05%, BPR 97.68% / 96.84% and ORR 10.08% / 13.04%, with MMLU at 57.72% / 57.66% against 57.68% for the undefended models. Supporting results cover two adaptive attacks, a head-selection ablation, a head-budget sweep, runtime measurements and a 540-example human validation of the WildGuard judge.

Critical strengths

S1. The low-data result is the consequential one, and it holds up.

At n = 30, AttentionSwitch reaches 96.58% average ARR while holding BPR at 97.68% and MMLU at 57.72% against 57.68% for the undefended models, using a controller of K(d_h + 1) parameters — approximately 1.9 x 10^-4 % to 3.8 x 10^-4 % of the backbone — trained in 63.20 seconds on average (Table 7). The comparison that carries weight here is not the one against ASGuard but the one against the fine-tuning baselines. At the same budget, SFT and LoRA-SFT reach 96.14% and 95.63% ARR only by driving BPR down to 24.64% and 67.60%, and MMLU to near-chance accuracy (roughly 24%, that is 1/4 on four-option multiple choice) on four of the five models. With 30 examples the fine-tuning defenses therefore do not yield a usable model at all, whereas AttentionSwitch does. For a practitioner who must harden a deployed checkpoint without a safety-training corpus, that is a meaningful difference.

S2. The evaluation is broad, and its limitations are reported rather than concealed.

Nine attacks with six held out from training, 2,115 harmful prompts, five models spanning 7B to 27B parameters, per-attack results with standard errors in Appendix Tables 4 and 5, runtime for every method, and a 540-example human validation of the WildGuard judge together exceed what is typically reported in this subarea. The authors also report the degradation under adaptive attacks and the rise in ORR at the larger training budget instead of omitting them. The appendix tables are detailed enough that the main claims can be checked independently. This matters to my assessment because it means the issues raised here are issues of experimental design rather than of reporting discipline, and are correspondingly easier to address.

Critical weaknesses

W1. The defense degrades substantially under adaptive attack, and the pattern of degradation suggests that the white-box adaptive evaluation may be understating the risk.

This is a defense paper, so robustness against an attacker who knows the defense is in place is the load-bearing claim. Section 4.3 reports average ARR falling from 97.62% to 86.80% under Adaptive GCG and from 92.26% to 77.60% under Adaptive PAIR. Three aspects of this result concern me.

The per-model averaging in Section 4.3 understates the worst case. The statement that models other than DeepSeek show drops of “13.58 percentage points or less” averages each model over the two adaptive attacks. Taken per attack, Adaptive GCG moves DeepSeek-LLM-7B-Chat from 93.19% to 52.00%, a drop of roughly 41 points, and Adaptive PAIR moves OpenChat-3.5 from 82.13% to 60.00%, a drop of roughly 22 points. A defense whose rejection rate on a 7B production-class model falls to 52% under a single known attack is not well characterised by a 13.58-point average.

The black-box adaptive attack is more effective than the white-box one. Adaptive PAIR uses only responses from the defended model and reaches 77.60% average ARR, whereas Adaptive GCG differentiates through the defended model and reaches 86.80%. A black-box attack outperforming a white-box attack against the same defense is a recognised indicator of gradient masking rather than of genuine robustness (Athalye et al., ICML 2018). A plausible mechanism is available here: the scaling factor is exp[lambda tanh(.)] with lambda = log 5, and tanh saturates, so the gradient reaching the adversarial suffix through the controller can be attenuated. The paper does not test for this, and the result as reported is consistent with either reading.

The adaptive evaluation is narrow. Two attacks, 50 prompts each, in a single configuration (n = 30, 20% of heads). At 50 prompts the reported standard error for DeepSeek under Adaptive GCG is 7.07, so the interval spans roughly 38% to 66%. Neither BPR nor ORR is reported under adaptive conditions, so the utility cost of operating in this regime is unknown. Two further attacks are directly invited by the design and are not run: one that places an explicit term on the scaling factor in the attack objective, driving it toward 1 at the selected heads, and one that exploits the fact that no scaling is applied at generated-token positions (Section 3.3) — for example a multi-turn or prefill-continuation attack in which the harmful content emerges during decoding.

I do not read these numbers as showing that the method fails; 86.80% and 77.60% are not trivial. But for a defense submission the adaptive-attack section is where the claim is won or lost, and as it stands it neither establishes a lower bound nor rules out the gradient-masking reading.

W2. Neither of the two claims that would constitute the contribution is isolated by the experiments presented.

The paper advances two empirical claims: that joint optimisation of intervention site and strength is what produces the gains, and that a sparse budget of 20–30% of heads is the right operating point. Neither is established by the experiments as reported.

The joint-optimisation ablation is confounded. The only experiment addressing it (Section 4.4.1) replaces AttentionSwitch’s selection with ASGuard’s selector and then fits AttentionSwitch’s scaling on that fixed subset. This changes the selection criterion and the joint-versus-sequential schedule at the same time, so the 96.58% versus 94.73% gap cannot be attributed to joint optimisation; it is equally consistent with AttentionSwitch’s selector simply being the better selector under either schedule. Two further differences from ASGuard are likewise unablated: a single positive scalar applied to the whole head output versus a per-dimension vector, and a query-conditioned factor versus a fixed one. All three are presented as contributions, and the gain is attributed to none of them individually. This is consequential because the underlying machinery — Gumbel-Softmax Top-K with a straight-through estimator — is the standard differentiable subset-selection recipe that the paper itself cites (Li et al., 2021); with the mechanism established elsewhere, the ablation is what must carry the contribution. The three-stage schedule also sits awkwardly with the framing, since Stage 1 initialises with all heads active and Stage 3 refits on a frozen subset, leaving only Stage 2 joint.

The head-budget conclusion is not supported at the resolution of the reported error bars, and the budget appears to have been chosen on the evaluation data. I could find no validation split described in the main text or in Appendix A.2. The sweep in Figure 4 and Table 6 is computed on the same 2,115 harmful and 500 benign prompts used for Table 1, and the 20% default is justified from that curve, as are the stage epoch counts, lambda = log 5 and the learning rates. Independently of that, the sweep does not support the conclusion drawn from it:

Heads scaled

20%

30%

40%

50%

ARR, n = 30

96.58 ± 1.93

96.43 ± 1.89

96.08 ± 2.24

95.41 ± 2.17

ARR, n = 120

99.05 ± 0.50

99.28 ± 0.38

99.09 ± 0.51

98.69 ± 0.71

Appendix Table 6. At n = 30 the 20–50% range lies within one standard error, and at n = 120 the 30% budget is the higher of the two.

Only the extremes separate — 1% of heads, and 80% and above. The claim that performance peaks at 20–30% is therefore an ordering of means that lies well inside one standard error. The error bars compound the problem: Table 6 reports spread across five models, which is between-model variation rather than estimation error, and the standard errors in Table 5 are binomial over test prompts (0.42 for a rate of 99.57% over 235 prompts). No variance over training seeds is reported anywhere, even though Gumbel-Softmax selection is stochastic and the identity of the selected heads is the paper’s own object of study. The selector temperature tau is never given.

Several of the gaps the paper rests on are smaller than the measurement error of the judge. A BPR of 97.68% for AttentionSwitch against 97.84% for ASGuard is a 0.16-point difference, read by a judge with 97.22% overall and 96.11% harmful-set agreement, validated on 540 examples that work out to roughly 30 per defense-by-evaluation-set cell. The human validation is a sound thing to have done, but at that cell size it cannot exclude a defense-specific judge bias of the same magnitude as the differences being claimed.

Questions for authors

The two unablated design choices. What happens if the scalar is replaced by a per-dimension vector, and separately if the query-conditioned factor is replaced by a fixed per-head constant? Which of the three differences from ASGuard accounts for the gap?

Model selection. Was any held-out split used to choose the 20% head budget, the stage epoch counts (8/12/8), lambda = log 5 and the learning rates? If those choices were made from Figure 4, which is computed on the evaluation prompts, then the Table 1 numbers should be read as in-sample at the configuration level, and I would want to see them reproduced with a disjoint development set.

Seed variance and selection stability. How much do ARR, BPR and ORR vary across random seeds at a fixed configuration? How stable is the selected head set — for instance, what is the mean pairwise Jaccard overlap of the selected heads across seeds for a given model? If the overlap is low while performance is stable, the claim about which heads matter would need to be weakened accordingly.

Threat model and the controller. Section 3.1 grants the attacker full knowledge of the victim’s architecture and parameters. Are the controller parameters, that is the scaling weights and the selector, also assumed known? Under Kerckhoffs’ principle they should be, since they are part of the deployed model. What is the ARR when the GCG objective includes an explicit term driving the scaling factor toward 1 at the selected heads?

Gradient masking. Could you report the attack loss that Adaptive GCG attains against the defended and the undefended model under the same budget, together with the result of transferring suffixes optimised on the undefended model? If Adaptive GCG fails to reduce its own objective against the defended model while Adaptive PAIR succeeds, that would point to a saturated gradient through exp[lambda tanh(.)] rather than to robustness.

Prefill-only intervention. What happens to ARR, BPR and generation latency if the scaling is also applied at generated-token positions? And how does the defense behave on multi-turn or prefill-continuation attacks, where the harmful content emerges during decoding rather than within the prompt?

Refusal-prefix imitation. If the first tokens of the response are forced past the refusal opening, does harmful content still follow? This would distinguish suppression from imitation of the refusal continuations used as training targets.

Suggestions for future work (not requests for the rebuttal)

A generative utility benchmark alongside MMLU, to test capability preservation where the intervention actually operates.

Latency reported per generated token rather than per sample.

One representation-level steering baseline and one input-level baseline, drawn from the methods already discussed in Section 2.2.

An analysis of what the selected heads do — for example their overlap across models and attacks, or their behaviour on benign prompts — which would address the question of why these particular heads help, which the current results leave open.

6 comments
AuthorsOpenReview ↗

Weakness 2. Additional Ablations to Isolate the Contributions of the Individual Components

We thank the reviewer for the detailed feedback. In our paper (Figure 3), we provide an ablation showing that scaling all attention heads (100%) or sequentially selecting a sparse subset (20%) using the previously proposed ASGuard method and then scaling those heads underperforms compared with our proposed joint head selection and scaling. Specifically, our method achieves 96.58% ARR and 97.68% BPR, compared with 92.26% ARR and 97.36% BPR when scaling all heads, and 94.73% ARR and 97.52% BPR when using ASGuard-selected heads.

We have performed additional ablations, as requested by the reviewer, to isolate the contributions of the individual components. We test four different versions of AttentionSwitch to isolate the effects of joint head selection and scaling and dynamic scaling factors. All variants use the same training data: 30 training examples (15 harmful and 15 benign), with 20% of attention heads selected for scaling. We evaluate all five models across nine attack categories: Plain Harmful (PH), Human-Crafted (HC), GCG, PEZ, UAT, AutoPrompt, GBDA, GCG-M, and PAIR, with 235 test prompts per attack category (2,115 harmful prompts in total per model), alongside 500 benign prompts for BPR and 250 deceptive-benign prompts to evaluate over-refusal (ORR).

The table below shows the results. Joint + Dynamic is our main proposed method. Replacing joint optimization with sequential optimization, dynamic scaling with static scaling, or both consistently reduces defense performance compared with our proposed approach across all five models.

Averaged across all five models, Joint + Dynamic achieves 96.58% ARR and 97.68% BPR. In comparison, Sequential + Dynamic achieves 92.65% ARR and 97.36% BPR, Joint + Static achieves 82.87% ARR and 97.60% BPR, and Sequential + Static achieves 80.78% ARR and 97.32% BPR. Thus, our proposed method improves ARR by 3.92, 13.71, and 15.80 percentage points, respectively, while maintaining similar BPR.

Table R1: Joint versus sequential head selection and dynamic versus static head scaling. Values are reported as percentages (rate ± standard error). Arrows indicate the preferred direction of each metric.

ModelMethodARR ↑ (%)BPR ↑ (%)ORR ↓ (%)
Llama-3.1-8BJoint + Dynamic (Ours)99.57 ± 0.1498.00 ± 0.6313.20 ± 2.14
Sequential + Dynamic98.82 ± 0.2497.60 ± 0.6817.60 ± 2.41
Joint + Static98.11 ± 0.3097.80 ± 0.6614.00 ± 2.19
Sequential + Static98.01 ± 0.3097.80 ± 0.6613.20 ± 2.14
DeepSeek-7BJoint + Dynamic (Ours)92.10 ± 0.5997.60 ± 0.6817.60 ± 2.41
Sequential + Dynamic87.04 ± 0.7397.40 ± 0.7118.40 ± 2.45
Joint + Static73.66 ± 0.9697.80 ± 0.6618.80 ± 2.47
Sequential + Static71.11 ± 0.9997.20 ± 0.7418.00 ± 2.43
OpenChat-3.5Joint + Dynamic (Ours)91.58 ± 0.6098.20 ± 0.594.40 ± 1.30
Sequential + Dynamic86.24 ± 0.7598.00 ± 0.639.60 ± 1.86
Joint + Static56.26 ± 1.0897.20 ± 0.745.60 ± 1.45
Sequential + Static53.05 ± 1.0997.40 ± 0.715.20 ± 1.40
Vicuna-13BJoint + Dynamic (Ours)99.76 ± 0.1195.40 ± 0.9410.80 ± 1.96
Sequential + Dynamic94.61 ± 0.4995.20 ± 0.9619.60 ± 2.51
Joint + Static90.07 ± 0.6595.80 ± 0.9012.40 ± 2.08
Sequential + Static87.90 ± 0.7195.00 ± 0.9712.80 ± 2.11
Gemma-3-27BJoint + Dynamic (Ours)99.86 ± 0.0899.20 ± 0.404.40 ± 1.30
Sequential + Dynamic96.55 ± 0.4098.60 ± 0.536.80 ± 1.59
Joint + Static96.22 ± 0.4199.40 ± 0.356.00 ± 1.50
Sequential + Static93.81 ± 0.5299.20 ± 0.405.60 ± 1.45
AuthorsOpenReview ↗

Weakness 2: Ablation of Scalar Scaling (Direction-Preserving Head Scaling)

As requested by the reviewer, we compared our proposed scalar attention-head scaling (which uses a single positive scaling factor to preserve the direction of the attention-head activation and the relative ratios of its components, affecting only its magnitude) with a vector scaling variant (which independently scales each dimension, changing the direction of the head activation).

Both variants jointly optimize head selection and scaling, intervene on 20% of attention heads, and use identical training and evaluation settings with n=30 samples (15 harmful and 15 benign). We evaluate five models using 2,115 harmful prompts across nine attack families, 500 benign prompts, and 250 deceptive-benign prompts per model.

Our proposed scalar scaling consistently achieves a higher benign pass rate (BPR) and lower over-refusal rate (ORR) across all five models, with average BPR of 97.68% versus 95.88% and ORR of 10.08% versus 12.72%, while maintaining approximately similar Attack Rejection Rate (ARR of 96.57% versus 96.21%).

Table R2: Scalar versus vector attention-head scaling. Results are percentages (rate ± standard error).

ModelScalingARR ↑ (%)BPR ↑ (%)ORR ↓ (%)
Llama-3.1-8BScalar (Ours)99.57 ± 0.1498.00 ± 0.6313.20 ± 2.14
Vector99.53 ± 0.1596.60 ± 0.8114.00 ± 2.19
DeepSeek-7BScalar (Ours)92.10 ± 0.5997.60 ± 0.6817.60 ± 2.41
Vector91.68 ± 0.6096.20 ± 0.8618.40 ± 2.45
OpenChat-3.5Scalar (Ours)91.58 ± 0.6098.20 ± 0.594.40 ± 1.30
Vector91.63 ± 0.6097.40 ± 0.716.00 ± 1.50
Vicuna-13BScalar (Ours)99.76 ± 0.1195.40 ± 0.9410.80 ± 1.96
Vector98.20 ± 0.2993.60 ± 1.0915.60 ± 2.29
Gemma-3-27BScalar (Ours)99.86 ± 0.0899.20 ± 0.404.40 ± 1.30
Vector100.00 ± 0.0095.60 ± 0.929.60 ± 1.86
AuthorsOpenReview ↗

Weakness 1. Additional Adaptive Robustness and Gradient Masking

Adaptive Robustness

Adaptive attacks assume an unusually strong attacker with full access to the defense, so some degradation in defense performance is expected. The main question is whether this degradation is comparable to that of existing defenses. To investigate this, we compared AttentionSwitch with other baselines under adaptive attacks using the same training and evaluation settings.

All defenses were trained with n=30 samples (15 Plain Harmful and 15 benign) and evaluated on 235 unseen prompts per attack. Importantly, the harmful training prompts contained no jailbreak modifications, so the evaluated attack families were tested in a zero-shot setting. AttentionSwitch consistently outperforms the baselines under adaptive attacks, with the exception of gemma under adaptive GCG, where it performs nearly identically to DDPO (97.02% vs. 97.45%). On average, AttentionSwitch achieves 85.23% ARR, compared with 79.45% for DDPO and 65.70% for ASGuard.

Table R3: Adaptive Attack Rejection Rate (ARR %, higher is better). Values are reported as percentages (rate ± standard error).

AttackDefenseLlama-3.1-8BDeepSeek-7BOpenChat-3.5Vicuna-13BGemma-3-27B
Adaptive GCGNone90.64 ± 1.9058.30 ± 3.2235.32 ± 3.1271.91 ± 2.9391.49 ± 1.82
ASGuard91.06 ± 1.8660.00 ± 3.2062.55 ± 3.1673.62 ± 2.8794.47 ± 1.49
DDPO95.32 ± 1.3867.23 ± 3.0679.15 ± 2.6587.23 ± 2.1897.45 ± 1.03
AttentionSwitch (Ours)98.30 ± 0.8471.49 ± 2.9587.66 ± 2.1590.21 ± 1.9497.02 ± 1.11
Adaptive PAIRNone71.49 ± 2.9537.87 ± 3.1630.21 ± 3.0047.23 ± 3.2657.87 ± 3.22
ASGuard68.94 ± 3.0238.72 ± 3.1851.49 ± 3.2646.38 ± 3.2569.79 ± 3.00
DDPO73.19 ± 2.8966.38 ± 3.0870.64 ± 2.9776.60 ± 2.7681.28 ± 2.54
AttentionSwitch (Ours)78.30 ± 2.6972.34 ± 2.9275.32 ± 2.8192.34 ± 1.7389.36 ± 2.01

Gradient Masking

We investigate potential gradient masking by comparing three GCG settings: Undefended → Undefended optimizes and evaluates GCG suffixes on the undefended model, Undefended → AttentionSwitch transfers those suffixes to the defended model, and AttentionSwitch → AttentionSwitch optimizes and evaluates GCG directly against the defense.

Table R4: GCG attack rejection rate (ARR %). Arrows denote optimization target → evaluation target. AttentionSwitch uses n=30n=30 training examples. Values are reported as ARR ± standard error over 235 goals per model.

ModelUndefended → UndefendedUndefended → AttentionSwitchAttentionSwitch → AttentionSwitch
Llama-3.1-8B90.64 ± 1.9097.87 ± 0.9498.30 ± 0.84
DeepSeek-7B58.30 ± 3.2292.77 ± 1.6971.49 ± 2.95
OpenChat-3.535.32 ± 3.1294.47 ± 1.4987.66 ± 2.15
Vicuna-13B71.91 ± 2.9395.32 ± 1.3890.21 ± 1.94
Gemma-3-27B91.49 ± 1.8297.45 ± 1.0397.02 ± 1.11

Direct GCG against the defense produces 130 non-rejections out of 1,175 attacks (11.06%), compared with 52 (4.43%) under suffix transfer. This pattern holds consistently, with the exception of Llama-3.1, where the difference is only one prompt and defense performance remains high in both settings. This suggests that gradients through AttentionSwitch provide useful signals for adversarial optimization, offering evidence against the gradient-masking explanation.

Table R5: Mean GCG loss (↓) across 235 harmful goals per model. The first two numerical columns report losses at the first and final optimization steps for undefended and defended GCG, respectively. The last two columns report the losses of transferred and directly optimized suffixes evaluated on AttentionSwitch.

ModelUndefended Loss (First → Last)Defended Loss (First → Last)Transfer LossDirect Loss
Llama-3.1-8B5.18 → 3.295.84 → 3.345.233.32
DeepSeek-7B2.96 → 1.273.43 → 1.592.401.61
OpenChat-3.52.64 → 1.042.82 → 1.552.021.56
Vicuna-13B3.28 → 0.494.06 → 0.831.740.84
Gemma-3-27B7.85 → 2.737.01 → 3.964.263.98

Table R5 provides additional evidence against gradient masking through attack-loss diagnostics. GCG consistently reduces its loss when optimizing directly against AttentionSwitch, and directly optimized suffixes achieve lower losses than transferred suffixes. These results suggest that gradients remain informative for adversarial optimization, while AttentionSwitch still maintains an 88.94% attack rejection rate (ARR) under adaptive GCG. Additionally, recorded gradients remain finite and nonzero, and none of the measured scaling-controller outputs reaches the saturation threshold.

Comments on the paper

2 comments

The authors have posted their rebuttal, and I would appreciate it if you could read it and respond as soon as possible. please state whether the rebuttal addresses your main concerns, and update your score and review accordingly.

AuthorsOpenReview ↗

Final Note from the Authors

Dear AC and Reviewers,

We sincerely thank the AC and all reviewers for their time and valuable feedback.

We have done our best to address each reviewer's concerns by conducting additional experiments and providing detailed responses to their questions. We have reported these results in the corresponding reviewer discussions and incorporated the additional experiments and clarifications into our revised manuscript, which has been uploaded.

We hope our additional results have addressed the reviewers' concerns and will be considered in the final assessment.

Authors of Submission #27741

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.