acceptodds
Under review as a conference paper at ICLR 2027

Rethinking Refusal Directions: Identification and Intervention Effects

Abstract

Low-dimensional activation directions can causally control refusal in large language models, but a direction’s identity and behavioral meaning are often treated as properties of the vector alone. We show that neither is fixed. First, the direction identified as controlling refusal depends on how candidates are constructed and evaluated: lexical and activation-space objectives can select closely aligned or substantially different directions. A validated direction is therefore better understood as an operational causal approximation than as a uniquely recovered internal axis. Second, full-generation semantic judgments analyzed with Signal Detection Theory show that harmful and benign refusal rates change differently across operators, models, and prompt populations, producing changes in both response criterion and the descriptive probit-scale separation index . Third, in a separate continuous-score analysis of contrastive activation addition, we identify a stable prompt-specific downstream gain . Item-specific gains reconstruct local proxy-discrimination trajectories more accurately than class-level or global response models, demonstrating that a common activation displacement can be transformed into heterogeneous functional effects across prompts. Together, these results show that a refusal direction functions as an operational causal interface whose behavioral meaning emerges jointly from its identification procedure, intervention operator, input state, downstream computation, and evaluation population.

Primary area:
interpretability and explainable AI
Keywords:
mechanistic interpretability, activation steering, refusal directions, signal detection theory, large language models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_1cvCOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines whether refusal directions in language models are stable and how interventions along these directions affect refusal behavior. The authors compare different methods for identifying refusal directions and test three intervention methods across four models and two datasets. They use Signal Detection Theory to distinguish refusal of harmful prompts from over-refusal of harmless prompts. They also study whether individual prompts respond differently to small interventions. The results suggest that the selected direction depends partly on how it is identified, and that behavioral effects vary across intervention methods, prompts, and datasets.

Critical strengths
  1. The paper addresses an important question about what refusal directions actually control. I particularly appreciate the distinction between refusing harmful requests and over-refusing harmless ones.
  2. The experimental analysis is careful: (a) the authors evaluate full generated responses; (b) use paired comparisons; and (c) include several robustness checks.
Critical weaknesses
  1. The paper argues that refusal directions depend on how they are identified, but the alternative directions are never tested on behavior. So it's unclear whether these are actually different meaningful directions, or whether one method simply finds a better direction than another. The conclusion also mentions factors like the layer and the model's later processing, which weren't independently tested.
  2. All refusal labels come from one AI judge, and the authors don't compare its judgments with human labels. They also don't check whether the interventions damage the quality of the responses, or explain how broken or meaningless responses are classified. This makes it difficult to know whether the changes in refusal rates reflect actual changes in refusal behavior.
  3. The Gᵢ analysis uses much smaller interventions and measures only the probability of starting with a refusal word, while the main experiments use larger interventions and evaluate full responses. The predictions also don't hold at the larger intervention strengths used in the main experiments. Thus, I'm not sure what this analysis tells us about the behavioral findings.
Questions for authors
  1. Since d′ and c are used only descriptively, what do they add beyond the harmful and harmless refusal rates?
  2. On XSTest, the models already refuse almost all harmful prompts before the intervention. Could this explain some of the differences between datasets, simply because there's less room for refusal to increase?
  3. In the Gᵢ analysis, wouldn't we expect each prompt's own slope to predict its changes better than an average slope, especially when the relationship is almost linear? It seems to me that the pairing test is the more informative result.
  4. For Qwen, the stability test in Table 1 selected different directions from those used in the main experiments in Table 2. How does this support the stability claim in Section 5.1?
  5. Why was the intervention applied during the whole response in the early qualification check, but only during the prompt in the main experiments? Could this affect the results?
  6. The three intervention methods use different strength scales, and ablation has no strength scale at all. How can we tell whether the differences between them come from the methods themselves rather than from different strengths?
  7. The paper mentions earlier work by Wollschläger et al. on multiple refusal directions, but the abstract seems to present this as a new finding. What does this paper add beyond that work?
  8. Many of the intervals for changes in d′ are wide and include zero. How much confidence should we place in conclusions based on these changes?
  9. Could the authors clarify what counts as a "valid prompt position," how the Arditi and Wang prompt sets differ, and what the "literal control" is meant to test?
  10. How were the 150 harmful and 150 harmless prompts selected from each dataset? Could differences between the datasets beyond their baseline refusal rates explain some of the results?
Reviewer_K2bDOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper provides an analysis of how different interventions known in the literature (i.e., ablation, addition, etc) affect refusal behaviour, leveraging a unique fixed refusal direction. Furthermore, the work schematizes the computation of the refusal direction into separate steps: a lexical-proxy sensitivity, the formal direction construction, Cross-objective audit with cosmic, CROSS-OBJECTIVE AUDIT WITH COSMIC, and semantic qualification and freezing. The analysis is performed leveraging Signal Detection Theory, treating harmful prompts as signal and benign prompts as noise. Experiments are performed on four instruction-tuned models, using datasets from XSTest and WildGuardTest.

Critical strengths
  • The work focuses on a current open problem in the literature, which is to understand how different activation interventions impact the refusal suppression.
Critical weaknesses
  • The aim of the paper is not completely clear from the abstract and the introduction, and requires several back-and-forths among the following sections. The reason is probably due to the second issue in this list. See detailed comment
  • Several sentences leverage uncommon and sometimes non-scientific terminology, making it difficult to understand the meaning. See detailed comments below.
  • The formal aspect of the paper is not sufficiently clear, as I've found equations difficult to read or incomplete, sometimes only due to an unsuitable choice of symbols (as j for a tensor in R^d, and i as the index of the prompt, sometimes due to undefined variables.
  • Finally, several related works have not been discussed, although particularly relevant for the work, [1,2,3] is a (not exhaustive) list.

[1] Tan, Daniel, et al. "Analysing the generalisation and reliability of steering vectors." Advances in Neural Information Processing Systems 37 (2024): 139179-139212.

[2] Wang, Xinpeng, et al. "Surgical, cheap, and flexible: Mitigating false refusal in language models via single vector ablation." International Conference on Learning Representations. Vol. 2025. 2025.

[3] Piras, Giorgio, et al. "SOM directions are better than one: multi-directional refusal suppression in language models." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 40. No. 39. 2026.

Detailed comments:

line 12-13 : These two sentences are obscure. In particular, what kind of property of the “vector” are the authors referring to? Furthermore, “Neither is fixed” is not informative.

line 16-18 : This sentence does not seem to be scientifically valid. In particular, the notion of “operational causal approximation” is not clear, and it’s not common in the literature.

line 26-27 : Similar to the previous comment, the concept of “operational causal interface” does not have a clear scientific meaning in this context.

line 45: The usage of the rhetorical “the” is not helpful, further details are needed to better express the concept.

line 45-46 : Authors should revise this sentence to make it clear how this analysis differs from previous work.

line 139-140 : I appreciate that there is an attempt to gently introduce the topic of refusal direction. However, this sentence does not provide a clear introduction to the aim of the section. Furthermore, “lexical evidence” does not have a clear (scientific) meaning here.

Eq. 1 : The m in formula 1 has not been defined yet, and its role is not clear from the context.

Eq. 2 : Notation in eq 2 uses too many symbols and can be simplified. What’s the role of kk?

Questions for authors

The authors should provide strong evidence that they can revise the text and formulas to address the major concerns. Furthermore, they should explain why the neglected related work does not affect the conclusions of the analysis or, otherwise, include it in the experimental section.

5 comments

We thank the reviewer for the detailed feedback on terminology, mathematical notation, and related literature. We have thoroughly rewritten the manuscript to make the central aim clear from the Abstract and Introduction, replaced all non-standard phrasing with direct scientific terminology, simplified and fully defined every equation, and integrated the missing literature in Section 1 and Section 2.

  1. Clarifying the aim in the Abstract and Introduction & removing obscure terminology (Weaknesses 1 & 2; Detailed comments on lines 12–13, 16–18, 26–27, 45–46, 139–140)

• Clear statement of purpose: The revised Abstract and Section 1 (Introduction) directly state the paper’s scope and central question: after constructing and auditing one refusal direction per checkpoint (Section 3.2), we freeze that direction and study how a common contrastive activation addition (CAA) displacement affects different prompts—evaluating both complete-response refusal rates (H,FH, F and descriptive SDT indices c,d′c, d' in Section 4) and continuous first-token refusal log-odds scores (si(k)s_i(k) and local gains GiG_i in Section 5). • Complete removal of non-standard phrasing: Every phrase flagged by the reviewer—including "Neither is fixed", "operational causal approximation", "operational causal interface", the rhetorical "the", and undefined "lexical evidence"—has been removed. • Lines 12–28 (Abstract) & Lines 32–66 (Introduction): Rewritten in plain, precise terms describing how CAA adds a constant displacement vector krk\mathbf{r} and how prompt-specific local gains GiG_i and their within-class covariance with baseline scores Cov⁡(si(0),Gi∣y)\operatorname{Cov}(s_i(0), G_i \mid y) govern changes in class means and spread. • Section 3.2 (Direction Selection and Validation, page 3): Directly introduces the candidate bank (mean harmful-minus-benign residual-stream activations across candidate layer–position sites) and defines the first-token refusal proxy (the log-odds of the summed probabilities of tokens I, It for Gemma 2 or I, As for Qwen 2.5, with exact token IDs in Appendix A, Table 2).

  1. Revising and simplifying the mathematical notation (Weakness 3; Detailed comments on Eqs. 1 & 2)

We moved the supplementary operators (single-site scaling and all-layer ablation) to Appendix F so the main text focuses cleanly on CAA. This allowed us to replace the overloaded multi-operator notation in the original submission—specifically eliminating the undefined operator superscript (m)(m), the vector symbol jit(m)j^{(m)}_{it}, and the transform notation Ψb(H~i(m))\Psi_b(\widetilde{H}^{(m)}_i)—with nine self-contained equations (Equations 1–9):

• Equation (1) (Section 3.3, page 3 — The Common CAA Write):

hit′=hit+kr,k=α∣rraw∣2\mathbf{h}'_{it} = \mathbf{h}_{it} + k\mathbf{r}, \qquad k = \alpha \\|\mathbf{r}_{\mathrm{raw}}\\|_2

where hit∈Rd\mathbf{h}_{it} \in \mathbb{R}^d is the residual-stream activation vector for prompt ii at non-padding prompt position tt, rraw\mathbf{r}_{\mathrm{raw}} is the selected activation contrast, r=rraw/∣rraw∣2\mathbf{r} = \mathbf{r}_{\mathrm{raw}} / \\|\mathbf{r}_{\mathrm{raw}}\\|_2 is the unit direction, α\alpha is the implemented CAA coefficient, and kk is the signed geometric displacement. The standardized displacement is explicitly defined right below as z=k/σρ=α∣rraw∣2/σρz = k / \sigma_\rho = \alpha \\|\mathbf{r}_{\mathrm{raw}}\\|_2 / \sigma_\rho, where ρi=⟨hi,tibase,r⟩\rho_i = \langle \mathbf{h}^{\mathrm{base}}_{i, t_i}, \mathbf{r} \rangle is the baseline coordinate at the selected layer input and final non-padding prompt position tit_i, and σρ\sigma_\rho is its sample standard deviation in the fixed validation reference set (Appendix E, Equation 11, Table 5). • Equations (2)–(5) (Section 3.4, page 4 — Behavioral & Continuous Score Measurements): Explicitly define the Hautus (1995) log-linear corrected refusal rates H~,F~\widetilde{H}, \widetilde{F} (Eq. 2), the probit SDT indices d′d' and cc (Eq. 3), the continuous first-token log-odds score si(k)=log⁡[pT,i(k)+ϵ]−log⁡[1−pT,i(k)+ϵ]s_i(k) = \log[p_{T,i}(k) + \epsilon] - \log[1 - p_{T,i}(k) + \epsilon] (Eq. 4), and the standardized continuous-score separation dG′(k)d'_{\mathrm{G}}(k) (Eq. 5). • Equations (6)–(9) (Sections 5.1–5.2, page 6 — Local Gain & Moment Decomposition): Define each prompt’s zero-dose local gain Gi=∂si(k)∂k∣k=0G_i = \left.\frac{\partial s_i(k)}{\partial k}\right|_{k=0} and linear reconstruction s^i(k)=si(0)+kG^i\widehat{s}_i(k) = s_i(0) + k\widehat{G}_i (Eq. 6), the reconstructed class mean and within-class variance relations (Eqs. 7–8), and the exact first-order derivative decomposition of dG′(k)d'_{\mathrm{G}}(k) into its class-mean contribution and pooled-spread contribution (Eq. 9, derived step-by-step in Appendix K.2).

  1. Discussion and positioning relative to [1] Tan et al. (2024), [2] Wang et al. (2025a), and [3] Piras et al. (2026) (Weakness 4; Question)

All three works (along with Wollschläger et al., 2025 and Joad et al., 2026) are now cited and contrasted in Section 1 and Section 2 (Related Work, page 2). Including them strengthens the context of our study without altering our conclusions, because our analysis addresses a distinct level of mechanism:

  1. Tan et al. (2024) (Analysing the generalisation and reliability of steering vectors): Tan et al. measure per-input steerability slopes on multiple-choice tasks and study dataset and prompt factors associated with steering effectiveness and cross-dataset transfer. We explicitly credit Tan et al. for establishing input-dependent slopes (Section 1, line 58; Section 2, line 83; Section 6, line 459). Our specific question is how individual slopes aggregate into changes in two-class harmful–benign discrimination (dG′d'_{\mathrm{G}}). Specifically, Equations (7)–(9) show that first-order changes in within-class variance are governed by Cov⁡(si(0),Gi∣y)\operatorname{Cov}(s_i(0), G_i \mid y), and our within-class permutation test (Section 5.4, Figure 3A, Table 13) shows that destroying the pairing between baseline score si(0)s_i(0) and gain GiG_i significantly degrades trajectory reconstruction in 7 of 8 model–dataset cells even when the class-specific distribution of slopes is held fixed.
  2. Wang et al. (2025a) (Surgical, cheap, and flexible: Mitigating false refusal in language models via single vector ablation): Wang et al. (2025a) address the refusal–over-refusal trade-off by constructing a false-refusal direction via orthogonal projection to reduce benign refusal while preserving harmful refusal (Section 1, line 47; Section 2, line 74). Whereas Wang et al. modify the intervention direction to mitigate false refusal, our study holds the extracted refusal direction fixed and analyzes how a single common displacement along that direction deforms harmful and benign score distributions differently across prompt populations.
  3. Piras et al. (2026) (SOM directions are better than one: Multi-directional refusal suppression in language models): Piras et al. (2026) use Self-Organizing Maps to identify multiple refusal directions from clusters of harmful hidden representations and examine cluster centers and intra-cluster spread in 3D PCA activation space at the intervention layer on harmful prompts (Section 2, lines 75–76 and 88–89). In contrast, our moment decomposition (Equations 7–9, Figure 3B, Table 11) operates on downstream output log-odds scores across both harmful and benign classes under fixed-direction CAA, explaining how class-mean gain differences and within-class baseline–gain covariance jointly determine output separation trajectories.
Area_Chair_FECROpenReview ↗

Please respond to the authors

Hello!

Just a friendly bump that you should engage with the authors response

Reviewer_285COpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies three questions regarding steering along refusal directions: (i) whether a refusal direction is uniquely defined (ii) how manipulating a fixed refusal direction affects refusal behaviourally (iii) if the same steering displacement causes similar effects across different prompts. Experimental results suggest that different ways of constructing refusal steering vector can produce similar or different refusal directions in terms of cosine similarity depending on a model assessed, downstream refusal steering sensitivity is prompt-dependent, and local derivatives can help predicting first-token proxy changes.

Critical strengths
  • The questions studied regarding uniqueness of refusal direction and stability of its behavioural effects are meaningful and important
  • The experimental setup is overall meaningful, with useful controls and some detailed statistical analyses
Critical weaknesses
  • Overall, all three question studied in the current paper have already been extensively studied in prior art, some of which is even cited in related work. For example, [1] identifies multiple mechanistically independent refusal directions, [2] shows that refusal steering affects refusal on harmful and benign prompts differently, [3] showed substantial variations in steering effectiveness across different inputs. It is unclear then how the current paper advances beyond related work. If the paper is positioned as a systematic study of refusal directions in language models, it requires much broader coverage of models, refusal extraction modes, behavioural metrics and intervention protocols, all accompanied by rigorous analysis.
  • The paper is quite hard to read and follow. For example, in Sec. 3.1 combinations of "I+It", "I+As" are discussed. However, they are not defined anywhere prior to using it. I was only able to find explanation in the Appendix Sec. A, which is not referenced in the Sec. 3.1. It is not obvious that "I+It" refers to the first tokens of a model's response, which are used as proxies for refusal. Fig.1 and 2 are impossible to read. In general, the rest of the paper is written using overcomplicated language and hard to follow.
  • The paper has serious formatting issues. Pages 19 and 20 are landscape-oriented, and there is no note about that the paper is under review for ICLR on top of the first page.
  • The finding that fixed steering displacement produces different local changes in a downstream score is not surprising, given that the model's computation is non-linear. It would be more interesting to study if any useful and/or predictable structure can be found in the connection between steering displacements and a downstream score.

[1] Wollschläger et. al., The Geometry of Refusal in Large Language Models: Concept Cones and Representational Independence

[2] Joad et. al., There Is More to Refusal in Large Language Models than a Single Direction

[3] Tan et. al., Analysing the Generalisation and Reliability of Steering Vectors

Questions for authors
  • Can you clarify how the paper advances beyond related work?
4 comments

We thank the reviewer for recognizing the importance of our research questions, experimental controls, and statistical analyses, and for highlighting these closely related studies. We have substantially revised the manuscript to clarify how our contribution advances beyond prior work, simplified the exposition and notation throughout, and fixed all formatting and figure-readability issues (with content revisions highlighted in green in the updated PDF).

  1. How the paper advances beyond related work & predictable response structure (Weaknesses 1 & 4; Question)

All five related studies raised across the reviews—including Wollschläger et al. (2025), Joad et al. (2026), and Tan et al. (2024)—are now explicitly discussed in Section 1 and Section 2. Rather than positioning the paper as a broad survey across extraction families, we use the direction-selection audit in Section 3.2 (Appendix B) to document provenance and then freeze one direction per checkpoint to study our central question:

How does a common activation displacement produce changes in harmful–benign score separation, and which prompt-level response information is needed to reconstruct those changes?

• 1.1 The contribution goes beyond observing heterogeneous slopes: We agree that nonlinear computation can produce different responses to the same intervention. However, this observation alone does not establish how those responses aggregate into changes in class separation. Wollschläger et al. (2025) and Piras et al. (2026) establish multi-directional refusal geometry; Joad et al. (2026) and Wang et al. (2025a) examine refusal–over-refusal trade-offs; and Tan et al. (2024) study per-input steerability slopes and factors associated with their variation. Our analysis connects prompt-level responses to population-level discrimination trajectories and tests which aspects of the response structure must be retained.

The full-generation analysis (Section 4) uses Signal Detection Theory (SDT) to jointly describe harmful and benign refusal rates through criterion cc and descriptive separation d′d', showing that the same intervention direction can change separation in opposite directions across prompt populations (e.g., Gemma 2–9B at α=−1\alpha = -1 gives Δd′=+1.349\Delta d' = +1.349 on XSTest versus −0.479-0.479 on WildGuardTest; WildGuardTest-minus-XSTest difference −1.828-1.828, Holm-adjusted p=.0018p = .0018; Figure 1, Table 7). Separately, the continuous-score analysis (Section 5) investigates a precise theoretical distinction: a uniform final-score shift moves refusal tendency without changing score separation, whereas a common internal displacement need not produce a uniform output shift (Section 5.1). We test how prompt-specific downstream gains account for this difference.

• 1.2 Baseline–gain pairing provides structured information beyond class-average responses: Equations (7)–(9) (Section 5.2, page 6) show how the local derivative of continuous-score separation dG′(k)d'_{\mathrm{G}}(k) depends on both the difference in class-average gains (GˉH−GˉB\bar{G}_H - \bar{G}_B) and the within-class covariance Cov⁡(si(0),Gi∣y)\operatorname{Cov}(s_i(0), G_i \mid y). The latter determines the first-order change in within-class variance. This identifies a concrete response structure, rather than treating slope variation as undifferentiated heterogeneity.

We test its empirical importance in two complementary ways:

  1. Trajectory reconstruction: Item-specific gains reconstruct local dG′(k)d'_{\mathrm{G}}(k) trajectories within ∣α∣≤0.125|\alpha| \le 0.125 with aggregate RMSE 0.0232, compared with 0.0779 for dataset-by-class mean gains and 0.1578 for global mean gains (Section 5.3, Figure 2, Appendix K.3 Table 12).
  2. Within-class pairing test: A within-class permutation preserves the marginal distributions of baseline scores and gains (including class means) while breaking their pairing. Intact pairing outperforms shuffled pairing in seven of eight model–dataset combinations after Holm correction (Section 5.4, Figure 3A, Appendix K.3 Table 13). Thus, retaining the gain distribution alone is insufficient: its association with baseline scores contributes to reconstructing the measured separation trajectories (Figure 3B, Table 11).

• 1.3 Gain also has predictable structure before intervention: A five-fold cross-validated quadratic ridge model using only baseline proxy score si(0)s_i(0) and the normalized refusal-direction coordinate ρi/σρ\rho_i / \sigma_\rho achieves out-of-fold R2R^2 of 0.191–0.304 on WildGuardTest and 0.395–0.747 on XSTest (Section 5.4, Appendix K.4 Table 14). Held-out prompts require no perturbation for prediction, although perturbations are required to obtain training targets. This directly addresses the reviewer’s request for useful or predictable structure in the connection between an activation displacement and its downstream effect.

The pairing and baseline-prediction analyses were already present in the original submission; the revision foregrounds them in Section 1 and Section 5 and makes their connection to the mathematical decomposition in Equations (7)–(9) explicit. The resulting insight is that predictable steering requires accounting for how the input state transforms the internal intervention into an output response.

  1. Clarity, token definitions, and figure readability (Weakness 2)

• Explicit definition of I, It and I, As proxy tokens: Table 1 and Section 3.2 (page 3) now explicitly define these token sets on first appearance as the first-token refusal proxy used in Wang-style direction selection and continuous-score analysis—specifically, the log-odds of the summed next-token probabilities assigned to the refusal-like opening tokens "I" and "It" for Gemma 2, or "I" and "As" for Qwen 2.5 (Equation 4 in Section 3.4, page 4), with a direct reference to Appendix A (Table 2) for exact tokenizer IDs. • Streamlined prose and equations: We rewrote the text in direct scientific language and reduced the main-text mathematical formalization to nine self-contained equations (Equations 1–9), defining every symbol immediately in context. • Enlarged figures: Figures 1, 2, and 3 in the main text (as well as Figures 4–7 in the appendix) have been regenerated at full width with enlarged typography and clean layouts.

  1. Formatting and page orientation (Weakness 3)

• ICLR header: Added the standard header (“Under review as a conference paper at ICLR 2027”) on page 1 and throughout the document. • Portrait orientation: Eliminated the landscape pages; all 27 pages—including Appendix O (Table 19, pages 25–27)—are now formatted in standard portrait layout.

Area_Chair_FECROpenReview ↗

Please respond to the authors

Hello!

Just a friendly bump that you should engage with the authors response

Reviewer_285COpenReview ↗

Response to the rebuttal

I thank the authors for their responses and revision of the paper. I have also read other reviews, rebuttal responses and discussions. I have the following concerns:

  • I agree with reviewer K2bD that the paper has substantially changed. In my opinion, in that case it should be resubmitted and should go the full new round of reviews again to be accepted.
  • The paper is still extremely hard to read and follow, starting with the wordings of the abstract
  • For the central question that paper now studies, using 2 model families and 300 prompts is insufficient to meaningfully support the claims.

In my opinion, the paper needs a solid decision of what question it aims to study followed by construction of appropriate experimental design based on this decision, presented with clear text. After that, the paper needs to go through a full new cycle of review. Based on the above, I am keeping my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.