- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper examines whether refusal directions in language models are stable and how interventions along these directions affect refusal behavior. The authors compare different methods for identifying refusal directions and test three intervention methods across four models and two datasets. They use Signal Detection Theory to distinguish refusal of harmful prompts from over-refusal of harmless prompts. They also study whether individual prompts respond differently to small interventions. The results suggest that the selected direction depends partly on how it is identified, and that behavioral effects vary across intervention methods, prompts, and datasets.
Critical strengths
- The paper addresses an important question about what refusal directions actually control. I particularly appreciate the distinction between refusing harmful requests and over-refusing harmless ones.
- The experimental analysis is careful: (a) the authors evaluate full generated responses; (b) use paired comparisons; and (c) include several robustness checks.
Critical weaknesses
- The paper argues that refusal directions depend on how they are identified, but the alternative directions are never tested on behavior. So it's unclear whether these are actually different meaningful directions, or whether one method simply finds a better direction than another. The conclusion also mentions factors like the layer and the model's later processing, which weren't independently tested.
- All refusal labels come from one AI judge, and the authors don't compare its judgments with human labels. They also don't check whether the interventions damage the quality of the responses, or explain how broken or meaningless responses are classified. This makes it difficult to know whether the changes in refusal rates reflect actual changes in refusal behavior.
- The Gᵢ analysis uses much smaller interventions and measures only the probability of starting with a refusal word, while the main experiments use larger interventions and evaluate full responses. The predictions also don't hold at the larger intervention strengths used in the main experiments. Thus, I'm not sure what this analysis tells us about the behavioral findings.
Questions for authors
- Since d′ and c are used only descriptively, what do they add beyond the harmful and harmless refusal rates?
- On XSTest, the models already refuse almost all harmful prompts before the intervention. Could this explain some of the differences between datasets, simply because there's less room for refusal to increase?
- In the Gᵢ analysis, wouldn't we expect each prompt's own slope to predict its changes better than an average slope, especially when the relationship is almost linear? It seems to me that the pairing test is the more informative result.
- For Qwen, the stability test in Table 1 selected different directions from those used in the main experiments in Table 2. How does this support the stability claim in Section 5.1?
- Why was the intervention applied during the whole response in the early qualification check, but only during the prompt in the main experiments? Could this affect the results?
- The three intervention methods use different strength scales, and ablation has no strength scale at all. How can we tell whether the differences between them come from the methods themselves rather than from different strengths?
- The paper mentions earlier work by Wollschläger et al. on multiple refusal directions, but the abstract seems to present this as a new finding. What does this paper add beyond that work?
- Many of the intervals for changes in d′ are wide and include zero. How much confidence should we place in conclusions based on these changes?
- Could the authors clarify what counts as a "valid prompt position," how the Arditi and Wang prompt sets differ, and what the "literal control" is meant to test?
- How were the 150 harmful and 150 harmless prompts selected from each dataset? Could differences between the datasets beyond their baseline refusal rates explain some of the results?
We thank the reviewer for the detailed feedback on terminology, mathematical notation, and related literature. We have thoroughly rewritten the manuscript to make the central aim clear from the Abstract and Introduction, replaced all non-standard phrasing with direct scientific terminology, simplified and fully defined every equation, and integrated the missing literature in Section 1 and Section 2.
• Clear statement of purpose: The revised Abstract and Section 1 (Introduction) directly state the paper’s scope and central question: after constructing and auditing one refusal direction per checkpoint (Section 3.2), we freeze that direction and study how a common contrastive activation addition (CAA) displacement affects different prompts—evaluating both complete-response refusal rates (H,F and descriptive SDT indices c,d′ in Section 4) and continuous first-token refusal log-odds scores (si(k) and local gains Gi in Section 5). • Complete removal of non-standard phrasing: Every phrase flagged by the reviewer—including "Neither is fixed", "operational causal approximation", "operational causal interface", the rhetorical "the", and undefined "lexical evidence"—has been removed. • Lines 12–28 (Abstract) & Lines 32–66 (Introduction): Rewritten in plain, precise terms describing how CAA adds a constant displacement vector kr and how prompt-specific local gains Gi and their within-class covariance with baseline scores Cov(si(0),Gi∣y) govern changes in class means and spread. • Section 3.2 (Direction Selection and Validation, page 3): Directly introduces the candidate bank (mean harmful-minus-benign residual-stream activations across candidate layer–position sites) and defines the first-token refusal proxy (the log-odds of the summed probabilities of tokens I, It for Gemma 2 or I, As for Qwen 2.5, with exact token IDs in Appendix A, Table 2).
We moved the supplementary operators (single-site scaling and all-layer ablation) to Appendix F so the main text focuses cleanly on CAA. This allowed us to replace the overloaded multi-operator notation in the original submission—specifically eliminating the undefined operator superscript (m), the vector symbol jit(m), and the transform notation Ψb(Hi(m))—with nine self-contained equations (Equations 1–9):
• Equation (1) (Section 3.3, page 3 — The Common CAA Write):
hit′=hit+kr,k=α∣rraw∣2
where hit∈Rd is the residual-stream activation vector for prompt i at non-padding prompt position t, rraw is the selected activation contrast, r=rraw/∣rraw∣2 is the unit direction, α is the implemented CAA coefficient, and k is the signed geometric displacement. The standardized displacement is explicitly defined right below as z=k/σρ=α∣rraw∣2/σρ, where ρi=⟨hi,tibase,r⟩ is the baseline coordinate at the selected layer input and final non-padding prompt position ti, and σρ is its sample standard deviation in the fixed validation reference set (Appendix E, Equation 11, Table 5). • Equations (2)–(5) (Section 3.4, page 4 — Behavioral & Continuous Score Measurements): Explicitly define the Hautus (1995) log-linear corrected refusal rates H,F (Eq. 2), the probit SDT indices d′ and c (Eq. 3), the continuous first-token log-odds score si(k)=log[pT,i(k)+ϵ]−log[1−pT,i(k)+ϵ] (Eq. 4), and the standardized continuous-score separation dG′(k) (Eq. 5). • Equations (6)–(9) (Sections 5.1–5.2, page 6 — Local Gain & Moment Decomposition): Define each prompt’s zero-dose local gain Gi=∂k∂si(k)k=0 and linear reconstruction si(k)=si(0)+kGi (Eq. 6), the reconstructed class mean and within-class variance relations (Eqs. 7–8), and the exact first-order derivative decomposition of dG′(k) into its class-mean contribution and pooled-spread contribution (Eq. 9, derived step-by-step in Appendix K.2).