acceptodds
Under review as a conference paper at ICLR 2027

SafetyLens: Multi-Resolution Causal Analysis of Over-Refusal in Large Language Models

Abstract

Safety-aligned language models should carefully mitigate harmful requests. However, these safety mechanisms can also lead to over-refusal (False Positives), where models reject requests that are benign and should have been permitted. There are many reasons why such over-refusals manifest, including disagreements in model usage, usage policies, and, most importantly, the over-sensitivity of existing evaluations. While these evaluations can measure over-refusal behavior, they provide limited insight into the internal computations that give rise to it. To address this issue, we introduce SafetyLens, a mechanistic analysis pipeline that connects behavioral evaluation to internal model calculation. We evaluate SafetyLens on Phi-4, Gemma-4, and Qwen3.6. Using linear classifiers, we analyze the models’ internal representations and identify nodes, including tokens, layers, attention heads, and singular vectors, that are strongly associated with over-refusal. We then perform inference-time ablations on these identified nodes, observing reductions in over-refusal. We further apply this analysis to the input tokens, identifying those whose representations are strongly associated with over-refusal. By replacing these tokens with semantically equivalent alternatives that preserve the original sentence meaning, we reduce over-refusal by 69.03% on Phi-4, 68.72% on Gemma-4, and 59.11% on Qwen3.6. Based on these findings, we reconstruct the over-refusal circuits within each model and identify the components that contribute to the behavior. Intervening on or ablating these circuits at inference time, without modifying the input sentence, reduces over-refusal while preserving the models’ ability to reject harmful queries.

Primary area:
interpretability and explainable AI
Keywords:
Mechanistic Interpretability, Over-refusal, LLM

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_LTvmOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces SafetyLens, a multi-resolution mechanistic analysis pipeline for studying over-refusal in safety-aligned large language models. The proposed framework starts from behavioral evaluation, uses layer- and token-wise linear probing to identify refusal-related representations, attributes these representations to attention heads and singular vectors, and then applies inference-time ablations and circuit/path analysis to investigate causal relevance. The authors evaluate the approach primarily on Phi-4 and Gemma-4, with Qwen3.6 included as a reference model. They report that refusal-related information is linearly decodable in the hidden states of Phi-4 and Gemma-4, and that selected components can be manipulated to reduce over-refusal. They also use refusal scores to identify influential input tokens and report large reductions in over-refusal (69.03%, 68.72%, and 59.11% for Phi-4, Gemma-4, and Qwen3.6, respectively) after single-word substitutions. Finally, the paper constructs a recurring over-refusal circuit in Phi-4 and reports that circuit ablation reduces over-refusal without increasing toxic compliance.

Critical strengths

The paper provides evidence for causal intervention rather than relying exclusively on probing.

The use of inference-time ablation and path-level intervention is valuable. The results on Phi-4 show that targeted modifications to a very small subset of model components can change the over-refusal rate while leaving toxic-compliance relatively stable. The paper also reports that the circuit-level intervention reduces OR from 5.50% to 5.37% on Phi-4 while leaving TC unchanged at 10.96%. Although the magnitude of the behavioral change is modest, the attempt to close the loop from correlation to causal intervention is an important strength and makes the work more compelling than a purely probing-based analysis.

Critical weaknesses
  1. The problem and method of the paper isn't novel.

Over-refusal of LLMs, and using internal vectors as indicators to mediate the problem, are well-researcher topics. Some of the highly relevant work are listed below, and I think the author's method doesn't propose authenticly novel ideas beyond them:

  • Dabas et al., Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning (ICML 2025)

  • Han et al., SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signals (ICML 2026)

  • Maskey, Dras & Naseem: Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs (2026)

  1. The causal evidence is narrower and weaker than the paper's framing suggests.

The paper presents SafetyLens as a general framework for reconstructing causal over-refusal circuits across models, but the strongest circuit-level causal result is demonstrated only on Phi-4. The circuit-ablation table does not report a corresponding circuit result for Gemma-4, while Qwen3.6 is explicitly acknowledged to have poor linear-probe performance (AUROC around 0.6) and is largely excluded from the subsequent mechanistic analysis. More importantly, even on Phi-4, the reduction from 5.50% to 5.37% OR is relatively small in absolute terms. Therefore, the current experiments demonstrate that the identified components can influence behavior, but they provide more limited evidence that the reconstructed circuit is a minimal or sufficient causal mechanism for over-refusal. The paper would be substantially stronger with intervention experiments across more models, stronger effect sizes, and comparisons against appropriately matched random or non-causal components.

I'm willing to raise the score if the author addresses my concerns effectively.

Questions for authors

The paper's writing is somewhat questionable, especially their usage of figures and tables. For instance:

  • Fig 1, 2: They're not informative, and their presentation style is a bit sloppy.
  • Fig 3c: Using word cloud to illustrate the most frequent words is unnecessary (it doesn't provide more information than listing them).
  • Fig 4b, 5: Their font is too small, making it hard to read. Also, some of the characters overlap with each other.
1 comment
  1. We thank Reviewer LTvm for pointing out these relevant works. We would like to clarify several distinctions between our contributions and prior studies.

Our work differs from that of Dabas et al. in examining both internal interventions and prompt rewriting. The former provides causal evidence for the identified mechanisms, while the latter evaluates whether these insights can inform practical refusal-behavior mitigation without modifying model parameters. Importantly, all model parameters remain frozen in both approaches (Line 273). This provides a complementary perspective to Dabas et al.'s work, which primarily focuses on improving safety through fine-tuning.

Han et al. proposed a correlation-based method to quantify unsafe and compliance scores and introduced a lightweight conditional refusal head to steer refusal behavior. Although their approach shares the use of ablation with ours, it does not investigate how refusal-related information is propagated through the model. As shown in Figure 3(a) of Han et al., their probes fail to detect safety information in the early layers, and thus their ablation experiments are limited to later layers. However, the inability of a probe to detect safety information does not necessarily indicate that such information is absent. Instead, as we explain in our paper (Line 415), “The absence of a detectable refusal direction in the residual stream does not necessarily imply that refusal-related information is absent from the model. Instead, the information may initially be represented within specific attention heads and is not yet directly accessible to a linear classifier applied to the summation of attention output and the residual stream.”

Although both our work and that of Maskey, Dras, & Naseem involve the concept of a refusal direction, our work does not aim to characterize refusal representations in high-dimensional spaces. Instead, we use the refusal direction as an analytical tool to investigate how refusal-related information is propagated through the model and identify the components involved in this process. As clarified in Line 478, an in-depth representation-level analysis of over-refusal signals is beyond the scope of this work.

Overall, our work differs from prior studies in its focus on understanding how refusal-related information propagates through model components, combining complementary causal and behavioral analyses using parameter-free interventions and prompt rewriting. Rather than focusing on fine-tuning, refusal representation characterization, or refusal steering alone, we investigate the underlying mechanisms of refusal behavior and explore how these insights can inform practical mitigation strategies, although mitigation is not our primary objective.

  1. Modest improvement for ablation: We agree that the reduction in over-refusal from ablation is modest, particularly on Phi-4, and we will revise the paper to avoid presenting these interventions as the primary practical mitigation strategy. That said, as reported in line 431, the results showed a consistent reduction in over-refusal across different intervention methods. This consistency is particularly noteworthy given that the baseline over-refusal rate is already relatively low (5.50%). In this setting, arbitrary modifications to the model's hidden states could just as easily increase the over-refusal rate, yet all four interventions resulted in a reduction. Importantly, these modest but consistent reductions in over-refusal are themselves informative, suggesting that the interventions can influence Phi-4's refusal behavior, even if their effects are limited in magnitude. The purpose of these experiments is primarily mechanistic rather than practical: after identifying components associated with over-refusal using probes and circuit analysis, the goal is to intervene on those components to test whether they have a causal role in the behavior. Thus, internal intervention results provide causal validation of identified mechanisms whereas prompt rewriting (by swapping words) evaluates whether the mechanistic insights can inform an effective refusal behavior mitigation without modifying model params.

  2. We appreciate the reviewer's feedback on font size and presentation style. We'll make modifications accordingly in the next revision. In addition, we would like to clarify that Fig 3c is not the "most frequent words." Instead, they are the "harmful score" predicted by the linear classifier, as explained in Line 357.

Reviewer_WkYsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

SafetyLens studies over-refusal through a pipeline combining linear probing, attention-head and singular-mode attribution, inference-time interventions, and counterfactual rewriting. It uses refusal-flipping prompt pairs to identify computational paths through activation patching and aggregates these paths into candidate over-refusal circuits. The paper evaluates whether targeted interventions reduce over-refusal while maintaining refusal of harmful requests.

Critical strengths
  1. The paper addresses a relevant problem: understanding the internal mechanisms of inappropriate refusal on benign requests.
  2. The pipeline connects behavioral evaluation with component-level analysis and intervention, and reports both over-refusal and toxic compliance.
  3. Minimal prompt substitutions provide a useful basis for constructing contrastive inputs for mechanistic analysis.
Critical weaknesses
  1. The contribution beyond prior work is insufficiently established. Refusal-direction interventions [1], safety-head attribution and ablation [2], representational analysis of over-refusal [3], and refusal-trigger analysis [4] already cover substantial parts of the motivation and methodology. Although the paper cites [1], it does not adequately distinguish its contributions from [2–4]. Combining multiple analysis granularities does not, by itself, establish a new mechanistic finding.
  2. The evidence does not establish an over-refusal-specific circuit. Contrasting refused and accepted benign prompts can identify general refusal mechanisms. An unchanged aggregate toxic-compliance rate after intervention does not establish that the identified circuit contributes exclusively to over-refusal, as suggested in Section 7.
  3. Behavioral improvements from internal interventions are limited or involve safety degradation. In Table 1, Phi-4 circuit ablation reduces over-refusal from 5.50% to 5.37%, without uncertainty estimates. On Gemma, orthogonal ablation reduces over-refusal from 2.61% to 2.09% but increases toxic compliance from 15.07% to 20.55%. These results provide limited support for mitigation without compromising safety. The much larger reductions highlighted in the abstract concern prompt rewriting, not circuit intervention.
  4. Circuit validation relies heavily on a proxy objective. Path selection maximizes recovery of a learned probe’s refusal score, which does not necessarily imply recovery of actual refusal behavior. The reported behavioral evaluation offers limited corroboration, and matched random-component controls and systematic circuit sufficiency/necessity tests are missing.
  5. The rewriting results do not isolate the value of probe-guided token selection. Without random-token substitution or unguided minimal-rewriting baselines, the reported reductions cannot establish that the probe identifies uniquely informative triggers. Semantic preservation is not systematically validated, and restricting circuit discovery to successful refusal-flipping edits limits the scope of the resulting conclusions.

References

[1] Refusal in Language Models Is Mediated by a Single Direction.

[2] On the Role of Attention Heads in Large Language Model Safety.

[3] Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs.

[4] Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment.

Questions for authors

See weaknesses

1 comment
  1. We thank Reviewer WkYs for highlighting these relevant works. We would like to clarify several distinctions between our contributions and prior studies.

Unlike [2], which primarily uses attention-head scaling or zeroing, we identify a refusal direction and use it to guide targeted interventions, including orthogonal ablation, to examine refusal-related information.

Unlike [3], which focuses on characterizing refusal representations, we use the refusal direction as an analytical tool to trace how refusal-related information propagates through the model and identify the components involved.

Finally, our work investigates both internal interventions and prompt rewriting. The former provides causal evidence for the identified mechanisms, while the latter evaluates whether these insights can inform practical strategies for mitigating over-refusal behavior without modifying model parameters. Importantly, all model parameters remain frozen in both approaches (Line 273). In this respect, our work offers a complementary perspective to [4], which primarily focuses on improving safety through fine-tuning, and [2], which also explores parameter modification as part of its broader investigation.

Overall, our work differs from prior studies by using a refusal direction to guide targeted internal interventions, investigating how refusal-related information propagates through model components rather than characterizing its representation, and exploring prompt rewriting as a practical mitigation strategy. These complementary analyses provide both mechanistic insights into over-refusal and parameter-free approaches to mitigating it, with all model parameters kept frozen.

  1. Modest improvement for ablation: We agree that the reduction in over-refusal from ablation is modest, particularly on Phi-4, and we will revise the paper to avoid presenting these interventions as the primary practical mitigation strategy. That said, as reported in line 431, the results showed a consistent reduction in over-refusal across different intervention methods. This consistency is particularly noteworthy given that the baseline over-refusal rate is already relatively low (5.50%). In this setting, arbitrary modifications to the model's hidden states could just as easily increase the over-refusal rate, yet all four interventions resulted in a reduction. Importantly, these modest but consistent reductions in over-refusal are themselves informative, suggesting that the interventions can influence Phi-4's refusal behavior, even if their effects are limited in magnitude. The purpose of these experiments is primarily mechanistic rather than practical: after identifying components associated with over-refusal using probes and circuit analysis, the goal is to intervene on those components to test whether they have a causal role in the behavior. Thus, internal intervention results provide causal validation of identified mechanisms whereas prompt rewriting (by swapping words) evaluates whether the mechanistic insights can inform an effective refusal behavior mitigation without modifying model params.

  2. Meaning of circuit: We would like to clarify that our goal is not to establish that the identified circuit is exclusively responsible for over-refusal, but to identify components that can be targeted to mitigate over-refusal while minimally affecting other safety-related behaviors. Thus, the behavioral results support the intended role of the identified circuit, even though they do not imply that it exclusively governs refusal behavior.

  3. Semantic preservation: Although the reductions in over-refusal suggest that the estimated refusal scores provide useful information for identifying lexical cues associated with over-refusal, the primary purpose of our counterfactual datasets was togenerate minimally modified pairs of inputs for subsequent causal circuit validation, rather than to establish the effectiveness or uniqueness of individual token-level refusal scores. In particular, we use these paired inputs for circuit pruning and validation. We agree that comparisons with random-token substitution and unguided rewriting, together with a more systematic evaluation of semantic preservation, would strengthen our claims about probe-guided token selection. We will clarify the experimental objective and narrow our claims accordingly in the revised manuscript.

Reviewer_Zg3kOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The authors study over-refusal in language models. They use linear probes to find related model components. They test internal interventions and prompt rewriting. Reducing over-refusal while preserving safety is a useful goal.

Critical strengths
  • Clear analysis process. Figure 1 presents a clear analysis process. The authors connect over-refusal evaluation with linear probes, attention heads, singular vectors, and internal interventions. They test model behavior after these interventions. This process gives readers a clear way to study over-refusal inside the model.
  • Useful observations about refusal information. The authors report useful observations about where refusal information appears. Figure 3 compares layers and token positions. Figure 5 shows paths from an edited token to the final token. It also shows paths that recur across prompts. These observations give later studies specific components and paths to test.
Critical weaknesses
  • Small gains from internal interventions. In Table 1, circuit ablation only slightly lowers Phi-4's over-refusal rate. A change this small may come from random variation.

    The authors report much larger gains from prompt rewriting. They keep the model unchanged in these tests. The model may still reject the original harmless prompt. The main practical gain comes from changing the prompt. The authors should make this gap clear when they describe the contribution of the internal interventions.

  • Missing checks on meaning after prompt rewriting. The authors claim that the word replacement in Figure 2 keeps the meaning of the request. In Section 8, they state that they have not done a full check of meaning changes.

    Replacing a word can change the task or make the request less risky. The model may answer because it receives a different request. The authors should check the task, constraints, and safety label before and after rewriting. They use these prompt pairs to find circuits. Changes in meaning would affect that analysis.

  • The internal interventions can weaken safety. In Table 1, orthogonal ablation gives a small reduction in over-refusal on Gemma-4. It also causes a much larger increase in the harmful-request acceptance rate.

    This safety cost is large compared with the benefit. The authors should describe this result as a safety trade-off. They should revise their general claim that the interventions preserve safety.

Questions for authors

Please see the weaknesses.

1 comment

We appreciate reviewer Zg3k's comments and would like to clarify a few ideas in this paper

  1. Modest improvement for ablation: We agree that the reduction in over-refusal from ablation is modest, particularly on Phi-4, and we will revise the paper to avoid presenting these interventions as the primary practical mitigation strategy. That said, as reported in line 431, the results showed a consistent reduction in over-refusal across different intervention methods. This consistency is particularly noteworthy given that the baseline over-refusal rate is already relatively low (5.50%). In this setting, arbitrary modifications to the model's hidden states could just as easily increase the over-refusal rate, yet all four interventions resulted in a reduction. Importantly, these modest but consistent reductions in over-refusal are themselves informative, suggesting that the interventions can influence Phi-4's refusal behavior, even if their effects are limited in magnitude. The purpose of these experiments is primarily mechanistic rather than practical: after identifying components associated with over-refusal using probes and circuit analysis, the goal is to intervene on those components to test whether they have a causal role in the behavior. Thus, internal intervention results provide causal validation of identified mechanisms whereas prompt rewriting (by swapping words) evaluates whether the mechanistic insights can inform an effective refusal behavior mitigation without modifying model params.

  2. Semantic preservation: We agree that preserving the underlying task and constraints is critical. In our rewriting experiments, we took care not to significantly alter the meaning of the original queries. Our intention was to modify only the refusal-triggering lexical cues identified during the probing experiments, while keeping the user's underlying request unchanged. We appreciate the reviewer pointing out that our current writeup does not adequately explain how semantic preservation was addressed in our experimental setup.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.