- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper introduces SafetyLens, a multi-resolution mechanistic analysis pipeline for studying over-refusal in safety-aligned large language models. The proposed framework starts from behavioral evaluation, uses layer- and token-wise linear probing to identify refusal-related representations, attributes these representations to attention heads and singular vectors, and then applies inference-time ablations and circuit/path analysis to investigate causal relevance. The authors evaluate the approach primarily on Phi-4 and Gemma-4, with Qwen3.6 included as a reference model. They report that refusal-related information is linearly decodable in the hidden states of Phi-4 and Gemma-4, and that selected components can be manipulated to reduce over-refusal. They also use refusal scores to identify influential input tokens and report large reductions in over-refusal (69.03%, 68.72%, and 59.11% for Phi-4, Gemma-4, and Qwen3.6, respectively) after single-word substitutions. Finally, the paper constructs a recurring over-refusal circuit in Phi-4 and reports that circuit ablation reduces over-refusal without increasing toxic compliance.
Critical strengths
The paper provides evidence for causal intervention rather than relying exclusively on probing.
The use of inference-time ablation and path-level intervention is valuable. The results on Phi-4 show that targeted modifications to a very small subset of model components can change the over-refusal rate while leaving toxic-compliance relatively stable. The paper also reports that the circuit-level intervention reduces OR from 5.50% to 5.37% on Phi-4 while leaving TC unchanged at 10.96%. Although the magnitude of the behavioral change is modest, the attempt to close the loop from correlation to causal intervention is an important strength and makes the work more compelling than a purely probing-based analysis.
Critical weaknesses
- The problem and method of the paper isn't novel.
Over-refusal of LLMs, and using internal vectors as indicators to mediate the problem, are well-researcher topics. Some of the highly relevant work are listed below, and I think the author's method doesn't propose authenticly novel ideas beyond them:
-
Dabas et al., Just Enough Shifts: Mitigating Over-Refusal in Aligned Language Models with Targeted Representation Fine-Tuning (ICML 2025)
-
Han et al., SafeSwitch: Steering Unsafe LLM Behavior via Internal Activation Signals (ICML 2026)
-
Maskey, Dras & Naseem: Over-Refusal and Representation Subspaces: A Mechanistic Analysis of Task-Conditioned Refusal in Aligned LLMs (2026)
- The causal evidence is narrower and weaker than the paper's framing suggests.
The paper presents SafetyLens as a general framework for reconstructing causal over-refusal circuits across models, but the strongest circuit-level causal result is demonstrated only on Phi-4. The circuit-ablation table does not report a corresponding circuit result for Gemma-4, while Qwen3.6 is explicitly acknowledged to have poor linear-probe performance (AUROC around 0.6) and is largely excluded from the subsequent mechanistic analysis. More importantly, even on Phi-4, the reduction from 5.50% to 5.37% OR is relatively small in absolute terms. Therefore, the current experiments demonstrate that the identified components can influence behavior, but they provide more limited evidence that the reconstructed circuit is a minimal or sufficient causal mechanism for over-refusal. The paper would be substantially stronger with intervention experiments across more models, stronger effect sizes, and comparisons against appropriately matched random or non-causal components.
I'm willing to raise the score if the author addresses my concerns effectively.
Questions for authors
The paper's writing is somewhat questionable, especially their usage of figures and tables. For instance:
- Fig 1, 2: They're not informative, and their presentation style is a bit sloppy.
- Fig 3c: Using word cloud to illustrate the most frequent words is unnecessary (it doesn't provide more information than listing them).
- Fig 4b, 5: Their font is too small, making it hard to read. Also, some of the characters overlap with each other.
Our work differs from that of Dabas et al. in examining both internal interventions and prompt rewriting. The former provides causal evidence for the identified mechanisms, while the latter evaluates whether these insights can inform practical refusal-behavior mitigation without modifying model parameters. Importantly, all model parameters remain frozen in both approaches (Line 273). This provides a complementary perspective to Dabas et al.'s work, which primarily focuses on improving safety through fine-tuning.
Han et al. proposed a correlation-based method to quantify unsafe and compliance scores and introduced a lightweight conditional refusal head to steer refusal behavior. Although their approach shares the use of ablation with ours, it does not investigate how refusal-related information is propagated through the model. As shown in Figure 3(a) of Han et al., their probes fail to detect safety information in the early layers, and thus their ablation experiments are limited to later layers. However, the inability of a probe to detect safety information does not necessarily indicate that such information is absent. Instead, as we explain in our paper (Line 415), “The absence of a detectable refusal direction in the residual stream does not necessarily imply that refusal-related information is absent from the model. Instead, the information may initially be represented within specific attention heads and is not yet directly accessible to a linear classifier applied to the summation of attention output and the residual stream.”
Although both our work and that of Maskey, Dras, & Naseem involve the concept of a refusal direction, our work does not aim to characterize refusal representations in high-dimensional spaces. Instead, we use the refusal direction as an analytical tool to investigate how refusal-related information is propagated through the model and identify the components involved in this process. As clarified in Line 478, an in-depth representation-level analysis of over-refusal signals is beyond the scope of this work.
Overall, our work differs from prior studies in its focus on understanding how refusal-related information propagates through model components, combining complementary causal and behavioral analyses using parameter-free interventions and prompt rewriting. Rather than focusing on fine-tuning, refusal representation characterization, or refusal steering alone, we investigate the underlying mechanisms of refusal behavior and explore how these insights can inform practical mitigation strategies, although mitigation is not our primary objective.
Modest improvement for ablation: We agree that the reduction in over-refusal from ablation is modest, particularly on Phi-4, and we will revise the paper to avoid presenting these interventions as the primary practical mitigation strategy. That said, as reported in line 431, the results showed a consistent reduction in over-refusal across different intervention methods. This consistency is particularly noteworthy given that the baseline over-refusal rate is already relatively low (5.50%). In this setting, arbitrary modifications to the model's hidden states could just as easily increase the over-refusal rate, yet all four interventions resulted in a reduction. Importantly, these modest but consistent reductions in over-refusal are themselves informative, suggesting that the interventions can influence Phi-4's refusal behavior, even if their effects are limited in magnitude. The purpose of these experiments is primarily mechanistic rather than practical: after identifying components associated with over-refusal using probes and circuit analysis, the goal is to intervene on those components to test whether they have a causal role in the behavior. Thus, internal intervention results provide causal validation of identified mechanisms whereas prompt rewriting (by swapping words) evaluates whether the mechanistic insights can inform an effective refusal behavior mitigation without modifying model params.
We appreciate the reviewer's feedback on font size and presentation style. We'll make modifications accordingly in the next revision. In addition, we would like to clarify that Fig 3c is not the "most frequent words." Instead, they are the "harmful score" predicted by the linear classifier, as explained in Line 357.