- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper decomposes LLM refusal into three depth-ordered SAE feature sets, risk encoding, safety integration, refusal execution, using probing-grounded localization and matched-control interventions. It validates the account with bidirectional mediation (necessity and sufficiency) across two models (Llama, Gemma), and reports a response-free refusal detector built on the localized features.
Critical strengths
- The causal methodology is rigorous: matched-control interventions, bidirectional mediation, and two-model replication are stronger than the correlational norm in this subfield.
- The stage definitions are grounded in probe curves rather than chosen ad hoc.
- The exploration is thorough, with full ablation grids over feature-set sizes that support scrutiny.
Critical weaknesses
- The most serious issue is that the localized risk-encoding set has a negative causal effect. At the selected size (n=80), ablating the risk-encoding set yields ΔM = −2.66 (Table 5), meaning that removing it increases the refusal margin, the opposite of what a pathway component supporting refusal should do. The authors explain this away as "distributed and redundant" (Line 377), but if the signal is reconstructable then the localized features are not causally mediating, and the three-stage claim collapses to two stages (integration and execution) plus an upstream correlate.
- The chance-level jailbreak probe is hard to reconcile with the activation dissociation. Figure 1's jailbreak probe sits at chance across all layers, yet Figure 4 reports graded jailbreak-versus-harmful activation dissociation, and the paper never explains how a non-separable linear probe is consistent with a dissociable causal account.
- The matched control degrades under suppression. In Figure 5(b), the matched-control execution strength itself drops from about 0.6 at v=0 to about 0.4 at v=−5. If the control isolates same-layer, activation-matched but unrelated features, it should not degrade when the target set is suppressed. The paper does not state whether the control is suppressed in parallel, which would be a valid difference-in-differences design, or merely collaterally damaged, which would invalidate the specificity claim.
- The detector framing is imprecise and oversells a non-improvement. The "response-free" detector is validated on held-out prompts drawn from the same distribution as the localization corpus, so this is in-distribution generalization rather than out-of-distribution zero-shot detection. More importantly, Table 1 shows it is beaten on accuracy and F1 by a plain linear probe, SIREN, WildGuard, and (on Gemma) GradSafe; the authors concede the readout "transfers but does not lead in accuracy" (p.8, l.1091–1092). Its only claimed value is stage-wise attribution of the decision, so it is a demonstration of readability rather than a deployable detector.
Questions for authors
See Weakness.
Thank you for the additional experiments and clarifications.