acceptodds
Under review as a conference paper at ICLR 2027

SafeEvo: Deciphering the Safety Alignment Mechanism and Evolution in Language Models

Abstract

Safety interpretability advances the study of Large Language Model (LLM) alignment from behavioral constraints driven by data or algorithms towards a deeper understanding of internal mechanisms. However, existing works have focused primarily on safety-related representations, attention heads, or neurons after alignment, while largely overlooking the safety mechanisms in pretrained-only models and their evolution across alignment checkpoints. To address this, we propose SafeEvo, an interpretability framework from the circuit (sparse subgraphs of an LLM) perspective. SafeEvo first applies an optimization-based extraction algorithm to identify weak refusal circuits in pretrained base LLMs that can independently express refusal behavior. Causally ablating these circuits completely eliminates the base model’s refusal of harmful inputs. SafeEvo then traces the evolution of refusal circuits across successive alignment checkpoints and finds that their structures change progressively, suggesting that the alignment tax may result from refusal-circuit updates affecting utility-related parameters. To validate this, SafeEvo introduces Safety Circuit Alignment (SCA), which confines safety updates to the refusal circuits. Experiments across three LLMs and two alignment algorithms show that, on average, SCA outperforms vanilla alignment in three aspects: (1) stronger alignment, lowering harmfulness score by 63.21%; (2) less over-refusal, yielding a 58.44% decrease in refusal rates for benign queries; and (3) better utility, retaining 99.58% of the original model capabilities.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Safety, Safety Alignment, Safety Interpretability

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_94x5OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether safety mechanisms exist in pretrained base LLMs and how they evolve during safety alignment, moving safety interpretability beyond static endpoint analyses of already-aligned models. SafeEvo casts refusal-circuit discovery as mask optimization: each MLP neuron (rows of gate/up/down projections) receives a learnable binary mask trained with a straight-through estimator under a dual-branch objective—the circuit branch is fine-tuned on refusal completions (sufficiency), the complement branch on harmful-compliance completions (ruling out the trivial full selection), plus an L1 sparsity penalty. Applied to Llama-3-8B, Qwen-2.5-7B, and Mistral-7B-Instruct-v0.1 (the abstract writes "Mistral-7B-v0.1," but §4.2 states the instruct variant is used), the extracted circuits contain under 1% of parameters, achieve a 100% out-of-distribution refusal rate on BeaverTails while their complements never refuse, and ablating them eliminates base-model refusal (refusal rates drop to 0–0.63%) while raising HarmBench ASR from 32.7–48.4% to 88.1–91.2% (density-matched random ablations raise ASR far less: 49.1/64.2/35.2% vs. 91.2/88.7/88.1% for circuit ablation). Tracking circuits across 20 SFT checkpoints on Llama-3-8B reveals that independent extractions at the same checkpoint share only 26% of components (non-uniqueness) and that warm-started consecutive-checkpoint overlap decays from 98% to 78% (gradual drift), from which the authors hypothesize that the alignment tax stems from underconstrained refusal-circuit updates spilling into utility parameters. To test this, they propose Safety Circuit Alignment (SCA), which confines LoRA updates (both SFT and DPO) to the extracted refusal circuit. Across three models and two objectives, SCA beats full-LoRA on safety (e.g., Llama DPO HarmBench ASR 0.63% vs 16.98%), utility (maximum drop 1.44 points; several benchmarks improve), and over-refusal (XSTest 1.6% vs 18.8% on Llama SFT), and beats a density-matched random-support baseline by 7.2–14.8 ASR points with prompt-level bootstrap CIs excluding zero over four training seeds.

Critical strengths

(I) A clean causal case that refusal circuits exist in pretrained base models. The evidence structure is exactly what this kind of claim requires: sufficiency (the isolated circuit refuses 100% of OOD harmful queries with under 1% of parameters, across all three families), necessity (ablating the circuit zeroes out refusal and roughly doubles or triples ASR), and a density-matched random-ablation control showing the effect is specific to the identified neurons rather than the lesion size (ASR 49.06/64.15/35.22% under random ablation vs 91.19/88.68/88.05% under circuit ablation). Prior evidence that base models "encode" harmfulness was representational and correlational; this paper upgrades it to a causal, circuit-level statement. (One caveat on generality: the Mistral family is the instruct variant—see Soundness (5)—so the "pretrained" claim strictly rests on Llama and Qwen.) This is consequential for my assessment because it establishes the paper's empirical foundation independently of the weaker downstream hypothesis work. (II) The SCA experiment is a well-controlled test that interpretability findings can pay off in alignment. The comparison design isolates exactly the right variable: SCA and the random-support baseline share the update parameterization (LoRA rank 16), the sparsity budget, the training data, and the objective, differing only in whether the updated rows lie in the extracted refusal circuit. The consistent advantage across three models (Δ ASR 7.2–14.8 points), four training seeds, and prompt-level bootstrap CIs excluding zero makes the "the circuit matters, not the sparsity" conclusion credible, and the simultaneous improvement on safety, utility, and over-refusal against full-LoRA is a practically useful demonstration that localized alignment is viable. The negative control provided by SN-Tune (safety neurons: ASR 0 but GSM8K −18.8 and over-refusal 86.4% on Llama) further sharpens the point that the granularity of the localized structure matters.

Critical weaknesses

(I) Insufficient delineation of the methodological contribution relative to the authors' own prior work. SafeSeek (Yu et al., 2026; arXiv 2603.23268, March 2026, with an author-list overlap on this submission's reference entry) proposes: (a) optimization-based extraction of functionally complete safety circuits via differentiable binary masks trained by gradient descent on safety data, and (b) Safety Circuit Tuning—fine-tuning confined to the extracted sparse circuits. These are precisely the two methodological pillars of this paper (Eq. 5–6 for extraction; Eq. 7–8 for SCA). The submission cites SafeSeek once, in Related Work, as adopting a "circuit-level view," and the introduction frames circuit extraction as this paper's departure from heuristic attribution without mentioning that the optimization-based formulation was already published by the same group. The genuinely new elements—the pretrained-base setting, the warm-started checkpoint-to-checkpoint tracking, the Jaccard non-uniqueness/drift analysis, and the alignment-tax hypothesis—are substantial and interesting; but readers cannot tell where the published method ends and the new contribution begins. Under ICLR norms this relationship should be stated explicitly in the introduction and contributions list. (II) The central mechanistic claim (the alignment-tax account) is hypothesized, not measured, and the supporting evidence admits unexcluded alternatives. The chain is: circuits drift during alignment → drift implies updates touch the circuit → unconstrained updates therefore also touch utility parameters → confining updates avoids the tax. Each link beyond the first is plausible but untested: the paper does not measure weight movement outside the circuit during vanilla alignment, does not correlate such movement with utility degradation, and does not rule out that SCA's utility preservation comes from the circuit's location being incidentally far from capability-critical parameters (a property a random circuit of the same size in a different location might also partially enjoy, which the random baseline only partially controls since it matches size, not location quality). Until the spillover is measured—or at least until utility loss is shown to correlate with circuit-adjacent update magnitude across checkpoints—the alignment-tax mechanism remains one of several live explanations, and the paper's framing ("validating our explanation for alignment tax") overstates what the SCA experiment establishes.

Questions for authors

(I) The abstract reports "lowering harmfulness score by 63.21%," "58.44% decrease in refusal rates," and "retaining 99.58% of capabilities," and the introduction reports "2.29/6.47-point (43.81%/74.94%)" ASR reductions. I was able to reverse-engineer all of these from Table 2 (63.21% = total SCA ASR over total full-LoRA ASR across all 12 model–objective–benchmark cells; 99.58% = mean of the 18 per-cell utility ratios; the introduction's figures use per-model means over the two safety benchmarks), but the paper never states any of these protocols, and the aggregates conceal per-cell reversals: on Qwen SFT, SCA's HarmBench ASR is 6.92% vs full-LoRA's 0.63%. Please state the aggregation explicitly and report per-cell values, or at least acknowledge where SCA loses to full-LoRA. (II) The evolution study (Section 3.3) is conducted only on Llama-3-8B with SFT. Does non-uniqueness (~26% overlap) and gradual drift (98%→78%) replicate on Qwen-2.5-7B/Mistral and under DPO? If yes, please report; if not, the generalization of the alignment-tax account to "vanilla alignment" needs qualification. (III) Extraction uses only the first 100 LLM-LAT prompts for training. How sensitive is the extracted circuit (and the downstream SCA results) to extraction-set size and source? Does the circuit remain functionally complete with 50 or 25 prompts, or with a different harmful-prompt corpus? (IV) Table 2 reports base Llama HarmBench ASR 50.31% while Table 3 reports 64.5%. Please state explicitly that the latter is computed on the 200 XSTest unsafe-contrast prompts (the judge is the same Llama-Guard-3, per Appendix D), and harmonize the presentation.

3 comments
AuthorsOpenReview ↗

[Part 1] Responses to Weakness (I) & (II) and Question (I) & (II)

Dear Reviewer 94x5,

Thank you for the detailed feedback and efforts for reviewing. Below, we provide point-to-point responses to your concerns and hope them are addressed.

For Weakness (I): Relationship to SafeSeek [1]

Thank you for pointing this out. We will draw this boundary explicitly in the introduction. We also politely point out the differences between them in terms of the research question and setting, which are new in SafeEvo:

  • Pretrained-only setting. SafeSeek attributes safety circuits in already-aligned models (LLaMA-3.1-8B-Instruct, Qwen-3-8B). SafeEvo asks where refusal comes from, and shows that refusal circuits already exist before any safety alignment (Table 1, Figure 3, Table 9).
  • Dynamic tracking. SafeEvo re-extracts the circuit across alignment checkpoints by warm-starting and measures non-uniqueness and drift (Section 3.3); SafeSeek analyzes fixed models.
  • Use of the circuit. SafeSeek's circuit tuning removes backdoors or preserves safety during helpfulness fine-tuning. SCA performs safety alignment itself (SFT and DPO) on the pre-alignment circuit, against full-LoRA, random supports, and SN-Tune (Tables 2–4).

For Weakness (II): What the SCA experiment establishes

  • This is indeed a hypothesis partially validated by our SCA experiments. Its main value is to offer a new and insightful explanation for alignment tax; a rigorous and complete proof requires more extensive further work.

  • Our core interpretability logic is consistent with prior work on safety attention heads [2] and neurons [3]: components whose ablation degrades model safety are safety-related, and these studies also compare against random baselines. Strictly following this logic, we study alignment mechanisms through more functionally complete safety circuits that can independently express safety capabilities, which prior work cannot achieve.

  • Thank you for pointing this out. We will revise the relevant wording in the next version.

For Question (I): Aggregation protocols and per-cell reversals

  • Aggregation protocol. With SiS_i and LiL_i denoting SCA and full-LoRA ASRs over the 12 model–objective–benchmark cells, the overall reduction is 100(1−∑iSi/∑iLi)=63.21%100(1-\sum_i S_i/\sum_i L_i) = 63.21\% (we will also replace "harmfulness score" with "ASR" in the abstract). Over-refusal uses the same ratio-of-sums formula over the six XSTest benign cells (58.44%); utility retention is the mean of the 18 per-cell ratios USCA/UbaseU_{\mathrm{SCA}}/U_{\mathrm{base}} (99.58%); and the introduction's 2.29/6.47-point figures are mean point reductions across the six SFT and six DPO safety cells, with 43.81%/74.94% the corresponding ratios of sums. We will state these formulas explicitly.

  • The aggregate results reflect trade-offs across individual settings. For example, on Qwen SFT in Table 2, SCA/full-LoRA HarmBench ASRs are 6.92%/0.63%, alongside benign refusal rates of 6.8%/28.4% and GSM8K changes of +0.08/−2.57 points. On Mistral SFT, the corresponding values are 2.52%/0.63%, 5.6%/7.6%, and −0.75/−2.80 points. Over-refusal and XSTest unsafe-contrast ASR also vary by setting (Tables 2–3). Overall utility retention is similar (99.58% for SCA vs. 99.55% for full-LoRA, Figure 1D), while SCA shows smaller GSM8K drops under SFT. We will clarify these per-cell trade-offs alongside the aggregation protocols next to Table 2.

For Question (II): Scope of the evolution study

We extended the Section 3.3 tracking protocol to two additional settings: Qwen-2.5-7B with SFT (a second model family) and Mistral-7B-Instruct-v0.1 with DPO (a second family and objective). For each new setting, we extract consecutive warm-started circuits across 20 alignment checkpoints and run three independent extractions (3 seeds) at a fixed checkpoint, alongside the Llama-3-8B trajectory from Section 3.3.

Table R1. Evolution of refusal circuits across settings (Jaccard overlap).

SettingSame-checkpoint overlap: mean (range)Warm-started chain: mean overlap at lag 1 / 10 / 20
Qwen-2.5-7B, SFT0.289 (0.261–0.312)0.990 / 0.865 / 0.724
Mistral-7B-Instruct-v0.1, DPO0.383 (0.378–0.388)0.994 / 0.902 / 0.839

Both qualitative patterns in our findings hold beyond Llama-3-8B SFT, with setting-dependent drift magnitude: (i) non-uniqueness and (ii) gradual drift.

Reference

[1] Yu et al. SafeSeek: Universal Attribution of Safety Circuits in Language Models. arXiv:2603.23268.

[2] Zhou et al. On the Role of Attention Heads in Large Language Model Safety. (ICLR 2025).

[3] Chen et al. Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons. (NeurIPS 2026).

AuthorsOpenReview ↗

[Part 2] Responses to Question (III) and (IV)

For Question (III): Extraction-set size and source

On Llama-3-8B, we conducted circuit functionality and ablation experiments using circuits extracted from 50 LLM-LAT prompts and 100 AdvBench prompts, and test ASR on held-out HarmBench respectively:

SettingBaseAblating C1C_1C1C_1
LLM-LAT-5048.4390.430.12
AdvBench-10048.4394.210.63

This results demonstrate C1C_1 is a refusal circuit and its ablation leads to safety degradation (the same in the main text), validating the robustness of our circuit extraction methods.

For Question (IV): Table 2 vs. Table 3 ASR, and model naming

Thank you for pointing this out. But the benchmark is stated clearly in the caption of Table3: Its 64.5% is ASR on the XSTest dataset, whereas Table 2's 50.31% is on HarmBench. We will stated more clearly with unsafe-contrast prompts.

Thank you again for your thoughtful feedback. We will incorporate these clarifications and additional results into the revised manuscript. We hope they help address your concerns and may warrant a more favorable assessment of our work.

Sincerely,

Authors

Thank you for the response. The added experiments on more model families and the clarified aggregation protocols are appreciated. However, the SafeSeek overlap is addressed only by a promised writing change, and the authors acknowledge the SCA claim remains partially validated. Rating unchanged

Reviewer_PZBQOpenReview ↗
Rating
3(+2)
Confidence
4
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper try to identify the safety-critical circuit in pretrained LLMs, and track their update during the later fine-tuning on specific dataset.

Critical strengths
  1. The paper is clearly written, and the research motivation is well articulated.
  2. The dual-branch mask optimization objective, which encourages refusal in the selected subnetwork and harmful compliance in its complement, is an interesting design choice.
Critical weaknesses
  1. The paper has a lot of overlap in the motivation, contribution, and conclusion with the paper "Superficial Safety Alignment Hypothesis", especially the tracing of safety-critical circuits.
  2. The technique is quite simple, which is often seen in work on soft/learnable pruning methods.
Questions for authors

The behavior of an extracted subnetwork does not necessarily reflect the mechanism underlying the intact model’s behavior. Although the model weights remain frozen, learning a mask still changes the network’s computation: disabling certain MLP outputs alters the inputs and activations of subsequent layers, as well as the final output distribution. Consequently, the optimization may identify a refusal strategy induced by selectively retaining and suppressing existing components.

The results therefore support the following claim:

  • There exists a sparse MLP mask under which the frozen model exhibits refusal behavior.

However, a stronger claim requires additional evidence:

  • The identified subnetwork constitutes the circuit that the intact model relies on to produce refusal behavior during normal operation.

Can the author resolve this concern?

3 comments
AuthorsOpenReview ↗

Responses to Weakness 1 & 2 and Quetsion 1

Dear Reviewer PZBQ,

Thank you very much for your thoughtful comments and constructive feedback on our submission. Below, we address each of your concerns in detail and hope our responses clarify the points you raised.

For Weakness 1: Relationship to SSAH in "Superficial Safety Alignment Hypothesis"

We agree that both works study safety-related components. However, the object of study and the findings differ: SafeEvo studies a sub-circuit that can independently express refusal, and how it changes during alignment.

Relative to SSAH [1], our new findings are:

  • Refusal circuits exist before safety alignment. SSAH's pretrained-to-aligned analysis categorizes pretrained units only as utility or redundant, "as the model has not yet undergone safety alignment." We find a refusal circuit already present in pretrained-only Llama-3-8B and Qwen-2.5-7B (Table 1 and Table 9).
  • Selected by function, and sufficient on its own. SSAH scores neurons by activation variance and validates them by pruning, which shows only that removing them hurts safety. We do not score activations; we optimize the mask directly for refusal behavior, so the extracted circuit expresses refusal by itself. This standalone function is what distinguishes a refusal circuit from a set of safety-important units.
  • How the circuit evolves. SSAH compares unit attribute labels between pretrained and aligned models. We re-extract the refusal circuit at successive alignment checkpoints and find that it is non-unique and drifts gradually (Section 3.3).
  • Which units to update. SSAH reduces alignment tax by fine-tuning redundant units in general alignment. SCA instead updates the refusal circuit itself during safety alignment for both SFT and DPO (Table 4).

Thank you again for pointing SSAH out and we will cite and discuss about SSAH in the future version.

For Weakness 2: Simplicity of the technique

We agree that our machinery (learnable binary masks with a straight-through estimator) is shared with learnable pruning, but the task and goal are different. Pruning removes parameters to deploy a smaller model that keeps overall capability; we never deploy the masked model, and use the mask only to attribute refusal, via an objective built for that goal: the circuit must refuse and its complement must comply.

The method is simple but effective. Its value lies in the mechanistic interpretability objective rather than in tool complexity, yielding instresting and useful insights for better and deeper LLM alignment. With the same SFT data, LoRA rank, and epochs, updating activation-selected safety neurons (SN-Tune) costs 4.6–18.8 GSM8K points and over-refuses 25.6–86.4% of benign prompts, whereas our SCA loses at most 1.4 points with 1.6–6.8% over-refusal (Table 2); equally sparse random supports yield higher ASR (Table 4).

For Question 1: Does the extracted circuit reflect the intact model's mechanism:

Our evidence goes beyond the existence of a refusal-inducing mask. Prior safety-head and safety-neuron studies [2, 3] all apply the interpretability logic that the components whose ablation leads to ASR increase in aligned LLMs (without loss of general abilities) are those related to LLM alignment mechanisms. We rigorously follow this logic.

Experimentally, we combine three tests, all on held-out datasets, so C1C_1's behavior is not specific to the extraction prompts:

  • Sufficiency. C1C_1 alone refuses all BeaverTails queries (Table 1). As the you notes, this alone could reflect an induced strategy, so we also test C1C_1 in the intact model.
  • Necessity in the intact model. We zero-ablate only C1C_1, keeping all other parameters intact. On HarmBench, refusal drops to at most 0.63% and ASR rises to 88–91% in all three models (Table 9). If the intact model refused through structures that bypass C1C_1, removing C1C_1 alone would leave substantial refusal.
  • Specificity. A count-matched random ablation of the same size leaves 22.6–42.1% refusal and 35–64% ASR (Table 9), so the effect is specific to the selected components rather than a generic consequence of perturbing the network.

Together, these results establish a causal link between C1C_1 and the intact model's refusal during normal operation, not merely the existence of a mask under which the frozen model refuses. We do not claim that C1C_1 is the unique or complete refusal mechanism, since independent extractions differ (Section 3.3); our claim is that the intact model's refusal depends on it.

We hope these responses address your concerns and would be grateful for a more favorable reconsideration of our work.

Sincerely,

Authors

Reference

[1] Li and Kim. Superficial Safety Alignment Hypothesis.

[2] Zhou et al. On the Role of Attention Heads in Large Language Model Safety. (ICLR 2025).

[3] Chen et al. Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons. (NeurIPS 2026).

Thanks for the rebuttal.

For weakness one, it resolves my concern. Please highlight "independently" in the paper when compared to SSAH. For weakness two, because you resolves the weakness one, it's totally okay. Regarding the question, I still have concerns, but it's okay.

I will increase the score to 3.

Dear Reviewer PZBQ,

Thank you again for your kind follow-up and for raising your score. We truly appreciate the time and care you have put into reviewing our work, and your thoughtful comments have helped us improve the paper. We have revised the manuscript based on your suggestions, and the changes are marked in orange.

Best regards,

The Authors

Reviewer_NTxvOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper proposes SafeEvo, an interpretability framework that investigates safety alignment from a circuit perspective. Specifically, the authors focus on the pretrained model and its evolution during post-training. The authors find that refusal circuits are unstable during post-training, which they believe could be the cause of the utility drop, and propose a novel alignment algorithm named SCA to reduce the utility drop in post-training by strictly restricting updates to a chosen circuit. Experiments demonstrate that SCA outperforms vanilla alignment in several aspects.

Critical strengths
  1. Insightful Mechanistic Perspective on the Alignment Tax: Investigating the causes of the alignment tax is an important and timely research problem. The paper provides an insightful interpretability perspective on why this utility degradation occurs by analyzing refusal circuits and their structural instability.
  2. Well-Written and Easy to Follow: The manuscript is clearly organized and very easy to follow, presenting a coherent narrative that flows logically from circuit discovery to evolution tracking and the resulting alignment design.
Critical weaknesses
  1. Timeliness of Evaluated Models: The evaluated LLMs are somewhat outdated. To better demonstrate generalizability and current relevance, the authors should validate the findings on more recent architectures, such as Qwen3.8.
Questions for authors
  1. The paper demonstrates that refusal circuits are non-unique. Given the existence of multiple redundant refusal circuits, is there any theoretical insight indicating that confining post-training updates to a single chosen circuit is sufficient to provide comprehensive safety coverage.
4 comments
AuthorsOpenReview ↗

Responses to Weakness 1 and Question 1

Dear Reviewer NTxv,

Thank you for finding the alignment-tax perspective insightful and the paper easy to follow and your dedicated efforts for reviewing. Below, we respond to each of the weaknesses and questions and hope that they can address your concerns.

For Weakness 1: Validation on a recent Qwen model

We have additionally validated our findings on a recent Qwen architecture, and the results remain consistent. Since Qwen3.8 is available only as post-trained checkpoints, whereas our analysis requires a pretrained-only model, we use Qwen3.5-9B-Base, which shares Qwen3.8's hybrid linear/full-attention architecture. We follow the same extraction and evaluation setups as those used for Qwen-2.5-7B in the main text. We denote the original Qwen3.5-9B-Base model as GG.

Table R1. Refusal circuit C1C_1 in Qwen3.5-9B-Base.

ConditionBeaverTails (Refusal Rate)HarmBench (Refusal Rate)HarmBenc (ASR)
Base model GG48.0083.6512.58
Circuit C1C_1 in isolation96.0099.371.26
C1C_1 ablation (G∖C1G\setminus C_1)2.000.0094.97
Random ablation46.0080.5016.35

Table R2. Safety alignment (SFT) on Qwen3.5-9B-Base.

MethodHarmBench (ASR)GSM8K (Accuracy)XSTest (Refusal Rate)
Starting checkpoint10.6985.674.40
LoRA0.0084.7657.20
SCA (Ours)0.6385.9022.40

The circuit alone refuses 96% of BeaverTails and 99.4% of HarmBench queries (Takeaway 1), and ablating it eliminates refusal on HarmBench (0.00% vs. 80.50% compared with random ablation) while raising ASR from 12.58% →\to 94.97% (Takeaway 2). SCA reaches safety comparable to LoRA-based alignment (0.63% vs. 0.00% ASR) with slightly higher GSM8K accuracy (22.4% vs. 57.2%) and less than half the over-refusal (Section 4.3). Our conclusions therefore generalize beyond the originally evaluated models to a recent hybrid architecture.

For Question 1: Why can one extracted circuit suffice, given non-uniqueness?:

A single circuit suffices because the intact model's refusal depends on it (casual ablation results like Table R1 support this), so updates confined to it reach the refusal computation itself. Non-uniqueness does not contradict this: it reflects multiple solutions to the extraction objective, and any solution can be a proxy for our later SCA alignment.

Non-unique masks, not independent pathways. All masks optimize the same objective on the same prompts, so the ~26% overlap (Section 3.3) shows that the objective admits many sparse solutions. If these were independent routes, the intact model would keep refusing after C1C_1 alone is removed. Instead, refusal on held-out HarmBench drops to at most 1/159 and ASR rises to 88–91%, far above count-matched random removal (Figure 3).

Why updating one circuit suffices. SCA restricts where weights change, not the forward pass (W′=W+MBAW' = W + MBA, Eq. 7). To first order, a step with learning rate γ\gamma changes the loss on any harmful prompt xx by ΔLx≈−γ⟨gxC,gC⟩\Delta\mathcal{L}_x \approx -\gamma\langle g_x^{C}, g^{C}\rangle, where gCg^{C} and gxCg_x^{C} are the training-batch and per-prompt gradients on the circuit parameters. Training on the circuit therefore also improves unseen prompts that route through it with aligned gradients. Held-out results confirm this. On HarmBench, which is used neither for extraction nor for SCA training, circuit updates yield lower ASR than equally sparse random updates in every model (Table 4; four seeds, paired prompt-level bootstrap):

ModelSCA ASRRandom-basesline ASRΔ (95% CI)
Llama-3-8B5.4820.2814.80 [11.48, 18.55]
Qwen-2.5-7B11.3219.818.49 [4.72, 12.42]
Mistral-7B-Instruct-v0.15.3512.587.23 [3.77, 10.85]

The coverage therefore comes from the circuit, not from sparsity.

Thank you again for the effort you put into reviewing and for your meaningful comments.

Best regards,

Authors

AuthorsOpenReview ↗

[Summary of Our Responses] Sincere Invitation to your Further Discussion and Evaluation

Dear Reviewer NTxv,

Thank you again for your thoughtful review. In brief, our response:

  • Recent models: We validated our findings on Qwen3.5-9B-Base, which shares Qwen3.8's architecture, and both the circuit findings and SCA's advantages hold.
  • Single circuit vs. non-uniqueness: The non-unique circuits are alternative solutions to the same extraction objective, not independent refusal pathways. Updating one circuit generalizes to held-out harmful prompts, beyond what equally sparse random updates achieve.

We would warmly welcome your further thoughts as the discussion and evaluation continue, and we are happy if our detailed responses address your concerns.

Thank you for your time and valuable feedback.

Best regards,

The Authors

Thank you for your detailed response. I believe the new experiments make a valuable addition to the paper. I've raised my soundness score and will keep my original positive overall rating.

Dear Reviewer NTxv,

Thank you very much for your thoughtful review and for reconsidering your soundness assessment. We greatly appreciate your recognition of the additional experiments and your valuable feedback, which helped us improve the paper.

Best regards,

Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.