acceptodds
Under review as a conference paper at ICLR 2027

INSEP: Refusal-Direction Inseparability as a Defense Against Abliteration

Abstract

Abliteration is a widely used white-box attack against the safety alignment of large language models (LLMs). It elicits harmful behavior by estimating a refusal direction in the model's activation space and projecting that direction out of the residual-stream write-out matrices. We propose INSEP, a defense that fine-tunes the model to reduce the separability of harmful and benign activations along the refusal direction. Abliteration estimates this direction as a difference in class means and projects it out, so reducing separability weakens the signal the attack relies on. An adversarial eigendecomposition selects the most separable prompt subsets, and a low-rank adapter reduces their separation along that direction; behavioral losses preserve the base model's refusal and the quality of its benign responses during training. We benchmark INSEP against six published defenses under five attacks, across eight LLMs from 1B to 14B parameters and several model families. Beyond response harm, we measure coherence, directness, over-refusal, and MMLU, to distinguish robust refusal from low harm produced by evasive or degraded responses. INSEP ranks first on the mean composite score across all eight base models while retaining utility, in the largest abliteration-defense evaluation reported to date.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
abliteration, refusal direction, AI safety, open-weight models, tamper resistance, adversarial training, linear separability, representation engineering, safety evaluation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_y3ydOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

INSEP fine-tunes low-rank adapters to reduce harmful–benign activation separation along the difference-in-means direction used by common abliteration attacks. An adversarial minibatch selector seeks highly separable prompt subsets, while behavioral losses preserve refusal, benign compliance, and refusal after affirmative prefills. The paper evaluates seven defenses against five attacks on eight base models. It reports harm alongside coherence, directness, over-refusal, and MMLU, and compares both average-attack and worst-attack composite scores.

Critical strengths
  1. The evaluation is broad and includes meaningful utility checks.

  2. The paper focus on an interesting motivation.

Critical weaknesses
  1. Prefill is 1/5 of the mean-aggregated harm that enters the composite, and INSEP has an explicit prefill training term

  2. The ablations do not clearly establish inseparability as the source of INSEP’s robustness, since behavioral losses alone preserve much of its performance and yield lower harm under standard ablation and SOM attacks.

Questions for authors
  1. On one representative model, does INSEP remain robust when the attacker searches post-instruction token positions and ranks candidate edits by validation-set harm under an explicit utility constraint? What exact refusal proxy and degradation penalty are used in the current layer selection?

  2. What is the composite restricted to the four weight-editing attacks? And what happens to ReFAT/ART/RR with a prefill-refusal term added?

I like the idea of this paper and I will raise my score if my concerns are addressed.

5 comments

W1 / Q2 (first half) — composite over only the four weight-editing attacks

tldr: prefill is an inexpensive attack in the wild, and dropping it does not change defense ranking

Thank you for the detailed review. We recomputed the composite over only the four weight-editing attacks, with the same methodology as Table 3.

The ranking does not change. INSEP is first on seven of eight base models and first on average, so the result does not depend on including prefill. We keep prefill in the main suite because it is the attack an adversary can run at inference time without any weight access, and a suite that excluded it would leave the most accessible attack untested. We will add the four-attack table to the appendix.

Composite excluding prefill, mean over defense ×\times attack seeds

MethodLllama-3.2 1BOLMo-2 1BGemma-2 2BMistral 7BQwen-2.5 7BOLMo-2 13BPhi-4 14BQwen-3 14BMean
INSEP.816.825.837.615.853.804.732.828.789
ART.606.585.668.621.596.428.513.302.540
RR.758.468.602.582.360.538.638.283.529
LAT.087.581.455.383.460.670.620.516.471
Booster.450.336.661.040.511.620.501.468.448
base.341.470.616.260.375.525.568.299.432
ReFAT.426.421.442.381.251.397.249.403.371

updated: table header for clarity

Q2 (second half) — a prefill-refusal term added to the baselines

tldr: we added prefill to defenses but INSEP still wins

We added the prefill term to the other methods to test whether it transfers, and it does. On a method that is vulnerable to prefill it cuts prefill harm substantially. For INSEP the point runs the other way. Even the prefill ablation of INSEP defends prefill better than the other methods do with the term added, and the full objective does better still. RR is already prefill-robust and gains nothing, and ART trains on harmful prefixes by construction. The prefill defense is therefore a general technique, and it is not where INSEP's advantage comes from. We will add this table to the appendix.

Harm, Llama-3.2 1B

methodprefill harm (without prefill defense)prefill harm (with prefill defense)
base (undefended).101—
RR.101.104
ReFAT.532 ± .012.199 ± .007
ART (harmful-prefix training by construction)—.286
INSEP.069 (λ_p=0).022

updated: column headings for clarity

W2 — ablations and the source of robustness

tldr: (see 9xXB W3 response) a larger model (Qwen2.5-7B) correctly shows the ablation effect

This is the observation in 9xXB W3, and our full answer with the Qwen2.5-7B ablation table is there. In brief, the 1B column ranking does not carry to a larger, more capable model. On Qwen2.5-7B, removing the separability term raises harm on every weight-editing attack, and the full objective is the strongest configuration, so robustness is not carried by the behavioral terms alone. All four lambda weight terms contribute, and separability contributes the most once it is removed from both the loss and the adversarial search. We will add this to Appendix G.1.

Reviewer_qzEwOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies a concrete open-weight defense: whether a model can be trained so that harmful and benign activations are harder to separate along the difference-in-means refusal direction used by abliteration-style weight editing, while keeping ordinary refusal and benign behavior. INSEP re-estimates that direction on each training minibatch, reduces separability on it, and adds terms aimed at preserving normal behavior. After training, rank-AUC on the estimated axis is near chance, the mean gap shrinks substantially, and the corresponding attacks fail. The authors also compare several attacks and base models. I read this as a well-scoped method that works against the attack family it targets.

Critical strengths
  1. The problem is well chosen and the method matches it. The paper studies one real way of stripping refusal from an open-weight model, and INSEP attacks the statistic that method uses, rather than making a general safety claim. That alignment is the main reason I think the submission is solid.
  2. The validation is not only end-to-end attack success. Near-chance rank-AUC and the large drop in mean difference show that the intended axis is disrupted, and the evaluated abliteration-style attacks fail. Together with the broader attack-and-model comparison, this is enough for me to accept the empirical claim.
Critical weaknesses
  1. The strongest account of why INSEP works is not directly measured. The results show that this axis becomes less useful and that these attacks fail. That is not the same as showing that the attacker’s estimator became unstable or noisy. This matters for how the mechanism is described, not for whether the defense works in the reported setting.
  2. The reliable result is narrower than some of the framing. Other linear directions remain highly separable (whitened / LDA) and did not yield a matching attack, so what is shown is that this style of difference-in-means abliteration gets harder, not that linear refusal structure in general has been removed. I also did not see a direct test that robustness is insensitive to the attacker’s contrastive prompts.
Questions for authors
  1. After INSEP, do you already report anything about the attacker’s estimator itself—for example variance across seeds, or stability of the estimated direction—or only separability on the re-estimated axis? If the former is in the paper, please point to it. That would affect how I read the “unstable / noisy estimator” wording.
  2. You report that other linear directions remain highly separable. Did you already run the same abliteration procedure along those axes, or is the point that the attacker you study does not use them? If they were attacked and failed, I will treat the contribution as more cleanly specific to difference-in-means abliteration.
  3. Is the claim that robustness does not depend strongly on the attacker’s contrastive prompts already tested on held-out or substantially different prompt sets? If there is a table or appendix, please point to it. If not, I will treat that sentence as not yet supported.
4 comments

Q1 — directional agreement of the attack estimator

tldr: INSEP degrades both the magnitude and direction of the attack estimator

Thank you. We will add this measurement to the appendix.

An abliteration attack depends on estimating one direction and its magnitude, and INSEP weakens both. The magnitude of the estimated direction drops several-fold, and independent estimates of the direction agree with each other far less than on the base model. The effect is largest in the mid and late layers, which is where abliteration is most effective. An attacker that cannot reproduce its own estimate across contrast sets cannot reliably choose a layer to edit. The evaluation uses disjoint 300-prompt contrast sets (Llama-3.2-1B, base and three INSEP seeds).

Attack estimate direction and magnitude, Llama-3.2 1B

bandcos (base)cos (INSEP)‖d‖ (base)‖d‖ (INSEP)reduction fraction
early (L1–5).99±.00.82±.160.720.16.30±.15 (3×)
mid (L6–10).99±.00.78±.212.370.22.10±.03 (10×)
late (L11–15).99±.00.82±.195.440.57.09±.06 (11×)

Q2 — attacking other axes

tldr: (see 9xXB response) the defended axes are separable but not attackable

We attacked the whitened direction directly, matched on layer, strength, and contrast set. It is a successful attack on the base model and fails against INSEP, even though that direction remains separable (Table 10, Appendix G.3). Separability along an axis is necessary but not sufficient for that axis to be a removal target.

Q3 — contrastive prompt sets

tldr: INSEP defends against disjoint attack prompts, and prompts from an unseen corpus

Thank you. Two tests in the paper speak to this.

The first is by construction. Each attack builds its contrast prompts from evaluation-reserved rows of public harm benchmarks that the defense never sees in training, and an automated check enforces the disjointness (Section 4.1, Appendices D and E).

The second varies the estimation corpus entirely. An attacker that estimates its direction from CatQA, a dataset the defense was not trained against, reaches 0.033 harm against INSEP while reaching 0.489 on the undefended model (Appendix G.3). We will state the scope in the introduction. Robustness holds across the contrast sets we test, including a disjoint corpus, as long as the training pools are diverse enough to cover the attacker's distribution.

Thank you for the detailed responses and for incorporating the additional analyses into the revision. The responses substantively address my questions and strengthen my support for acceptance.

First, the measurements across disjoint contrast sets provide direct evidence that the estimated difference-in-means direction becomes less consistent after INSEP, alongside the reduction in its magnitude. This goes beyond the separability measurements I discussed in my initial review. I would still distinguish evidence of reduced estimator consistency from a complete causal explanation of attack failure, but the additional analysis supports a more precise mechanistic account.

Second, the matched attack along the whitened direction is particularly useful. Its success on the base model and failure on the defended model, despite retained separability, support the important distinction between a direction that carries class information and one whose removal effectively bypasses refusal.

Third, the CatQA experiment provides useful evidence beyond train–evaluation prompt disjointness. It supports robustness across the tested contrast sets, including an estimation corpus not used in defense training, without requiring a broader claim of prompt-distribution independence.

Overall, I continue to view INSEP as a well-motivated defense whose empirical evaluation matches its intended threat model. The rebuttal strengthens both the interpretation of the results and the evidence for robustness within that scope. I maintain my weak-accept rating and encourage the authors to keep the mechanistic and generalization claims bounded by the reported experiments.

Reviewer_9xXBOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes INSEP, which reduces the separability of harmful and benign activations along the difference-in-means direction to defend against abliteration attacks, while using behavioral losses to preserve refusal and normal responses.

Critical strengths
  1. The defense idea is clear and directly targets the statistical signal that attacks rely on to estimate the refusal direction.

  2. The experiments cover multiple models and attacks, and evaluate safety, response quality, and capability retention.

Critical weaknesses
  1. The paper’s positioning needs further clarification. The paper describes methods such as ReFAT [1] as training against a fixed refusal direction. However, ReFAT resamples prompts and dynamically estimates the direction during training. Therefore, using the claim that “existing methods rely on a fixed direction” to motivate this work is not entirely accurate, and the distinction between the methods needs further clarification.

  2. The refusal mechanism after training is not sufficiently explained. Reducing separability along the difference-in-means direction does not mean that the two activation distributions are difficult to distinguish overall. For example, they may still be distinguished using other directions or simple information such as activation norms. The paper does not sufficiently explain what information the LLM uses after training to distinguish harmful from benign requests, or why this retained information is harder for attackers to exploit.

  3. The ablation results for the core loss need further explanation. Table 7 shows that removing the separability loss actually reduces harm under standard abliteration and SOM attacks, while this loss mainly improves performance against ACE and Heretic. The reasons for this difference have not been sufficiently explained, so the relationship between the core loss and the defense gains remains unclear.

[1] Lei Yu, Virginie Do, Karen Hambardzumyan, and Nicola Cancedda. Robust LLM safeguarding via refusal feature adversarial training. International Conference on Learning Representations (ICLR), 2025.

Questions for authors
  1. Given that ReFAT already updates refusal directions dynamically, could the authors revise the relevant description and explain more precisely how the proposed method improves upon this type of dynamic adversarial training?

  2. After training, what information does the LLM use to maintain refusal of harmful requests and normal responses to benign requests? If this information remains encoded in other directions or simple structures separable by a spherical boundary, could the authors further test whether attackers can exploit it?

  3. Why does the loss that directly targets the difference-in-means direction fail to improve defense against standard difference-in-means ablation in Table 7? Does this suggest that some of the robustness mainly comes from behavioral training? Does this observation also hold for other models?

[1] Lei Yu, Virginie Do, Karen Hambardzumyan, and Nicola Cancedda. Robust LLM safeguarding via refusal feature adversarial training. International Conference on Learning Representations (ICLR), 2025.

5 comments

W1/Q1 — ReFAT and the "fixed direction" wording

tldr: our Section 2 wording is wrong and we will correct it

Thank you for the detailed review. You are correct that ReFAT re-estimates the refusal direction during training, and our re-implementation does the same (Appendix K). The claim in Section 2 that prior methods treat the direction as fixed is wrong as written, and we will replace it.

The distinction we should have drawn is about what remains after training. ReFAT, ART, and DeepRefusal train refusal behavior to survive removal of an estimated direction, but they leave the harmful and benign representations separable. An attacker who re-estimates the direction at evaluation still recovers a usable one. INSEP reduces that separability, so there is no well-estimated direction left to remove. Both families re-estimate during training, yet they behave very differently against the attack, and Table 3 shows the gap.

W2/Q2 — other directions, norms, the spherical boundary

tldr: classes stay separable after INSEP

Thank you. You are correct, and this question led to analysis we should have included in the submission. Reducing separability along the difference-in-means direction does not make the two activation distributions hard to distinguish in general.

After INSEP the classes remain separable by a linear probe and by a kernel probe. Two readouts fall to chance. One is the difference-in-means axis, which is the statistic an abliteration attack estimates. The other is the activation norm, which is already at chance on the undefended model. INSEP removes the attack's readout, not the underlying information.

AUC, N=3 INSEP seeds, Llama-3.2 1B

diff-in-meansbest linearRBF (Euclid, best)RBF (whitened)norm
base1.0001.0001.0001.000.506
INSEP s0.594.999.9651.000.517
INSEP s1.556.997.949.997.541
INSEP s2.5331.000.9631.000.508

The reason this still yields robustness is that separability does not imply removability. A direction can stay perfectly separable and still be useless as a removal target. We show this directly. We attacked the defended whitened direction with matched rank-1 abliteration, the same attack that succeeds on the base model, and it fails against INSEP even though that direction is still fully separable (Table 10). We will surface this analysis in the main text.

W3/Q3 — Table 7: removing the separability loss appears to lower harm

tldr: a larger model (Qwen2.5-7B) correctly shows the ablation effect

Thank you. You are right about the 1B table as printed, and investigating this really improved our understanding of the method.

The drop between Full and any ablation does not hold on a larger, more capable model. We repeated the ablation on Qwen2.5-7B under the same protocol and metric. There, removing the separability term raises harm on every weight-editing attack, and the full objective is the strongest configuration in the table. The 1B model has sixteen layers, too few for the per-attack improvements to clear seed noise, so an individual column can flip (but the 1B composite ranking is still correct). The 7B model has twenty-eight layers, and the effect is stable.

Ablation result, Qwen2.5 7B

ObjectiveCompositeACEablSOMHereticprefill1−overrefcoherdirectMMLU
Full.878±.010.053.052.076.076.022.9331.000.995.782
λ_d=4, random search.856±.013.061.110.082.066.027.934.993.986.770
λ_r=0.798±.042.065.263.113.156.008.937.991.985.759
λ_p=0.774±.053.141.270.129.181.062.941.996.989.765
λ_d=0.705±.010.037.381.391.439.010.952.996.992.775
λ_d=0, random search.692±.020.042.430.448.381.009.946.998.994.775
λ_c=0.486±.232.035.030.015.063.010.708.587.577.681
λ_c=λ_r=0.469±.195.036.017.042.111.014.695.611.588.630

The ablation ordering shows dropping the refusal term affects the composite the least, while dropping separability or compliance affects the model the most.

Your question also showed that our description of the separability ablation was incomplete. Separability enters INSEP through two channels: (1) the loss term, and (2) the adversarial selection of the minibatch and layer to edit. Setting the separability weight to zero removes it from the loss but leaves selection targeting the most separable layer. A fully random search removes separability from both, and that fully separability-free ablation has the smallest composite of any non-degenerate ablation (.692).

The two lowest rows reach low harm only by degrading the model into over-refusal or incoherence. The composite is designed for this case. A harm-only score would rank the compliance ablation above Full.

MMLU alone does not capture this degradation. The MMLU drop on the compliance ablation is modest (.681), so the model still answers constrained multiple-choice questions but is neither direct nor coherent in open generation. A composite that used MMLU alone would rank the compliance ablation fourth of eight, though that model is degenerate. We will include this analysis in the appendix.

updated: table heading for clarity

Comments on the paper

1 comment

Thank you for the time you spent reviewing our paper. We had missed the instruction to resubmit the draft with the changes incorporated, and have now done so. The suggestions were unusually constructive. They strengthened the work and produced some qualitative results we find genuinely interesting. We hope you agree, and thank you again for your efforts.

Changes are highlighted in blue.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.