- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
INSEP fine-tunes low-rank adapters to reduce harmful–benign activation separation along the difference-in-means direction used by common abliteration attacks. An adversarial minibatch selector seeks highly separable prompt subsets, while behavioral losses preserve refusal, benign compliance, and refusal after affirmative prefills. The paper evaluates seven defenses against five attacks on eight base models. It reports harm alongside coherence, directness, over-refusal, and MMLU, and compares both average-attack and worst-attack composite scores.
Critical strengths
-
The evaluation is broad and includes meaningful utility checks.
-
The paper focus on an interesting motivation.
Critical weaknesses
-
Prefill is 1/5 of the mean-aggregated harm that enters the composite, and INSEP has an explicit prefill training term
-
The ablations do not clearly establish inseparability as the source of INSEP’s robustness, since behavioral losses alone preserve much of its performance and yield lower harm under standard ablation and SOM attacks.
Questions for authors
-
On one representative model, does INSEP remain robust when the attacker searches post-instruction token positions and ranks candidate edits by validation-set harm under an explicit utility constraint? What exact refusal proxy and degradation penalty are used in the current layer selection?
-
What is the composite restricted to the four weight-editing attacks? And what happens to ReFAT/ART/RR with a prefill-refusal term added?
I like the idea of this paper and I will raise my score if my concerns are addressed.
5 comments
Q2 (second half) — a prefill-refusal term added to the baselines
tldr: we added prefill to defenses but INSEP still wins
We added the prefill term to the other methods to test whether it transfers, and it does. On a method that is vulnerable to prefill it cuts prefill harm substantially. For INSEP the point runs the other way. Even the prefill ablation of INSEP defends prefill better than the other methods do with the term added, and the full objective does better still. RR is already prefill-robust and gains nothing, and ART trains on harmful prefixes by construction. The prefill defense is therefore a general technique, and it is not where INSEP's advantage comes from. We will add this table to the appendix.
Harm, Llama-3.2 1B
| method | prefill harm (without prefill defense) | prefill harm (with prefill defense) |
|---|---|---|
| base (undefended) | .101 | — |
| RR | .101 | .104 |
| ReFAT | .532 ± .012 | .199 ± .007 |
| ART (harmful-prefix training by construction) | — | .286 |
| INSEP | .069 (λ_p=0) | .022 |
updated: column headings for clarity
W2 — ablations and the source of robustness
tldr: (see 9xXB W3 response) a larger model (Qwen2.5-7B) correctly shows the ablation effect
This is the observation in 9xXB W3, and our full answer with the Qwen2.5-7B ablation table is there. In brief, the 1B column ranking does not carry to a larger, more capable model. On Qwen2.5-7B, removing the separability term raises harm on every weight-editing attack, and the full objective is the strongest configuration, so robustness is not carried by the behavioral terms alone. All four lambda weight terms contribute, and separability contributes the most once it is removed from both the loss and the adversarial search. We will add this to Appendix G.1.
W1 / Q2 (first half) — composite over only the four weight-editing attacks
tldr: prefill is an inexpensive attack in the wild, and dropping it does not change defense ranking
Thank you for the detailed review. We recomputed the composite over only the four weight-editing attacks, with the same methodology as Table 3.
The ranking does not change. INSEP is first on seven of eight base models and first on average, so the result does not depend on including prefill. We keep prefill in the main suite because it is the attack an adversary can run at inference time without any weight access, and a suite that excluded it would leave the most accessible attack untested. We will add the four-attack table to the appendix.
Composite excluding prefill, mean over defense × attack seeds
updated: table header for clarity