acceptodds
Under review as a conference paper at ICLR 2027

Refusal Localizes, the Damage Relocates: Safety Layers Under Few-Sample Fine-Tuning

Abstract

Fine-tuning adapts aligned large language models (LLMs) to downstream tasks, but a few dozen harmful examples can remove their refusal of harmful requests. Prior work localizes safety-related behavior to specific layers, directions, and tokens, suggesting targets for protection. We test whether successful localization and recovery support defenses that survive changes in the attack. Across six checkpoints from four model families, harmful and benign prompts remain linearly separable after attack, and patching full clean hidden states into the compromised model restores refusal at a reproducible transition depth. Building on a prior layer-freezing defense, we freeze every layer up to this depth and repeat the attack. At a hundred harmful examples, refusal remains near zero on all six checkpoints, with recovery transitions above the frozen boundary. In a second study, removing the update's top two singular directions restores refusal after short attention-only fine-tunes on four checkpoints. On Llama-3.1-8B, ordinary training changes weaken this repair and an attacker who spreads the update defeats it. A spectral detector calibrated on benign Llama fine-tunes misses most repair failures on that checkpoint. Localized freezing can nevertheless help preserve refusal when a few harmful examples enter training data unintentionally. These results show that an attacker can bypass a region identified by recovery and defeat a repair that works across multiple checkpoints, motivating five checks for defenses against adaptive fine-tuning.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Model Safety, Alignment Robustness, Harmful Fine-Tuning, Refusal Mechanisms, Activation Patching, Mechanistic Interpretability

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_ugUROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates how large language models, despite safety alignment, can still suffer safety degradation after fine-tuning with a small number of samples. It focuses on how such safety degradation manifests internally, analyzing whether refusal behavior concentrates in specific representation directions or network locations and how safety-related changes are distributed after fine-tuning. From the perspective of internal model mechanisms, it explores the fragility of safety alignment and provides experimental observations and analytical insights for understanding why few-shot fine-tuning may weaken LLM safety capabilities. However, the explanation of the safety damage relocation mechanism remains insufficient, and some conclusions rely mainly on empirical observations, lacking more direct causal verification.

Critical strengths

1.The research question is important and reveals that safety behavior localization and defense robustness are not necessarily correlated. 2.The layer-freezing experiments cover multiple models with relatively sufficient controls, and the finding regarding changes in the recovery transition position is of research value. 3.The authors distinguish between explicit refusal rates and actual unsafe output rates, and analyze different scenarios of defense effectiveness and failure.

Critical weaknesses

1.The logical connection among research motivation, experimental design, and core conclusions is not sufficiently natural, and writing clarity needs improvement. 2.The paper's novelty lies mainly in analyzing the failure of existing defense methods, and some core conclusions are limited by evaluation metrics and experimental coverage, making the novelty moderate. 3.Does a refusal rate near zero truly prove complete failure of safety defense? An explicit refusal rate near zero does not equate to a complete loss of safety capability. Some models still produce a large number of responses judged safe by safety evaluators after freezing, so the interpretation of defense failure requires more caution. 4.Insufficient evidence for the relocation mechanism. The change in the depth of the recovery transition layer is a clear experimental observation, but it is not sufficient to fully prove that safety-related damage undergoes mechanism-level relocation within the network. 5.Limited generality of adaptive evaluations. All attacks use the same type of supervised fine-tuning objective, and the most in-depth spectral repair adaptive evaluation is mainly limited to one model, weakening the generality of the conclusions.

Questions for authors

First, further explain the difference between refusal rate and safety. It would be better to provide more direct judgments of output harmfulness, especially explaining why some models have explicit refusal rates near zero yet still produce a considerable proportion of safe responses. Second, strengthen the mechanistic evidence for "Damage Relocates." At minimum, the shift in recovery transition position should be more clearly distinguished from a genuine change in damage location. If stronger evidence cannot be provided, it is recommended to reduce the strength of mechanistic claims. Third, add representative independent attack settings. In particular, extend the most critical weight-space adaptive evaluation to other models to test whether the findings are stable across models.

14 comments
AuthorsOpenReview ↗

Response to Reviewer ugUR (1/12)

Response overview and the paper's argument

Under both harm scorers we added, and under a second harmful corpus, the frozen attack stays above the clean model on every checkpoint and seed, and with the second corpus Llama-Guard rates it within 0.04 of the unrestricted attack trained on the same seed (Table R1 and Table R2). A direct intervention on the weight update locates the damage above the frozen boundary, where switching the update off restores refusal to 0.70 to 0.97 while keeping it only there leaves 0.00 to 0.35 (Table R5). We thank the reviewer for a careful review and for recognizing the controls and the separation of refusal from unsafe output.

Before writing this response, we ran the new experiments the reviews ask for and restructured the paper around their results, which took much of the discussion period. We realize that little time remains, and we respectfully ask the reviewer to read our response and the new measurements.

1.The logical connection among research motivation, experimental design, and core conclusions is not sufficiently natural, and writing clarity needs improvement.

Answer. The revision follows one question through localization, defense tests and the resulting checks, making their connection explicit in the main text.

The question is whether a place where refusal can be found is also a place where it can be defended (Section 1, see Lines 065-071). Section 4 finds the place, with a frozen probe and with clean-state patching that locates a transition depth. Section 5 defends it by freezing every layer up to that depth and attacking again with an attacker who knows the frozen layers. Section 6 asks the same question in weight space, where top-two removal repairs one training regime and an adaptive attacker defeats it. Section 7 turns each failure into a check that a localization must pass before it supports a defense. That chain is the connection the review found missing. The revision also rewords the abstract and Sections 1 to 7 for clarity, replaces the submission's nonstandard terms with standard ones defined at first use, and states the conclusion in the main text as Section 8. We would be grateful if the reviewer would read the chain in that form.

Localization, defense evaluation and the resulting checks form the argument stated in the revised main text.

The revised main text follows one question through each test, so the argument can be followed without the appendix.

Revised-PDF location. The revision states the chain in a new closing paragraph of Section 1, opens Sections 6 and 7 and the subsections of Sections 4 to 6 with what each tests.

AuthorsOpenReview ↗

Response to Reviewer ugUR (2/12)

W2. The missing adaptive comparison

2.The paper's novelty lies mainly in analyzing the failure of existing defense methods, and some core conclusions are limited by evaluation metrics and experimental coverage, making the novelty moderate.

Answer. The new matched comparison tests whether SPPFT's published localization adds protection under malicious training and finds no separable refusal benefit.

What the two defenses assume, and how SPPFT was evaluated. Freezing the safety layers (SPPFT, Li et al., 2025) and projecting the update (Safe LoRA, Hsu et al., 2024) were proposed on the premise that safety lives in a findable place. In a mixed setting of a thousand ordinary examples with malicious examples added, SPPFT's own Table 3 reports harmful rates of 68.3 and 78.8 percent at a malicious fraction of 0.2 against initial rates of 1.35 and 0.77 in its Table 2, so the paper reads the defense as mitigation.

That reading rests on one comparison the paper does not run, the published layers against a freeze of as many layers under malicious data. Its stated threat model gives the attacker "Limited Knowledge" of "the model's internal structure, parameters, and the location of safety layers" (Li et al., 2025, Appendix A.4.1), so an attacker who knows which layers are frozen falls outside that knowledge assumption. Our attack does not need knowledge of which layers are frozen. The three conditions share the same PKU-SafeRLHF pairs at a hundred examples, the same seeds, the same response-masked loss and the same three epochs at batch four, and differ only in which layers the provider freezes. The paper notes at Lines 297-298 that SPPFT was evaluated against backdoor and ordinary instruction data rather than against such an attacker. That is the attacker the lower block of Table 1 supplies, beside a freeze of as many layers.

At the layers Li et al. (2025) published as the ones to freeze, refusal after the attack is not separable from a freeze of as many layers on all four checkpoints they report bands for (Table 1, lower block, see Lines 292-296).

What this paper adds. Wei et al. (2024) show that restricting modifications to the sparse safety-critical regions they identify does not stop fine-tuning attacks, and Qi et al. (2025b) that small changes to the fine-tuning configuration defeat safeguards built into alignment. Li et al. (2025) ran that neuron freeze themselves at a matched proportion and report it "proved ineffective in preventing security degradation during fine-tuning, consistent with their original findings". They conclude that SPPFT "proves to be the most effective parameter-freezing defense" because it holds whole layers rather than discrete neurons. After Wei et al. (2024), then, the open question was precisely whether a whole-layer defense survives an attacker who knows it, and that is the question answered here.

What this response adds on metrics and coverage. On metrics and coverage, this response adds two further harm scorers (Table R1), a second harmful corpus (Table R2), two more harmful benchmarks (Table R3), a second training objective in DPO with the Llama 3 likelihood term (Table R9), and the weight-space adaptive evaluation on two more checkpoints (Table R6).

AuthorsOpenReview ↗

Response to Reviewer ugUR (3/12)

W2. Two further harmful benchmarks, continued.

some core conclusions are limited by evaluation metrics and experimental coverage

To test whether the freeze failure depends on which harmful benchmark scores it, we evaluated every frozen adapter of Table 1 on two further benchmarks, a hundred HarmBench prompts and sixty StrongREJECT prompts.

Table R3. The frozen adapters of Table 1 on two further harmful benchmarks, a hundred HarmBench prompts and sixty StrongREJECT prompts, at a dose of a hundred harmful examples, ranges over three training seeds. Refusal is coherent refusal, higher is better, and LG unsafe is the Llama-Guard-3-8B unsafe rate, lower is better. Frozen is the band the freeze protects, taken from Table 1.

BenchmarkModelFrozenRefusal cleanAttackFreezeLG unsafe cleanAttackFreeze
HarmBenchLlama-3.1-8B[0, 17].84.00-.01.00.14.88-.90.69-.84
HarmBenchTulu-3-8B-DPO[0, 15].93.00-.01.00-.05.05.83-.89.68-.79
HarmBenchOLMo-2-7B[0, 16].92.01-.02.00-.01.03.85-.92.65-.68
HarmBenchOLMo-2-13B[0, 20].86.00.00-.02.06.88-.92.50-.70
HarmBenchQwen2.5-14B[0, 31].70.00-.03.00-.01.10.90-.92.56-.66
HarmBenchYi-1.5-9B[0, 39].66.00-.01.02.30.90-.94.62-.81
StrongREJECTLlama-3.1-8B[0, 17].97.00.00.00.88-.93.83-.85
StrongREJECTTulu-3-8B-DPO[0, 15].92.00.00-.03.00.88-.93.80-.82
StrongREJECTOLMo-2-7B[0, 16].93.00.00.00.90-.93.77-.90
StrongREJECTOLMo-2-13B[0, 20].92.00-.02.00.00.92-.95.67-.78
StrongREJECTQwen2.5-14B[0, 31].70.00-.03.00.00.92-.93.67-.75
StrongREJECTYi-1.5-9B[0, 39].77.00-.02.00-.02.23.85-.92.63-.68

On both benchmarks the freeze leaves coherent refusal at or below .05 while Llama-Guard rates .50 to .90 of the frozen attack's completions unsafe, so the failure does not depend on the benchmark that scores it.


W2. Contribution and coverage, continued.

some core conclusions are limited by evaluation metrics and experimental coverage

To test whether the added lineages support the failure claim, we repeated the full procedure on the additional checkpoints.

Table R8. Three further checkpoints through the full procedure. AdvBench, a hundred harmful examples, ranges over three training seeds. Depth is the transition layer by seed, Frozen is the band given by the three-seed mean, and Matched freezes as many layers at the top. The last two rows repeat the two checkpoints whose deepest seed lies above the mean band with the freeze one layer deeper, so the frozen band covers the measured depth of every seed.

ModelDepthFrozenRefusal cleanAttackFreezeMatchedLG unsafe cleanAttackFreezeMatched
gemma-2-9b-it (42 layers)24, 24, 24[0, 24].99-1.00.00.00.00-.06.00.95-.97.94-.97.87-.96
Ministral-8B-Instruct-2410 (36)20, 20, 17[0, 19].81-.85.00-.01.00-.05.01-.02.16-.18.93-.97.57-.86.89-.95
Mistral-Nemo-Instruct-2407 (40)19, 19, 15[0, 18].68-.76.00-.01.00-.01.00-.01.22-.32.95-.98.88-.92.96-.98
Ministral-8B-Instruct-2410 (36), one deeper20, 20, 17[0, 20].81-.85.00-.01.00-.01.00-.02.16-.18.93-.97.76-.88.94-.95
Mistral-Nemo-Instruct-2407 (40), one deeper19, 19, 15[0, 19].68-.76.00-.01.00-.01.01-.03.22-.32.94-.98.83-.93.90-.98

All added checkpoints lose refusal under the freeze, with no separation from the equal-sized control.

It also adds three checkpoints from the Gemma and Mistral lineages, gemma-2-9b-it, Ministral-8B-Instruct-2410 and Mistral-Nemo-Instruct-2407, through the full procedure, with the transition depth measured on each seed and the freeze applied from layer 0 to the three-seed mean depth. All nine attacked seeds pass the validity checks of the depth measurement, and refusal after the frozen attack is 0.00 to 0.05 against clean rates of 0.68 to 1.00. That range includes Mistral-Nemo, which refuses only 0.68 to 0.76 when clean. On the two checkpoints whose deepest seed lies above the mean band, freezing one layer deeper so that the band covers every seed leaves refusal at 0.00 to 0.01. Appendix C of the revision reports that table.

Reviewer_kCtAOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper tests whether localizing refusal in specific layers, directions, or token positions actually helps defend against harmful LoRA fine-tuning. Six checkpoints across four model families are probed to find a transition depth (Section 4), then two localization-based defenses are tested against an attacker who knows the defense. Freezing fails on all six checkpoints under the adaptive attack, with damage relocating to just above the frozen boundary (Figure 5, Table 10). Truncation repairs refusal under one attack configuration but is defeated by ordinary training changes and a spread adaptive attack on Llama-3.1-8B.

Critical strengths

The source-mismatch and no-op self-patch controls are useful.

The base-vs-aligned probe decomposition is carefully designed and helps separate pretrained representations from alignment effects.

Table 1 and the matched-boundary controls in Figure 5 and Table 10 provide convincing evidence that freezing the localized refusal region can be bypassed by an adaptive attacker.

Critical weaknesses

The only working instance of the weight-space defense is not in-scope. Section 6.1 says outright that its positive repair result ($0.76$-$1.00$, four checkpoints) was produced under one attack configuration... and neither choice is part of our threat model. Widening to 7 projections drops relative repair from 0.99 to 0.74; 8 epochs drops it to 0.35, the adaptive attack drops it to 0.000 (Table 3). So the one case where the defense looks like it works is a case the paper itself says does not match its threat model, which is contradictory.

The six checkpoints, four model families framing overstates independence, and the weakest-refusing model was pre-screened out.

Two of the six are the same OLMo-2 lineage at different sizes (7B/13B), and a third (Tulu-3-8B-DPO) is a DPO fine-tune of the same Llama-3.1-8B that is already checkpoint #1 (Section 3.1). Mistral-7B-Instruct-v0.3 was excluded throughout because it refuses only 0.16 to 0.18 of AdvBench when clean (Section 3.2, Appendix A). With the effective sample closer to four independent lineages than six, and the weakest-refusing model dropped before any defense was tested, how confident can the authors be that freezing fails everywhere is not partly a selection effect of only testing models that already localize refusal cleanly?

Freezing never gets the hyperparameter sweep that is shown to defeat the other defense.

Questions for authors

Would the detector's near-total failure and the spread attack's success replicate on OLMo-2-13B or Qwen2.5-14B, where Table 2 says refusal is harder to strip?

Appendix B says 7% to 30% of freeze-arm completions are pattern-mismatched but Llama-Guard-unsafe, how much would the dose thresholds in Figure 4/Table 7 move if Llama-Guard were the primary metric?

Why present the top-2 truncation repair as an abstract-level finding when it is immediately shown to fail in-threat-model?

11 comments
AuthorsOpenReview ↗

Response to Reviewer kCtA (1/9)

Response overview

The freeze fails on seven lineages, six of them at a measured depth. On both added checkpoints the spread attack defeats top-two removal, leaving refusal at 0.00 on Qwen2.5-14B and 0.08 to 0.22 on OLMo-2-13B against clean 1.00. We thank the reviewer for the detailed reading and for finding the controls and the matched-boundary evidence convincing.

Before writing this response, we ran the new experiments the reviews ask for and restructured the paper around their results, which took much of the discussion period. We realize that little time remains, and we respectfully ask the reviewer to read our response and the new measurements.

Answer. The working repair and its defeat lie within one threat model, and the added experiments resolve the selection, training-sweep, replication and harm-metric questions.

The response compares the repair regime, additional checkpoints, training sweeps and Llama-Guard dose readings.

W1, Q3. Repair within the threat model. The top-two truncation evaluation tests repair and its adaptive defeat within the same threat model, without excluding the working default configuration. Their contrast tests robustness within one threat model (Section 6.1 and Table R6).

W2. Lineage independence and selection. The freezing evaluation selects checkpoints by clean refusal and counts pretrained lineages, without requiring cleanly localized refusal. A weak refuser from the excluded family also fails the full procedure, so failure persists without selection for clean localization (Table R8, Appendix C).

W3. Freezing's training sweep. Ordinary training changes erase the freeze's low-dose protection on the tested checkpoints. The protected regime is configuration dependent (Table R7 and Table R23).

Q1. Spread attack and detector replication. The spread attack defeats repair on OLMo-2-13B and Qwen2.5-14B, and benign-calibrated detection misses ordinary repair failures. Both findings extend beyond the original model (Table R6).

Q2. Llama-Guard dose thresholds. The quoted mismatch share counts safe-rated non-refusals, without counting Llama-Guard-unsafe outputs. Llama's breaking dose stays fixed, while other harm readings depend on checkpoint and corpus. Harm reduction is distinct from refusal preservation (Table R4 and Table R15).

Mistral-Nemo, which refuses only 0.68 to 0.76 of AdvBench when clean, goes through the full procedure with the freeze leaving 0.00 to 0.01. Failure extends beyond the original lineage and training configuration.

The failure is not confined to one lineage or training setting, and the repair of Section 6 fails on the added checkpoints as well.

AuthorsOpenReview ↗

Response to Reviewer kCtA (2/9)

W1, Q3. Truncation and the threat model

The only working instance of the weight-space defense is not in-scope.

So the one case where the defense looks like it works is a case the paper itself says does not match its threat model, which is contradictory.

Why present the top-2 truncation repair as an abstract-level finding when it is immediately shown to fail in-threat-model?

Answer. The top-two truncation evaluation tests whether a repair that works under the default attack survives attacker-chosen training changes. It does not exclude the working default configuration from the threat model.

We distinguish the default training configuration from the training choices that the threat model leaves unrestricted.

Top-two removal is a repair the defender applies to a finished update whose training it does not see. Section 3.1 states that the defender "holds the original and the compromised checkpoint but not the attack data", which leaves the update known and the training set unknown (see Lines 164-166). Neither the projections the attacker adapted nor the number of epochs it trained is therefore a choice this defense constrains. The threat model fixes neither of them, since the attacker picks both. The one place Section 3.1 gives the defender control over parameters is the freeze of Section 5, which assumes a provider that runs the fine-tuning (see Lines 158-160).

The default configuration is not a limit the threat model imposes. It is also the attack behind the LoRA block of Table 1, which the review calls convincing, so the regime where top-two removal works is inside the threat model, and so is its defeat.

What the repair achieves. Top-two removal is the kind of evidence a defense paper reports, a repair that restores refusal to 0.76 to 1.00 on four checkpoints over five doses (Table 16) and to 0.81 or above on three harmful benchmarks, at least 0.87 of the clean rate on each (Table 4).

Top-two removal supplies the working repair that the adaptive evaluation tests.

And how it is defeated. Section 6 then shows that choices inside the threat model weaken it. Seven projections and eight epochs take relative repair from 0.99 to 0.74 and 0.35 (see Lines 452-454, Table 17), and the spread attack defeats it at 0.000 (Table 3).

Training choices left to the attacker weaken the repair and let the spread attack defeat it.

The abstract states the repair and its defeat together because that pair is the finding, a repair that works across checkpoints yet does not survive an attacker who knows it (see Lines 019-025).

The sentence the review quotes. The submitted sentence the reviewer read, "neither choice is part of our threat model", says that the threat model fixes neither choice rather than that the configuration lies outside it. The same paragraph also states "Neither change is an attack on the defense" (see Lines 447-455). The revision states that reading in full, "The threat model fixes neither the projections nor the training length, since the attacker picks both, so this configuration and the changes below all lie inside it".

The same pair holds on Qwen2.5-14B and OLMo-2-13B, where top-two removal restores 1.00 after the attention-only three-epoch attack and the spread attack defeats it, leaving 0.00 to 0.22 (Table R6).

The paper studies a working repair and its adaptive defeat within the same threat model. This matters because repair under a fixed configuration does not establish robustness to attacker-chosen changes, as the revised subsection “Weight-space truncation repairs one training regime” explains.

AuthorsOpenReview ↗

Response to Reviewer kCtA (3/9)

W2. Further lineages and selection

The six checkpoints, four model families framing overstates independence, and the weakest-refusing model was pre-screened out.

With the effective sample closer to four independent lineages than six, and the weakest-refusing model dropped before any defense was tested, how confident can the authors be that freezing fails everywhere is not partly a selection effect of only testing models that already localize refusal cleanly?

Answer. The freezing evaluation tests refusal preservation across pretrained lineages using checkpoints selected by clean refusal. It does not require cleanly localized refusal for checkpoint selection. The added weak-refuser experiment tests whether selection explains failure.

To test whether freeze failure reflects selection for cleanly localized refusal, we evaluated added checkpoints, including a weak refuser, using attack validation, depth measurement and matched freezes.

A weak refuser from the family the submission excluded, put through the whole procedure, fails the same way. Mistral-Nemo-Instruct-2407 comes from the family of the excluded Mistral-7B and refuses only 0.68 to 0.76 of AdvBench when clean, below every checkpoint of Table 1.

It puts three checkpoints from the Gemma and Mistral lineages, gemma-2-9b-it, Ministral-8B-Instruct-2410 and Mistral-Nemo-Instruct-2407, through the whole procedure with a clean coherent refusal of at least 0.5 as the only selection criterion. All nine attacked seeds pass the validity checks and none is excluded. Freezing one layer deeper on the two Mistral checkpoints, to [0, 20] and [0, 19], covers the measured depth of every seed. That deeper freeze leaves refusal at 0.00 to 0.01.

The added weak refuser fails even when the frozen band covers every seed’s measured depth.

Table R8. Three further checkpoints through the full procedure. AdvBench, a hundred harmful examples, ranges over three training seeds. Depth is the transition layer by seed, Frozen is the band given by the three-seed mean, and Matched freezes as many layers at the top. The last two rows repeat the two checkpoints whose deepest seed lies above the mean band with the freeze one layer deeper, so the frozen band covers the measured depth of every seed.

ModelDepthFrozenRefusal cleanAttackFreezeMatchedLG unsafe cleanAttackFreezeMatched
gemma-2-9b-it (42 layers)24, 24, 24[0, 24].99-1.00.00.00.00-.06.00.95-.97.94-.97.87-.96
Ministral-8B-Instruct-2410 (36)20, 20, 17[0, 19].81-.85.00-.01.00-.05.01-.02.16-.18.93-.97.57-.86.89-.95
Mistral-Nemo-Instruct-2407 (40)19, 19, 15[0, 18].68-.76.00-.01.00-.01.00-.01.22-.32.95-.98.88-.92.96-.98
Ministral-8B-Instruct-2410 (36), one deeper20, 20, 17[0, 20].81-.85.00-.01.00-.01.00-.02.16-.18.93-.97.76-.88.94-.95
Mistral-Nemo-Instruct-2407 (40), one deeper19, 19, 15[0, 19].68-.76.00-.01.00-.01.01-.03.22-.32.94-.98.83-.93.90-.98

All added checkpoints lose refusal under the freeze, with no separation from the equal-sized control.

Beyond Table R8, the paper counts four pretrained lineages and computes its cross-model test on four (Section 3.1, see Lines 175-178, and Section 4.2, see Line 246). Table 1 (lower block) applies the freeze at the bands that Li et al. (2025) published for four further checkpoints, Llama-3-8B-Instruct, Llama-2-7b-chat, gemma-2b-it and Phi-3-mini-4k, which were chosen by their localization and not by ours.

Refusal after the frozen attack is 0.00 to 0.19 against clean rates above 0.92, and on none of the four is the published band separable from a control that freezes as many layers (see Lines 292-296). The exclusion of Mistral-7B-Instruct-v0.3 follows from a precondition of any refusal attack, since it refuses only 0.16 to 0.18 of AdvBench when clean and leaves the attack nothing to remove (Section 3.2, Appendix A).

The paper studies freezing across pretrained lineages, with failures spanning seven lineages and independently published localizations. This matters because a weak refuser from the excluded family also fails after the full procedure, as Appendix C, “Three further checkpoints through the full procedure,” reports.

Reviewer_MzYNOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper tests if freezing safety-relevant layers can maintain safety-refusals and conducts experiments across models.

Critical strengths

(1) the paper is timely.

Critical weaknesses

(1) limited novelty : most part of the paper is coming from existing literature and I also dont find the results surprising.

(2) threat model is limited : for open-models such defenses don't apply anyway. That leaves only some limited cases if any where such defenses can even apply in my opinion.

(3) Writing is very bad and AI slop-ish. Please rewrite as an academic paper.

Questions for authors

see above

19 comments
AuthorsOpenReview ↗

Response to Reviewer MzYN (1/17)

Response overview

At the layers Li et al. (2025) published as the ones to freeze, refusal after the attack is 0.00 to 0.19 against clean rates of 0.93 to 1.00. That refusal is not separable from a freeze of as many layers on any of the four checkpoints they report bands for. Seed by seed the two differ by at most 0.12 refusal in either direction over the twelve pairs, and the published band is the higher of the two on six of them and the lower on four (Table 1, lower block, see Lines 292-297). We thank the reviewer for finding the paper timely.

Before writing this response, we ran the new experiments the reviews ask for and restructured the paper around their results, which took much of the discussion period. We realize that little time remains, and we respectfully ask the reviewer to read our response and the new measurements.

Answer. The paper contributes the missing adaptive tests of localization-based defenses and identifies the damaging update, rather than proposing the published freeze as a new defense.

W1. Novelty beyond localization. The paper uses prior work's localization premise and freezing defense for a malicious-data matched-freeze comparison and damage-localizing interventions, without claiming the premise or defense as new. These results do not follow from the localization premise (Table 1 and Table R5).

W2. Provider deployment. The provider-side evaluation tests training restrictions and repair of received adapters, without studying an attacker serving its own open-weight model. The mixed-adapter measurements test the practical benefit of retaining useful adaptation and refusal (Section 3.1 and Table R20).

W3. Academic writing. The revision connects the tests, defines standard terms and places the conclusion in the main text. The requested rewrite carries the argument through its conclusion (Sections 1 to 7 and Section 8).

Beyond layer freezing, the paper also evaluates weight-space repair, an adaptive attack against it and a spectral detector (Section 6). The paper states that scope in the abstract and in the introduction's contribution list, at Lines 019-022 and Lines 096-099. The question is whether finding refusal identifies a place where it can be defended. The revision is written from the question the paper answers. It states the full scope in the abstract and at the close of Section 1, and carries the conclusion in the main text as Section 8. We would be grateful if the reviewer would read the paper in that form. The response explains what is new relative to the literature, which deployments the defenses serve, and what the revision changes in the writing.

The published localization does not distinguish the defense from its matched freezing control.

The matched comparison identifies a failure not established by the localization premise, and the provider setting gives that test its practical purpose.


W1. Novelty of the adaptive comparison

(1) limited novelty : most part of the paper is coming from existing literature and I also dont find the results surprising.

Answer. The paper uses prior work's refusal-localization premise and layer-freezing defense to test adaptive robustness and locate the damaging update. It does not claim the premise or the defense as new. The paper supplies SPPFT's missing malicious-data matched-freeze comparison and finds that its published layers provide no separable refusal protection.

The comparison holds the attack configuration fixed and changes the layers frozen at the provider.

AuthorsOpenReview ↗

Response to Reviewer MzYN (2/17)

W1. The published bands.

At the layers Li et al. (2025) published as the ones to freeze, refusal after the attack is not separable from a freeze of as many layers on all four checkpoints they report bands for (Table 1, lower block, see Lines 292-296).

MeasurementReported result
HarmfulnessAt those layers the HarmBench classifier rates the frozen attack 0.53 to 0.80 harmful against 0.40 to 0.84 for the unrestricted attack (Table R1).

The published bands do not separate refusal from a matched freeze and overlap unrestricted harm.

To compare direct harm judgments at the published bands with the localization premise, Table R1 collects the scorer evidence and corpus controls.

Table R1. All completions of Table 1 judged by two further harm scorers, the HarmBench classifier and the StrongREJECT evaluator. On every row and seed of Table 1 the frozen attack is more harmful than the clean model of the same seed on both scorers.

Full fine-tuning at the bands published by Li et al. (2025). AdvBench, a hundred harmful examples, ranges over seeds. HB is the share judged harmful, SR the mean score in [0, 1], higher is more harmful for both.

ModelFrozenHB cleanAttackFreezeSR cleanAttackFreeze
Llama-3-8B-Instruct, SPPFT band[6, 12].00-.02.41-.74.57-.80.00-.01.31-.62.43-.63
Llama-2-7b-chat, SPPFT band[6, 14].00-.01.50-.78.53-.72.00-.01.48-.77.45-.66
gemma-2b-it, SPPFT band[6, 11].03-.06.40-.67.54-.63.03-.06.42-.61.45-.50
Phi-3-mini-4k, SPPFT band[11, 15].00.69-.84.60-.77.00.61-.76.57-.72

Both added scorers rate the frozen attack above clean levels, with unrestricted-attack overlap at the published bands.

MeasurementReported result
HarmfulnessThe HarmBench classifier judges it harmful at 0.07 to 0.83, against 0.00 to 0.08 for the clean model of the same seed, with the checkpoint by checkpoint values in Table R1.
ComparisonAt the published SPPFT bands it overlaps the unrestricted attack on all four checkpoints, 0.53 to 0.80 against 0.40 to 0.84.
Scorer agreementOver the 12,454 distinct completions the HarmBench classifier agrees with Llama-Guard on 82.5 percent, at Cohen's kappa 0.64.
Scorer agreementIt agrees with StrongREJECT, thresholded at 0.5, on 81.5 percent, at Cohen's kappa 0.62.
Scorer agreementThe refusal metric itself agrees with Llama-Guard on 90.4 percent at kappa 0.76.
HarmfulnessOf the 9,682 completions the refusal metric counts as not refusing, Llama-Guard rates 8,521, or 88.0 percent, unsafe.
HarmfulnessOf the 2,772 it counts as refusing, it rates 36 unsafe.
RefusalOf the 12,454 completions, the 1,161 that match no refusal pattern and are rated safe by Llama-Guard are the restate-and-comment completions Appendix B characterizes.
ComparisonIn the six frozen LoRA conditions alone they number 535 of 1,800 with PKU-SafeRLHF, against 131 of 1,800 under the unrestricted attack.
ComparisonRetraining the same conditions on the second harmful corpus gives 68 of 1,800.

The scorer checks distinguish failure at the published bands from the lower LoRA harm reading that changes with the corpus.


W1. Prior work and the missing control, continued.

(1) limited novelty : most part of the paper is coming from existing literature and I also dont find the results surprising.

What prior work supplies, and how SPPFT was evaluated. What comes from the literature is the premise, that refusal is localized in layers, directions or tokens (Arditi et al., 2024, Li et al., 2025, Qi et al., 2025a), and the defenses built on it. SPPFT (Li et al., 2025) freezes the localized safety layers under a stated threat model whose attacker has "Limited Knowledge" and "lacks knowledge of the model's internal structure, parameters, and the location of safety layers" (Li et al., 2025, Appendix A.4.1). It was evaluated on ordinary, implicit-attack and backdoor data, and on a parameter-level attack, reverse DPO.

MeasurementReported result
ComparisonTwo checkpoints were also fine-tuned on a thousand ordinary examples with malicious examples added at fractions of 0.05, 0.1 and 0.2.

The published malicious-mixture setting includes harmful examples, but its control that freezes other layers uses none.

AuthorsOpenReview ↗

Response to Reviewer MzYN (3/17)

W1. The comparison not run.

The comparison that paper does not run.

MeasurementReported result
HarmfulnessIn that mixed setting their own Table 3 reports harmful rates of 68.3 and 78.8 percent at a malicious fraction of 0.2.
ComparisonTheir Table 2 gives initial rates of 1.35 and 0.77 for those two checkpoints, and full fine-tuning reaches 93.5 and 98.8 at the same fraction, so the paper reads the defense as mitigation.

Mitigation under malicious data cannot identify protection from localized layers without the corresponding matched freeze.

Reading the defense as mitigation rests on one comparison the paper does not run, the published layers against a freeze of as many layers under malicious data. The lower block of Table 1 runs it on all four of the checkpoints they report bands for. The SPPFT threat model excludes an attacker who knows which layers are frozen. The attack in the lower block needs no such knowledge. The three conditions share the same PKU-SafeRLHF pairs at a hundred examples, the same seeds, the same response-masked loss and the same three epochs at batch four, and differ only in which layers the provider freezes, so the published bands fail without the attacker using what that model withholds.

The previously absent matched comparison finds no refusal protection specific to the published layers, answering what this paper adds beyond localization. Revised-PDF location. Related work, “Where refusal is computed, and freezing it as a defense.”


W1. Novelty of the adaptive comparison, continued.

most part of the paper is coming from existing literature and I also dont find the results surprising.

Whole-layer freezing fails in the matched adaptive comparison, a result not established by earlier sparse-neuron failures.

We distinguish the prior sparse-neuron result from whole-layer freezing, damage location, weight-space repair and detection.

What this paper adds. Wei et al. (2024) show that restricting modifications to the sparse safety-critical regions they identify does not stop fine-tuning attacks, and Qi et al. (2025b) that small changes to the fine-tuning configuration defeat safeguards built into alignment. Li et al. (2025) ran that neuron freeze themselves at a matched proportion and report it "proved ineffective in preventing security degradation during fine-tuning, consistent with their original findings". They conclude that SPPFT "proves to be the most effective parameter-freezing defense" because it holds whole layers rather than discrete neurons. After Wei et al. (2024), then, the open question was precisely whether a whole-layer defense survives an attacker who knows it, and that is the question answered here. The paper supplies that evaluation. It goes beyond Wei et al. (2024) in locating where the damage goes once the localized layers are frozen and in testing weight-space repair and a spectral detector against an adaptive attacker, which Wei et al. (2024) do not study. In adversarial robustness, adaptive evaluation of this kind became the standard after defenses that looked sound failed once the attacker knew them (Athalye et al., 2018, Tramèr et al., 2020).

Whole-layer survival remained an open question after the sparse-region failures, and the added interventions test what happens afterward.

The contribution evaluates defenses beyond the localization premise supplied by prior work. Revised-PDF location. Related work.


W1. Results beyond the premise, continued.

most part of the paper is coming from existing literature and I also dont find the results surprising.

Update interventions establish damage relocation, and adaptive attacks eliminate the projection's advantage over norm-matched shrinkage.

We compare layer-window interventions, dose dependence and repair with a norm-matched null.

The layers that carry the damage under the freeze lie inside the region above the boundary.

MeasurementReported result
RefusalWhen the attacker may write everywhere, an update kept only in that region keeps 0.79 to 0.99 refusal.
Update interventionThe freeze therefore changes what those layers learn rather than where the attacker can write.
Update interventionThe damaging window starts one to three layers above the boundary on all six checkpoints.
Update interventionIt is 4 of 16 writable layers on Tulu-3-8B-DPO and 3 of 15 on OLMo-2-7B.
Update interventionOn those two it holds only .30 to .34 of the squared update norm.
Update interventionOn Llama a second window at layers 28 to 31 joins it, and the two together cover 10 of the 14 adapted layers there.
RefusalSwitching off these windows restores refusal to 0.70 to 0.97.
Update interventionSwitching off any block of the same grid outside the windows leaves 0.00 to 0.11, and 0.00 to 0.15 on prompts the windows were not selected on.

Comments on the paper

6 comments
AuthorsOpenReview ↗

General Response (1/5)

General Response

We thank the three reviewers. Every measurement the reviews ask for was run with the submission's own protocol, three seeds and the same evaluation, and every one of them is in the revision. Every failure the submission reports is reproduced by the new measurements, and where a scorer or a matched control reads it differently the answers below say so and give the numbers. Two of the submission's readings depended on the attack corpus, and the new measurements turn those two into stronger ones.

Answer. The added measurements reproduce the reported failures and identify which favorable readings depend on the attack corpus.

Refusal and harmfulness. Direct scorers confirm degradation independently of refusal. The harm reduction depends on the attack corpus (Table R1 and Table R2).

Adaptive novelty. The published bands add no separable refusal protection over an equal-sized freeze. Localization alone does not establish a defense (Section 5.1).

Damage relocation. Update intervention identifies damage above the frozen boundary. This supplies causal evidence beyond a recovery-depth shift (Table R5).

Generality. Added lineages, objectives and checkpoints reproduce failure. The findings extend beyond the original settings (Table R6, Table R8 and Table R9).

The two readings that depended on the attack corpus. The freeze's lower Llama-Guard rate belongs to that corpus, and the harmful fraction at which its one protective regime ends is set by the checkpoint and the corpus. With a second harmful corpus the frozen attack is within 0.04 of the unrestricted attack by Llama-Guard on every checkpoint, at 0.92 to 0.98 unsafe against 0.95 to 0.99. On Llama-3.1-8B the 5% mixed adapter holds 0.16 to 0.50 refusal with that corpus against 0.74 to 0.92 with PKU-SafeRLHF.

The corpus changes both unsafe-rate reduction and low-fraction refusal protection.

The defense at the bands its authors published. The published SPPFT bands are the defense as its authors proposed it. At those bands refusal after the attack is 0.00 to 0.19 against clean rates of 0.93 to 1.00. On those same four checkpoints both of the harm scorers we add give the frozen attack a harm range overlapping the unrestricted attack's range, at 0.53 to 0.80 against 0.40 to 0.84 on the HarmBench classifier. Under all three scorers the frozen attack is more harmful than the clean model on every checkpoint and seed of Table 1.

Harm at the published bands overlaps the unrestricted attack and exceeds clean levels.

The favorable readings followed the attack corpus and scorer, and the published bands do not separate the defense from an equal-sized freeze.

Table and line numbers refer to the revised PDF. Revised-PDF location. This is the gap Section 5.1 of the revision reports, and the second corpus closes it.

AuthorsOpenReview ↗

General Response (2/5)

Novelty and deployment

Answer. The contribution is the adaptive comparison that SPPFT did not receive, together with locating the damaging update.

The comparisons use the published bands, matched freezes, direct update interventions and adaptive weight-space attacks.

SPPFT is the defense built most directly on a localization of refusal. Its threat model gives the attacker "Limited Knowledge" and says the attacker "lacks knowledge of the model's internal structure, parameters, and the location of safety layers" (Li et al., 2025, Appendix A.4.1). The defender, by contrast, is given "the location and functionality of the safety layers". No experiment in that paper trains the writable layers on harmful pairs alone. Its control that freezes as many layers is reported only on data with no malicious examples. Whether the published layers protect refusal better than a freeze of as many layers under harmful training was therefore never measured, though the defense itself is reported there under malicious mixtures. The lower block of Table 1 supplies that measurement (see Lines 292-296).

Three results of our evaluation do not follow from the localization premise. At the bands Li et al. (2025) published, refusal after the attack is 0.00 to 0.19 against clean rates of 0.93 to 1.00. On all four checkpoints they report bands for, that refusal is not separable from a freeze of as many layers, which differs from it by at most 0.12 with the same seed (Table 1, lower block, see Lines 292-296). When the attacker may write everywhere, the update confined to the layers above the boundary keeps most of the clean refusal, at 0.79 to 0.99. Once the localized layers are frozen, the damage is in a window inside that same region, and switching that window off restores refusal to 0.70 to 0.97 (Table R5). Any block of the same grid outside those windows leaves at most 0.15.

Locating refusal does not predict where damage occurs after the localized layers are frozen.

The repair and its defeat. Top-two removal repairs five checkpoints, the four of Table 16 over five doses and OLMo-2-13B (Table R6), and an attacker who spreads the update defeats it on Llama-3.1-8B (Table 3) and on Qwen2.5-14B and OLMo-2-13B (Table R6). The spectral detector calibrated on benign fine-tunes of each checkpoint misses every ordinary fine-tune whose repair fails (Table R6). A working repair therefore does not establish adaptive robustness, and the detector does not identify the ordinary failures a provider needs to catch.

Where these defenses apply. These defenses are provider-side, applying where a provider runs the customer's fine-tuning or receives the finished adapter (Section 3.1, see Lines 158-165, stated also at Lines 070-071). The paper names those two settings as the ones that make few-sample attacks cheap and widely accessible (see Lines 050-051).

The localization premise does not predict where damage lands, and ordinary training changes defeat the repair a provider could run.

AuthorsOpenReview ↗

General Response (3/5)

The new measurements, Table R1 to Table R5

Every measurement below was run with the submission's protocol, three seeds and the same evaluation, and the revised PDF reports each one in the appendix of the section it extends. The response to each reviewer prints the tables that answer that review.

  • Table R1. All completions of Table 1 judged by two further harm scorers, the HarmBench classifier and the StrongREJECT evaluator. On every row and seed of Table 1 the frozen attack is more harmful than the clean model of the same seed on both scorers. The HarmBench classifier judges it harmful at 0.07 to 0.83, against 0.00 to 0.08 for the clean model of the same seed, with the checkpoint by checkpoint values in Table R1. At the published SPPFT bands it overlaps the unrestricted attack on all four checkpoints, 0.53 to 0.80 against 0.40 to 0.84.

  • Table R2. The freeze repeated with a second harmful corpus, at the same frozen bands on all six LoRA checkpoints of Table 1. Each seed draws a hundred pairs from 400 prompt and response pairs of the harmful dataset released with latent adversarial training (Sheshadri et al., 2025). Those pairs share no prompt with the evaluation sets. Refusal after the frozen attack is 0.00 to 0.16, and its Llama-Guard unsafe rate of 0.92 to 0.98 is within 0.04 of the unrestricted attack trained with the same seed on every checkpoint. With this corpus the HarmBench classifier rates the frozen attack 0.78 to 0.95 harmful, against 0.89 to 0.99 for the matched freeze.

  • Table R3. The Table 1 freeze evaluated on two further harmful benchmarks, 100 HarmBench and 60 StrongREJECT prompts. Refusal stays at or below 0.05 on all six checkpoints over three seeds. Llama-Guard unsafe is 0.50 to 0.84 on the HarmBench prompts against 0.03 to 0.30 for the clean models.

  • Table R4. The low-dose freezes of Figure 4 and Table 7 read with Llama-Guard beside refusal, with new freezes at twenty-five harmful examples on Yi and at fifty on Tulu, OLMo-2-7B, Qwen and Yi. On Llama both scorers place the breaking dose between ten and twenty-five examples. At fifty examples refusal under the freeze is 0.00 to 0.14 on Tulu, OLMo-2-7B, Qwen and Yi, and Llama-Guard separates the four checkpoints. On Qwen2.5-14B the frozen attack has by then reached the matched freeze, 0.68 to 0.80 unsafe against 0.71 to 0.83, so there the protection is gone.

  • Table R5. The location of the damaging update, found by switching the LoRA update off in a window of layers or keeping it only there, on the six checkpoints of Table 1. Without a freeze the band's update alone removes refusal (0.00 to 0.17) and the update above it leaves 0.79 to 0.99. With the band frozen a window starting one to three layers above the boundary carries the damage, joined on Llama by a second window at layers 28 to 31, and switching these windows off restores refusal to 0.70 to 0.97.

The measurements confirm harm above clean levels and quantify agreement between refusal patterns and harm scorers. The safe-rated non-refusal counts track the attack corpus rather than refusal preserved by freezing.

Both added scorers place the frozen attack far above clean levels, and its safe-rated completions nearly vanish after the corpus change.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.