acceptodds
Under review as a conference paper at ICLR 2027

AlignLock: Align and Freeze Safety-Critical Layers for Robust LLM Fine-Tuning

Abstract

Safety alignment in large language models (LLMs) is fragile under downstream adaptation: even benign fine-tuning can weaken refusal behaviors and increase susceptibility to jailbreaks. Such degradation arises from the interference between safety alignment and downstream task specialization when they modify shared model parameters. Existing defenses mainly focus on mitigating safety degradation during or after adaptation, which cannot prevent direct overwrite of safety-critical parameters. We propose AlignLock, a proactive layer-level framework for robust safety preservation under downstream adaptation. AlignLock decouples safety alignment and task specialization through a localize-align-lock paradigm: it first identifies a safety-critical layer block via layer-wise sensitivity analysis, performs targeted safety alignment by restricting safety updates to this block, and then locks the same block during downstream fine-tuning to prevent direct overwrite from task gradients. We provide an interference-based analysis showing that locking the safety-critical block eliminates direct overwrite from downstream gradients, while remaining safety drift is limited to indirect interactions through updated task parameters. Experiments across multiple open-weight LLM families and benign downstream domains show consistent safety gains over vanilla fine-tuning with competitive task utility. In the four main model comparisons, AlignLock attains the lowest malicious rates and StrongREJECT scores of the compared methods. It also mitigates malicious fine-tuning under provider-enforced layer locking.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM Fine-Tuning, Safety Alignment

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_iwfWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates the degradation of safety alignment during downstream adaptation, including benign task fine-tuning and malicious adaptation. It introduces AlignLock, a framework that first identifies a contiguous block of Transformer layers sensitive to refusal behavior, then concentrates the main safety alignment updates within that block, and finally freezes it during downstream adaptation. The localization stage uses a brief safety alignment pass followed by weight scaling perturbations to measure changes in refusal tendency. The main alignment stage begins again from the original instruction-tuned checkpoint, while downstream training updates only the remaining parameters. The authors provide an upper bound on safety loss drift, showing that this design prevents direct overwriting of the protected alignment parameters but does not eliminate indirect effects through the unlocked parameters.

Critical strengths
  1. The paper considers safety preservation across the alignment and adaptation stages rather than treating downstream adaptation in isolation. Concentrating safety alignment updates in a block that is subsequently frozen is a clear and meaningful distinction from methods that only protect selected layers during task training.
  2. The empirical evaluation covers code, mathematics, and finance tasks, several model families, malicious adaptation with enforced training constraints, and multiple jailbreak attacks. The main results and component ablations provide useful evidence that coordinating the two stages can improve safety retention.
Critical weaknesses
  1. The generalization of safety layer localization across harmful data distributions remains insufficiently examined. The localization procedure uses refusal boundary prompts drawn from Over-Refusal or ADVBench, but the paper does not systematically compare the selected intervals across different harmful datasets or categories, assess their overlap or ranking stability, or test whether an interval identified on one distribution transfers to another. StrongREJECT and jailbreak evaluations provide evidence of behavioral robustness beyond the localization prompts, but do not establish that the selected block remains stable or effective across unseen harmful distributions. This is important because the localization objective depends on the probe distribution.
  2. The contribution of layer localization is not adequately separated from the effects of restricting trainable parameters. On Qwen-2.5-7B, freezing 17 contiguous layers protects a substantial portion of the model. Appendix Table 10 shows that freezing 15 layers improves MMLU and code Pass@1 at the cost of safety, whereas freezing 19 layers further reduces MMLU. Selecting the 17 most sensitive layers without requiring contiguity lowers the malicious rate to 7.4%, but MMLU and Pass@1 fall to 0.572 and 0.453, respectively. Comparisons with randomly chosen or position-based blocks under matched parameter budgets, along with controls for task sensitivity, would help clarify whether the improvements arise from the proposed localization criterion rather than freezing alone.
  3. The evidence supports separation of parameter updates, but not necessarily functional disentanglement of safety and task capabilities. The two training stages update disjoint parameter sets by construction. However, refusal sensitivity alone does not demonstrate that the selected layers predominantly encode safety, that they are unimportant for task learning, or that the unlocked layers have little influence on safety. Indeed, Eq. (8) explicitly allows safety drift through updates to the unlocked parameters. Without quantifying the influence of each parameter block on both objectives, the stronger interpretation of structural separation between safety and task functions remains insufficiently supported.
Questions for authors
  1. How consistent are the selected safety critical intervals when localization is performed separately using Over-Refusal, ADVBench, or distinct harmful categories? Could the authors report interval overlap and assess transfer by localizing on one dataset and evaluating safety retention on an unseen dataset under the same downstream training conditions?
  2. What empirical evidence supports functional separation of safety and task capabilities beyond the disjoint parameter updates? Could the authors measure safety and task sensitivities in both the locked and unlocked blocks, or use controlled interventions to evaluate how modifications to each block affect both objectives?
  3. How does AlignLock compare with randomly selected contiguous blocks, blocks selected solely by layer position, and selections informed by task sensitivity when the number of frozen parameters is matched? Are the conclusions consistent across different downstream domains and training budgets?
  4. Could the authors report the complete time and computational cost of localization, including the perturbation-based interval search, together with the number of candidate intervals and probe examples evaluated? Since the reported 82 minutes total excludes interval search, an estimate of the full cost and its amortization across multiple downstream tasks would make the efficiency comparison more informative.
5 comments

Q1: Localization consistency across probe distributions.

How consistent are the selected safety-critical intervals when localization is performed separately using Over-Refusal, ADVBench, or distinct harmful categories? Could the authors report interval overlap and assess transfer by localizing on one dataset and evaluating safety retention on an unseen dataset under the same downstream training conditions?

A1:

We independently localized safety-critical layers using AdvBench and OR-Bench probes, applying the same perturbation-based criterion (Eq. 9). The block used in the main AlignLock pipeline was selected using AdvBench, while the OR-Bench-based localization serves as an independent consistency analysis. On Qwen2.5-7B, both datasets select [2,18], yielding an intersection-over-union (IoU) of 100%. On Llama3-8B, the selected intervals are [4,12] and [4,13], respectively: they differ by one layer and have an IoU of 90%. Thus, the two probe distributions identify highly overlapping intervals on both tested models.

The layer-count ablation in the submitted paper also provides evidence that safety gains persist under small changes to the interval boundaries. On Qwen2.5-7B, nearby intervals containing 15–19 layers yield malicious rates of 8.3–12.4%, compared with 27.8% for Vanilla. Although safety and utility vary across these intervals, all retain substantial safety improvements. This complements the localization-overlap results, without establishing equivalent downstream performance for the two Llama3-8B intervals.

To assess transfer beyond the localization probes, we use the AdvBench-selected Qwen2.5-7B block [2,18] for targeted safety alignment and code fine-tuning, then evaluate on the 313 harmful StrongREJECT prompts. The safety-critical block is localized using AdvBench probes, while the 313 StrongREJECT prompts are reserved for downstream safety evaluation. The StrongREJECT evaluation results are not used to determine or select the localized block. Under the same downstream training protocol, AlignLock achieves an 8.3% malicious rate and a 0.125 StrongREJECT score, compared with Vanilla's 27.8% and 0.321. These results support safety retention on a held-out dataset beyond the probes used for localization.

Together, the interval overlap, the nearby-interval ablation reported in the submitted paper, and the held-out evaluation support the consistency and transferability of the localization strategy in the tested settings. Category-wise localization and systematic transfer comparisons across all probe sources remain unevaluated.

Q2: Functional separation of safety and task capabilities.

What empirical evidence supports functional separation of safety and task capabilities beyond the disjoint parameter updates? Could the authors measure safety and task sensitivities in both the locked and unlocked blocks, or use controlled interventions to evaluate how modifications to each block affect both objectives?

A2:

We clarify that AlignLock assigns non-overlapping layers to safety-alignment and downstream fine-tuning updates; it does not assume that safety behavior and general capabilities are encoded in separate layers. Localization is motivated by prior studies of layer-wise safety sensitivity, as discussed in the submitted paper, and supported by our perturbation analysis and ablations. This evidence motivates selecting a safety-sensitive contiguous block for targeted alignment. Starting from the instruction-tuned checkpoint, we restrict the main safety-alignment updates to this block and keep all other parameters fixed. This is a choice of where to perform alignment updates, not a claim that the selected layers serve only a safety function.

After alignment, we freeze the selected block and restrict downstream fine-tuning to the remaining layers. Freezing the aligned block prevents parameter updates; it does not remove the block from the model's computations. During downstream inference, the updated unlocked layers and the locked layers jointly produce the model's outputs, supporting performance on the downstream task. The competitive general usability and downstream results in Tables 2–3 show that this update restriction retains sufficient adaptation capacity for the evaluated downstream tasks and models. Our objective is therefore to preserve safety-alignment updates while maintaining useful adaptation of the full network, not to establish functional separation between its layers.

Figure 2 (Appendix H) provides perturbation-based behavioral evidence beyond the disjoint parameter updates. We apply layer-wise scaling interventions and measure their effects on refusal tendency across training stages. Within the localized safety-critical block [2,18], the mean signed relative refusal-tendency sensitivity decreases from 0.161 to 0.026 after Vanilla downstream fine-tuning. In contrast, AlignLock largely preserves the refusal-related sensitivity pattern established during targeted safety alignment, with a much smaller change from 0.206 to 0.172. This contrast is consistent with the localized block remaining influential for refusal behavior after downstream adaptation, whereas unrestricted fine-tuning substantially attenuates its measured signed relative sensitivity.

Table 4 provides further evidence through controlled ablations. When downstream fine-tuning is allowed to update the aligned block, the malicious rate increases from 8.3% to 23.6% and the StrongREJECT score from 0.125 to 0.274. Moreover, removing the restriction on safety-alignment updates increases the malicious rate to 18.2%. These results demonstrate that both concentrating safety-alignment updates within the selected block and protecting that block during downstream adaptation contribute to safety retention.

Together, these results support the effectiveness of assigning disjoint update locations to safety alignment and downstream adaptation, rather than functional disentanglement. Locking the aligned block prevents direct overwrite of its alignment updates, but changes to unlocked layers can still affect safety indirectly, as characterized by Eq. (8). We will make this distinction explicit in a revised version of the paper.

Q3: Matched-size layer selection and downstream generalization.

How does AlignLock compare with randomly selected contiguous blocks, blocks selected solely by layer position, and selections informed by task sensitivity when the number of frozen parameters is matched? Are the conclusions consistent across different downstream domains and training budgets?

A3:

Thank you for suggesting these controls. During the rebuttal, we prioritized matched-size comparisons that directly test whether the gains depend on the choice of layers, rather than merely the number of protected layers. On Qwen2.5-7B, we compare our localized 17-layer block [2,18] with Bottom-17 [0,16], Top-17 [11,27], and the 17 most safety-sensitive layers selected individually (non-contiguous Top-17). Layer indices are zero-based and inclusive. All variants follow the same two-stage align-then-lock procedure, with matched safety-alignment and downstream fine-tuning schedules. MR denotes malicious rate and SR denotes StrongREJECT score.

Table R1: Comparison of 17-layer selection strategies on Qwen2.5-7B under code fine-tuning.

Layer SelectionMR ↓SR ↓MMLU ↑Pass@1 ↑
AlignLock [2,18]8.3%0.1250.6650.528
Bottom-17 [0,16]11.6%0.1580.6710.532
Top-17 [11,27]14.8%0.1890.6080.493
Non-contiguous Top-177.4%0.1210.5720.453

Both fixed-position alternatives improve safety over Vanilla fine-tuning (27.8% malicious rate), but AlignLock achieves lower malicious rates: 8.3%, compared with 11.6% for Bottom-17 and 14.8% for Top-17, with corresponding improvements in StrongREJECT scores. Bottom-17 has slightly higher MMLU and Pass@1, while Top-17 performs worse on both safety and utility. The non-contiguous Top-17 achieves slightly better safety metrics than AlignLock (7.4% vs. 8.3% MR; 0.121 vs. 0.125 SR), but MMLU falls from 0.665 to 0.572 and Pass@1 from 0.528 to 0.453. Thus, AlignLock achieves stronger safety than the position-based controls and preserves substantially more utility than the non-contiguous selection.

The same localized block also supports multiple downstream adaptations without relocalization. As reported in Appendix Table 5 of the submitted paper, AlignLock reuses [2,18] on Qwen2.5-7B across Code, Math, and Finance, reducing malicious rates relative to Vanilla from 27.8% to 8.3%, 20.1% to 4.2%, and 32.6% to 9.9%, respectively. These results support the practical value of localizing once per model and reusing the block across downstream domains.

Together, the matched-size controls provide direct evidence that layer selection affects the safety–utility trade-off beyond the number of protected layers. Random and task-sensitive selection controls, comparisons of selection strategies across domains, and training-budget sweeps would broaden this evaluation; the present controlled comparisons substantiate the importance of selecting an appropriate block under the tested conditions. We will incorporate the results presented here and clarify their scope in a revised version of the paper.

Reviewer_MndeOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes AlignLock, a framework that coordinates safety alignment and downstream task fine-tuning through a localize-align-lock procedure. The method identifies safety-critical layers, concentrates safety-alignment updates within these layers, and freezes them during subsequent task adaptation to preserve safety-related behavior.

Critical strengths

The method is clearly designed and straightforward, treating safety alignment and downstream fine-tuning as two stages that should be jointly considered. The key ablation experiments directly support the proposed design.

The experimental evaluation is comprehensive, with extensive baseline comparisons including SPPFT, SPF, SOT, SafeLoRA, and RESTA. The authors evaluate multiple model families, including Qwen, Llama, Gemma, and OLMoE. The evaluation also extends beyond benign fine-tuning to malicious fine-tuning and jailbreak attacks such as GCG, AutoDAN, ReNeLLM, and Crescendo.

Critical weaknesses

1. Potential over-refusal and the safety–utility trade-off.

The core localization mechanism of AlignLock relies on the model's refusal tendency. However, reducing the non-refusal rate on harmful requests does not necessarily demonstrate an improvement in the overall safety–utility trade-off. The model may simply become more inclined to refuse requests involving safety-sensitive topics, even when those requests are legitimate.

Although the authors use the Over-Refusal dataset to construct safety-boundary probes, they do not report the final model's false-refusal rate on a dedicated benchmark of benign but safety-sensitive requests. General utility metrics alone may not adequately capture this issue.

2. Reliability and stability of improvements over strong baselines.

In Table 1, AlignLock outperforms SPF, but the differences are relatively small in several settings, sometimes below one percentage point.

Although the paper reports results averaged over five runs, it is unclear whether these runs capture variability across independently trained models or only evaluation randomness. The current evidence is therefore insufficient to establish whether the relatively small improvements over SPF are consistently reproducible across different training seeds.

Questions for authors

1. Over-refusal and legitimate safety-sensitive requests.

Can the authors clarify whether AlignLock preserves the model's ability to answer legitimate safety-sensitive questions, rather than merely increasing its refusal tendency? In particular, is there evidence that the final model does not exhibit substantially increased false refusals on benign but safety-sensitive requests?

2. Stability of improvements over SPF.

Are the improvements reported in Table 1 stable across training runs with different random seeds? Is there sufficient evidence to support the conclusion that AlignLock consistently outperforms SPF, rather than the observed differences being attributable to variability between runs?

1 comment

Q1: Over-refusal and legitimate safety-sensitive requests.

Can the authors clarify whether AlignLock preserves the model's ability to answer legitimate safety-sensitive questions, rather than merely increasing its refusal tendency? In particular, is there evidence that the final model does not exhibit substantially increased false refusals on benign but safety-sensitive requests?

A1:

To assess whether AlignLock preserves the ability to respond to legitimate safety-sensitive requests, we conducted a dedicated over-refusal evaluation on 731 benign questions from OR-Bench using Qwen2.5-7B. Notably, the safety-critical block [2,18] for the evaluated Qwen2.5-7B checkpoint was selected using AdvBench probes, rather than OR-Bench. The 731 benign OR-Bench evaluation questions were therefore not used to select the block for this checkpoint.

Table R1: Over-refusal evaluation on 731 benign questions from OR-Bench.

MethodOver-Refusal Rate ↓
Base Model1.4%
Vanilla9.3%
AlignLock (Ours)9.6%
SPPFT9.2%
SPF11.5%
SOT10.8%
SafeLoRA9.7%
RESTA14.7%
EnchTable11.5%
SGDG8.6%
LISA12.3%

AlignLock's observed over-refusal rate is close to Vanilla's (9.6% vs. 9.3%), a difference of 0.3 percentage points, and lower than SPF's (11.5%) and several other safety-preservation methods. Meanwhile, its malicious rate on harmful requests falls from 27.8% under Vanilla to 8.3%, a reduction of 19.5 percentage points, with competitive general usability and downstream task performance (Tables 1–3 of the submitted paper).

These results suggest that AlignLock's safety gain over Vanilla is not explained simply by a broad increase in refusal tendency. This conclusion is limited to the comparison with Vanilla: both adapted models exhibit higher over-refusal rates than the base model (1.4%). Accordingly, the results indicate that AlignLock does not introduce a substantial additional increase in false refusals over Vanilla on the evaluated benign requests, but do not establish full retention of the base model's original benign responsiveness.


Q2: Stability of improvements over SPF.

Are the improvements reported in Table 1 stable across training runs with different random seeds? Is there sufficient evidence to support the conclusion that AlignLock consistently outperforms SPF, rather than the observed differences being attributable to variability between runs?

A2:

To address this concern, we report mean ± standard deviation over five independent random seeds for all fine-tuning methods in the revised Table 1 provided in this rebuttal, which we will incorporate into a revised version of the paper. The comparison between AlignLock and SPF across four models is summarized below; malicious-rate standard deviations are in percentage points.

Table R2: Safety performance of AlignLock and SPF over five random seeds.

ModelMethodMalicious Rate ↓StrongREJECT ↓
Qwen2.5-7BAlignLock(8.3 ± 0.2)%0.125 ± 0.003
SPF(8.9 ± 0.3)%0.132 ± 0.003
Qwen2.5-0.5BAlignLock(0.0 ± 0.0)%0.000 ± 0.000
SPF(1.6 ± 0.3)%0.030 ± 0.002
Llama3-8BAlignLock(9.3 ± 0.2)%0.147 ± 0.003
SPF(10.2 ± 0.3)%0.151 ± 0.002
Llama3-1BAlignLock(1.6 ± 0.1)%0.030 ± 0.001
SPF(1.9 ± 0.2)%0.033 ± 0.002

AlignLock achieves lower mean malicious rates and StrongREJECT scores than SPF across all four evaluated models, with small standard deviations. On Qwen2.5-7B, its malicious rate is no higher than SPF's in any of the five matched-seed runs, providing individual-run evidence in addition to the aggregate statistics. Together, these results support lower mean safety metrics across the four models and consistent matched-seed performance on Qwen2.5-7B. We interpret these findings as evidence of consistency over the evaluated seeds, rather than a claim of statistically significant superiority in every setting.

Reviewer_u6ZdOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AlignLock, which is a localize-align-lock pipeline to preserve safety during downstream adaptation. The pipeline first does a short alignment pass to identifies a sensitive contiguous layer block. and then only update that block in the downstream finetunning. The authors provides an through experiments includes different model families, components and layer selection ablations etc. The results shows the improvement under proposed method.

Critical strengths
  1. Broad evaluation with meaningful utility measures. This paper include executable code performance and MMLU, math, finace, with Gemma and MoE model.
  2. The method goes beyond protecting layers only during adaptation by also restricting where the prededing alignment updates are learned.
  3. the layer-sensitivity analysis before and after finetunning gives mechanistic support.
Critical weaknesses
  1. Margins compared to the best baselines are small -- AlignLock vs SPF is 8.3 % vs 8.9%. could you please provide std on the table?
  2. A same-size controls is missing: could you please help add the same block size control maybe selected at random or at the bottom or top of the network under same align-then-lock protocol?
Questions for authors

As mentioned above

1 comment

Q1: Margins compared to the best baselines are small -- AlignLock vs. SPF is 8.3% vs. 8.9%. Could you please provide std on the table?

A1:

To address this concern, we report mean ± standard deviation over five independent random seeds for all fine-tuning methods in the revised Table 1 of the manuscript, which we will incorporate into the revised version. For clarity, the comparison between AlignLock and SPF across four models is summarized below; malicious-rate standard deviations are expressed in percentage points.

Table R1: Safety performance of AlignLock and SPF over five random seeds.

ModelMethodMalicious Rate ↓StrongREJECT ↓
Qwen2.5-7BAlignLock(8.3 ± 0.2)%0.125 ± 0.003
SPF(8.9 ± 0.3)%0.132 ± 0.003
Qwen2.5-0.5BAlignLock(0.0 ± 0.0)%0.000 ± 0.000
SPF(1.6 ± 0.3)%0.030 ± 0.002
Llama3-8BAlignLock(9.3 ± 0.2)%0.147 ± 0.003
SPF(10.2 ± 0.3)%0.151 ± 0.002
Llama3-1BAlignLock(1.6 ± 0.1)%0.030 ± 0.001
SPF(1.9 ± 0.2)%0.033 ± 0.002

AlignLock achieves lower mean malicious rates and StrongREJECT scores than SPF across all four evaluated models, with small standard deviations. On Qwen2.5-7B, its malicious rate is no higher than SPF's in any of the five matched-seed runs. Although the absolute malicious-rate improvement over SPF is modest (0.6 percentage points on Qwen2.5-7B), these results support its consistency across the evaluated seeds. Moreover, Tables 2 and 3 of the submitted paper show higher MMLU and downstream task scores (Pass@1, ROUGE-L, and AI-score) than SPF across all four models. Thus, the observed safety gains are accompanied by better general and downstream utility than SPF in these comparisons.


Q2: A same-size controls is missing: could you please help add the same block size control maybe selected at random or at the bottom or top of the network under same align-then-lock protocol?

A2:

To address this concern, we conducted matched 17-layer block selection ablations on Qwen2.5-7B. Specifically, we compare AlignLock's safety-sensitive block [2,18] with two fixed-position contiguous blocks: Bottom-17 [0,16] and Top-17 [11,27], using zero-based, inclusive layer indices. All three variants follow the same two-stage align-then-lock protocol, aligning only their respective selected blocks during safety alignment and freezing the same blocks during downstream fine-tuning. The training datasets, optimization settings, and budgets are kept identical. In the table, MR denotes malicious rate and SR denotes StrongREJECT score.

Table R2: Same-size block selection ablations on Qwen2.5-7B under code fine-tuning.

Layer SelectionMR ↓SR ↓MMLU ↑Pass@1 ↑
AlignLock [2,18]8.3%0.1250.6650.528
Bottom-17 [0,16]11.6%0.1580.6710.532
Top-17 [11,27]14.8%0.1890.6080.493

Both fixed-position blocks provide substantial safety improvements over Vanilla fine-tuning (27.8% malicious rate), but neither matches the safety retention achieved by AlignLock. AlignLock reduces the malicious rate to 8.3%, compared with 11.6% for Bottom-17 and 14.8% for Top-17, with corresponding improvements in StrongREJECT scores. In terms of utility, Bottom-17 achieves only marginally higher MMLU and Pass@1, while Top-17 performs worse on both safety and task performance. These results demonstrate that the effectiveness of AlignLock depends not merely on protecting a fixed number of layers, but also on selecting an appropriate safety-sensitive block. The matched comparisons further support the importance of sensitivity-guided localization within the two-stage align-then-lock framework.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.