acceptodds
Under review as a conference paper at ICLR 2027

Coordinate Concentration of Safety Alignment in Low-Rank Adapters

Abstract

Fine-tuning a safety-aligned language model on benign data is enough to degrade its refusal behavior. The established defense projects the low-rank update away from an alignment-sensitive subspace, which assumes that the update has to be steered around safety rather than kept out of it. We ask where that subspace sits among the coordinates of the adapter itself. If safety energy concentrates on a small fraction of coordinates rather than spreading across all of them, those coordinates can be identified once and held fixed during any subsequent fine-tuning, replacing the projection with a simpler intervention that does not depend on the downstream task. To answer this, we approximate the curvature of the loss on harmful prompts and measure how sensitive the model's safety behavior is to changes at each adapter coordinate: coordinates where the loss curves sharply are the ones whose movement during fine-tuning would most disrupt alignment. Across three model families, we find that the distribution of safety-sensitive energy is heavy-tailed, and one tenth of the coordinates carries to of it, a property we call coordinate concentration. Holding that tenth fixed during fine-tuning preserves refusal behavior across four base models and two harmful-prompt benchmarks, performing comparably to three published defenses while requiring nothing of the downstream user beyond attaching the mask. The concentration also explains why targeted selection matters even at a heavy lock rate. A random mask of equal size locks nearly the same coordinates as a targeted one, yet performs substantially worse, because the small set of coordinates each leaves free to train are almost entirely different, and safety is determined by that free set.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
AI safety, safety degradation, safety-critical parameters, LoRA

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_XtdwOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether fixing a small number of key coordinates in LoRA can avoid complex safety subspace projections when benign fine-tuning leads to safety degradation. Because existing research has identified safety-related neurons, layers, and subspaces, it hasn't yet tested how this structure is distributed within the adapter itself. The hypothesis is that if energy is concentrated in a few coordinates, it may be possible to construct a frozen mask that can be reused for different downstream tasks once.

Critical strengths

Figures 1 and 3 provide specific measurement results. In the three models, the top 10% of coordinates carry approximately 93%, 96%, and 90% of the defined safety membership, respectively. Figure 3 further distinguishes between coordinate concentration and spectral orientation concentration, illustrating that "concentration in a few parameters" does not necessarily equate to "concentration in a few directions."

The control experiments in Table 2 are more valuable than simply comparing the main table. The authors compare the same budget random freezing, reverse selection, and reduced LoRA rank. For example, on Llama-3.1-8B, the ASR of the random mask is 20.4%, while that of the oriented mask is 6.0%, indicating that the improvement does not entirely stem from reducing the number of trainable parameters.

Critical weaknesses

W1

The 10% freezing percentage is inconsistent with the actual method. The abstract states that one-tenth of the coordinates are fixed; however, Section 3 and Algorithm 1 freeze 90% of the coordinates when p=10, leaving only 10% for training. In Table 8, under the same light configuration, the ASR is 15.0% when freezing 10% (p=90) and 1.8% when freezing 90% (p=10). Therefore, the current behavioral results support the conclusion that "large-scale targeted freezing is effective," but do not yet support the conclusion that "fixing only the safe decile is sufficient."

W2

The correlation coefficient between importance score and membership in Figure 2 is 0.55–0.67, which does not necessarily mean selecting the same coordinates. HOWEVER, simple gradient/Fisher sorting, and its mask highly overlap, with some conditions yielding the same results. Fisher may even perform better with continued learning.

W3

Table 3 shows an average task performance drop of approximately 7–9 percentage points; some models also require reestimating curvature at each task boundary, taking about an hour. This limits the deployment narrative of "build once, user only needs to attach a mask".

Questions for authors

Q1

If only the first 10% of the safety coordinates defined in Figure 1 are frozen, what are the differences in safety and task capability? Why are different freezing ratios used for the summary and the algorithm?

Q2

Can the main table be rewritten using the formal behavior classifier, and the Pareto curves for safety and task utility reported?

Reviewer_tpBFOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a parameter-freezing idea to tackle harmful fine-tuning. The idea is extended from Wei el al. by defining the importance metrics with curvature of the harmful loss.

Critical strengths
  1. The studied problem is important.
  2. The experimental results are comprehensive and good.
Critical weaknesses
  1. The writing is very hard to read. While I am not 100% sure, it seems to be that the writing is done by AI. If that is true, I personally suggest not using such a tool for writing, as it is very hard to read.

  2. The novelty is quite limited. There are already quite a few literature explaining the same idea of parameters freezing. For example [1][2] is a direct application and [3][4] [5] follow the projection way, which is similar as well. The novelty of this paper to them are not clear, because i) safety neurons contraint in a small subset of neurons are not new finding, and this paper does not advance a deeper interpretation vision towards this finding, ii) there are several ways to identify the safety neurons, using harmful loss curvature to estimate is not obvious to me that this is better than the other methods.

[1] Superficial safety alignment hypothesis [2] Assessing the brittleness of safety alignment via pruning and low-rank modifications [3] AsFT: Anchoring Safety During LLM Fine-Tuning Within Narrow Safety Basin [4] Gradient Surgery for Safe LLM Fine-Tuning [5] Understanding and Preserving Safety in Fine-Tuned LLMs

  1. The definition of the equation is problematic. In the first equation, what is qi[c]^2 and what is m(c)?

Overall, it is hard to evaluate this paper as the notation does not make sense to me, as well as its core idea. For example, why it makes senses to use curvature to estimate the importance of the paparameter instead of the gradient norm? Why we need the second-order information and why first order information is not sufficient? I suggest that the authors rewrite the paper without AI assistance for the next submission.

Questions for authors
  1. What is the Lforget(θ) you are using. It is the harmful prompt-harmful response or the harmful prompt-safe response, or the harmful prmopts itself? The definition is so unclear.
Reviewer_DzKxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies how a curvature-based safety signal is distributed across LoRA coordinates. Across three models, the global top 10% of coordinates carries about 90–96% of the measured membership energy. The proposed defense uses a related importance score and, in its main setting, locks 90% of each adapter tensor. Experiments compare it with existing defenses and several targeting controls. The results support targeted masking as a useful way to reduce safety degradation during cooperative fine-tuning, with a cost in task learning.

Critical strengths
  1. The empirical study is substantial. It includes four base models, two harmful-prompt benchmarks, three-seed main comparisons, and random, inverted, reduced-rank, and layer-level controls. These controls provide useful evidence that coordinate selection matters beyond the number of trainable parameters.

  2. The paper reports practical costs and limitations clearly in the experiments. These include task loss, over-refusal, capability changes, and failures during continual learning. The classifier validation and the comparison with simpler gradient-based scores also make the results more informative.

Critical weaknesses
  1. The main claim does not match the main intervention. The abstract and Section 4.2 describe freezing the global safety decile, but Algorithm 1 and the main experiments lock 90% per tensor using a different score. The existing light-lock experiments do not directly test the global membership decile. Appendix B.2 does measure coverage, but neither these results nor the score correlations establish that freezing only the measured decile gives the reported protection. Also, set overlap does not force weighted coverage to be close. The coverage measurements do not rule out a nonlinear link between uncovered energy and behavior. The empirical masking contribution remains useful, but these claims need correction.

  2. Some details needed to reproduce and interpret the results remain unclear. The paper should specify the target tokens or completions used for the forget loss, the history/probe split, and the parameter state at which the probe gradient is evaluated. The classifier validation is useful, but its aggregate correlations do not show whether every close comparison in Table 1 holds within each model.

Questions for authors
  1. Can you clearly separate the global membership decile, the per-tensor importance ranking, and the setting that locks 90% of coordinates? Is there a direct experiment freezing only the global membership decile, or should the headline claim be narrowed to the tested masks?

  2. What target tokens or completions define the forget loss? How are the history and held-out probe examples separated, and at which adapter state is the probe gradient evaluated?

  3. Can you report the existing classifier-based ASR results for each model and method in Table 1, especially where methods have similar keyword-based scores?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.