acceptodds
Under review as a conference paper at ICLR 2027

Dual-Null LoRA: A Training Dynamics Approach to Preserving Safety Alignment

Abstract

The safety alignment of large language models, typically established as part of post-training pipelines, is known to degrade under subsequent fine-tuning. This phenomenon occurs not only when fine-tuning on harmful examples but also when using harmless, domain-specific utility data. We use recent developments in the theory of deep learning training dynamics to understand this phenomenon and derive a mitigation for the special case of LoRA fine-tuning. In a first-order approximation of fine-tuning dynamics, gradient steps on potentially harmful training examples change the probability of obtaining an aligned response to any other prompt. This change is mediated by the empirical neural tangent kernel (eNTK), which couples each training example to each observed prompt. We introduce dual-null LoRA, a standard LoRA adapter built to eliminate this coupling: Our adapter is initialized in approximate null spaces of both model activations and gradients, and every update during training is projected back into these null spaces. Evaluated on two base models and three fine-tuning datasets (SST-2, AG News, GSM8K), dual-null LoRA effectively keeps harmfulness (HS) and attack-success (ASR) at the levels of the pre-fine-tuning baselines without negatively impacting task learning or general capability, even at 20% harmful training examples.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
safety alignment preservation, (harmful) fine-tuning, LoRA, deep learning training dynamics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pgEVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses safety alignment degradation during LoRA fine-tuning. The authors extend the learning-dynamics framework of Ren & Sutherland (2025) to the LoRA parameterization: by differentiating the adapted layer f(x)=Wx+sBAxf(x)=Wx+sBAx with respect to AA and BB separately, the first-order alignment-score change decomposes into two channels: the B-channel (proportional to ⟨Axo,Axu⟩\langle Ax_o, Ax_u\rangle) and the A-channel (proportional to ⟨B⊤go,B⊤gu⟩\langle B^\top g_o, B^\top g_u\rangle). Setting the observation-side factor in each channel to zero yields the dual-null conditions: (DN1) Axo=0Ax_o=0 and (DN2) B⊤go=0B^\top g_o=0.

Based on this decomposition, the authors propose dual-null LoRA, a fine-tuning-stage defence in three phases: (1) a calibration phase collects per-layer activations and gradients from a small safety demonstration set, eigendecomposes their covariances, and extracts approximate null spaces within an energy budget bb; (2) the LoRA factors are initialized within these null spaces via truncated SVD of the doubly projected base weight; (3) during training, gradients of AA and BB are projected back onto the constrained subspaces at every step to maintain the dual-null conditions throughout fine-tuning.

Critical strengths
  1. Principled theoretical foundation. The core learning-dynamics framework is not original, but the specialization to LoRA is complete: the two-channel kernel decomposition yields a concrete design criterion, and the algorithm follows directly from it rather than from heuristic intuition. This is a notable contrast to most existing defences in this area.

  2. Thorough ablation. The ablation (Table 2) systematically isolates initialization, DN1 and DN2, which honestly exposes cases where components are insufficient. Under the authors' setting, dual-null LoRA achieves the best safety–helpfulness trade-off, though the experimental setup (2–3B models, SST-2/AG News/GSM8K, evaluation protocol) closely follows prior work from 2–3 years ago.

Critical weaknesses

Gap between theory and practice. The derivation in Sections 3-4 rests on a first-order Taylor expansion and a single gradient step. The subspaces used to enforce Axo≈0A x_o \approx 0 and B⊤go≈0B^\top g_o \approx 0 are calibrated once at initialization and then kept fixed. During multi-epoch training the parameters keep changing, so the activations xox_o and the pulled-back gradients gog_o drift, and the fixed subspaces may no longer annihilate them. The first-order invariance of the alignment score is therefore guaranteed only near initialization, and its validity over the full training trajectory is assumed rather than shown.

Questions for authors
  1. Direct validation of the mechanism. (a) The first-order term ΔS(χo)\Delta S(\chi_o) is never measured. Could the authors report the alignment score S(χo)S(\chi_o) before and after fine-tuning for plain LoRA and dual-null LoRA? (b) For the 256 calibration pairs (harmful prompts paired with aligned refusals from BeaverTails), does the average change ΔS\Delta S before and after fine-tuning directly track the change in model safety (HS and ASR)? That is, does a smaller ΔS\Delta S on these pairs consistently correspond to a smaller increase in HS and ASR?

  2. Model scale. The experiments use two small and relatively old models (2-3B). Could the authors report one set of results (plain LoRA vs. dual-null LoRA) on a newer, larger model (e.g., 7-8B)?

  3. Asymmetry of DN1 and DN2. In the ablation (Table 2), DN1 alone is nearly as protective as the full method at the protocol learning rate, while DN2 alone gives no protection. The benefit of adding DN2 appears only at the fivefold learning rate, and mainly in AdvBench ASR. Can the authors explain this asymmetry, theoretically or empirically?

4 comments
AuthorsOpenReview ↗

Response to Reviewer pgEV: Thanks and Notation

We sincerely thank the reviewer for the careful reading of our work and for suggesting concrete follow-up measurements. We hope our author rebuttal can address the reviewer's concerns and answer any outstanding questions.

Our evaluation metrics are the same as in our manuscript and are added here again for easier reading of our rebuttal:

SymbolMeaning
HS ↓Harmful responses on the BeaverTails test prompts (beaver-dam-7b)
ASR ↓Attack success rate on the same responses (HarmBench classifier, stricter than HS)
HS$_A$, ASR$_A$ ↓The same two scores on AdvBench
FA ↑Accuracy on the fine-tuning task
OR ↓Overrefusal, the refusal rate on safe XSTest prompts
GC ↑General capability, the accuracy on MMLU
pp, rr, bbFraction of harmful fine-tuning examples, LoRA rank, null-space budget

All scores in percent.

AuthorsOpenReview ↗

Author Rebuttal I to Reviewer pgEV

W1: Gap between theory and practice. The derivation in Sections 3-4 rests on a first-order Taylor expansion and a single gradient step. The subspaces used to enforce Axo=0A x_o = 0 and B⊤go=0B^{\top} g_o = 0 are calibrated once at initialization and then kept fixed. During multi-epoch training the parameters keep changing, so the activations xox_o and the pulled-back gradients gog_o drift, and the fixed subspaces may no longer annihilate them. The first-order invariance of the alignment score is therefore guaranteed only near initialization, and its validity over the full training trajectory is assumed rather than shown.

The drift exists but stays small under dual-null LoRA over the full training trajectory.

We measured after every epoch how far each condition is violated. With XoX_o and GoG_o the activations and pulled-back gradients of the 256 calibration pairs, recomputed at the current weights while the null spaces stay fixed, and every matrix scaled to unit Frobenius norm:

εDN1=∥A^X^o∥F,εDN2=∥B^⊤G^o∥F,withM^=M/∥M∥F.\varepsilon_{\mathrm{DN1}} = \big\lVert \hat A \hat X_o \big\rVert_F, \qquad \varepsilon_{\mathrm{DN2}} = \big\lVert \hat B^{\top} \hat G_o \big\rVert_F, \qquad \text{with} \quad \hat M = M / \lVert M \rVert_F .

Both leaks lie between 0 and 1 and are 0 if and only if the condition holds exactly on all calibration pairs. A growing leak is the drift described in W1. We report the median over the adapted modules.

Under dual-null LoRA both conditions stay close to satisfied: εDN1\varepsilon_{\mathrm{DN1}} grows from 0.003 to 0.005 and εDN2\varepsilon_{\mathrm{DN2}} from 0.002 to 0.005 over ten epochs, while plain LoRA reaches 0.092 and 0.038. Each single condition keeps only its own leak small (Table 1 of this response).

Table 1. Leaks after every epoch at the fivefold learning rate and b=10−3b = 10^{-3} (Qwen2.5-3B-Instruct, AG News, p=0.10p = 0.10). Epoch 0 is the initialization. For plain LoRA εDN2\varepsilon_{\mathrm{DN2}} is undefined at epoch 0, since B=0B = 0.

LeakConfiguration012345678910
εDN1\varepsilon_{\mathrm{DN1}} ↓Plain LoRA0.0220.0400.0450.0520.0580.0670.0760.0790.0840.0880.092
DN1 projection, DN1-only init0.0030.0030.0040.0050.0050.0060.0060.0060.0060.0060.006
DN2 projection, DN2-only init0.0360.0370.0370.0370.0370.0370.0370.0380.0380.0370.037
dual-null LoRA (ours)0.0030.0040.0040.0040.0040.0050.0050.0050.0050.0050.005
εDN2\varepsilon_{\mathrm{DN2}} ↓Plain LoRA0.0630.0530.0470.0450.0400.0410.0400.0390.0380.038
DN1 projection, DN1-only init0.0250.0250.0250.0270.0260.0260.0250.0250.0260.0250.026
DN2 projection, DN2-only init0.0020.0070.0080.0090.0090.0090.0100.0100.0100.0100.010
dual-null LoRA (ours)0.0020.0030.0030.0040.0040.0040.0040.0040.0050.0050.005

Recomputing the null spaces during training would most likely reduce the drift further, but is not necessary at this rank and budget.

Our measurement is complemented by Afzal et al. (2025), who provide theoretical and empirical support for linearized fine-tuning dynamics of LoRA in large language models under suitable conditions.

These results are documented in the new Appendix H of the revised manuscript.

AuthorsOpenReview ↗

Author Rebuttal II to Reviewer pgEV

Q1: Direct validation of the mechanism. (a) The first-order term is never measured. Could the authors report the alignment score SS before and after fine-tuning for plain LoRA and dual-null LoRA? (b) For the 256 calibration pairs (harmful prompts paired with aligned refusals from BeaverTails), does the average change ΔS\Delta S before and after fine-tuning directly track the change in model safety (HS and ASR)? That is, does a smaller ΔS\Delta S on these pairs consistently correspond to a smaller increase in HS and ASR?

Under dual-null LoRA the alignment score SS tracks its initial value closely, and the changes of HS and ASR follow the change of SS.

(a) The first-order term is never measured. Could the authors report the alignment score SS before and after fine-tuning for plain LoRA and dual-null LoRA?

S(xo)S(x_o) is the probability of a safe response to the harmful prompt xox_o, between 0 and 1. We estimate it as the share of safe responses among four sampled responses and report SS, its average over the prompts of a set. For dual-null LoRA, SS changes by −0.016-0.016 on the calibration prompts and by +0.011+0.011 on held-out harmful prompts. For plain LoRA it changes by −0.357-0.357 and −0.513-0.513.

(b) For the 256 calibration pairs (harmful prompts paired with aligned refusals from BeaverTails), does the average change ΔS\Delta S before and after fine-tuning directly track the change in model safety (HS and ASR)? That is, does a smaller ΔS\Delta S on these pairs consistently correspond to a smaller increase in HS and ASR?

Yes. Across the four configurations a smaller ∣ΔS∣\lvert \Delta S \rvert on the calibration pairs corresponds to a smaller increase in HS and ASR. The same holds on the held-out prompts.

Table 2. SS after fine-tuning and its change ΔS\Delta S against the released model, HS and ASR (setting of Table 1 of this response).

ConfigurationSS (calibration) ↑ΔS\Delta S (calibration)SS (held-out) ↑ΔS\Delta S (held-out)HS ↓ASR ↓
Before FT0.7660.61135.45.1
Plain LoRA0.409−0.357-0.3570.098−0.513-0.51391.464.7
DN1 projection, DN1-only init0.435−0.331-0.3310.158−0.453-0.45389.165.6
DN2 projection, DN2-only init0.447−0.319-0.3190.148−0.463-0.46389.665.1
dual-null LoRA (ours)0.750−0.016-0.0160.622+0.011+0.01137.64.6

Q2: Model scale. The experiments use two small and relatively old models (2-3B). Could the authors report one set of results (plain LoRA vs. dual-null LoRA) on a newer, larger model (e.g., 7-8B)?

The default setting transfers to an 8B model without retuning, at a cost of 7 points of task accuracy. We ran a pilot evaluation on Qwen3-8B with GSM8K at p=0.20p = 0.20. Dual-null LoRA keeps HS at the level of the released model (33.6 against 34.8), while plain LoRA rises to 43.2. These results are documented in the new Appendix F.2 of the revised manuscript.

Table 3. Qwen3-8B on GSM8K at p=0.20p = 0.20.

MethodHS ↓ASR ↓FA ↑
Before FT34.88.131.5
Plain LoRA43.211.764.0
dual-null LoRA (ours)33.68.156.8

Further tuning of the budget would likely allow for an even better trade-off between protection and task learning at larger model sizes.

AuthorsOpenReview ↗

Author Rebuttal III to Reviewer pgEV

Q3: Asymmetry of DN1 and DN2. In the ablation (Table 2), DN1 alone is nearly as protective as the full method at the protocol learning rate, while DN2 alone gives no protection. The benefit of adding DN2 appears only at the fivefold learning rate, and mainly in AdvBench ASR. Can the authors explain this asymmetry, theoretically or empirically?

[...] DN1 alone is nearly as protective as the full method at the protocol learning rate, while DN2 alone gives no protection. [...] Can the authors explain this asymmetry, theoretically or empirically?

A theoretical explanation is that DN1 preserves itself over training while DN2 does not.

1. DN1 preserves itself. Suppose DN1 holds exactly at step tt, that is Atxo=0A_t x_o = 0 for every calibration activation xox_o. The adapter output s BtAt xos \, B_t A_t \, x_o is then zero, so the model computes on the calibration inputs what the released model computes, and xox_o does not change. The update is restricted to the same null space, ΔAt xo=0\Delta A_t \, x_o = 0, hence At+1xo=(At+ΔAt) xo=0A_{t+1} x_o = (A_t + \Delta A_t) \, x_o = 0. By induction DN1 holds at every step.

2. DN2 does not. The update likewise keeps Bt+1⊤go=0B_{t+1}^{\top} g_o = 0 for the same gog_o. The pulled-back gradient gog_o, however, depends on the current prediction at xox_o (Equation 9 of the revised manuscript). Under DN2 alone the adapter output on xox_o is nonzero, the prediction changes, and gog_o changes with it. The induction step fails.

This argument is idealized and holds exactly only under exact null spaces. Empirically the leaks behave as predicted by this argument (Table 1 of this response in the comment above): Under DN2 alone, εDN2\varepsilon_{\mathrm{DN2}} grows fivefold over training (0.002 to 0.010). Under DN1 alone, εDN1\varepsilon_{\mathrm{DN1}} grows less (0.003 to 0.006).

[...] The benefit of adding DN2 appears only at the fivefold learning rate, and mainly in AdvBench ASR. [...]

With a tighter budget the benefit of DN2 is larger and appears in all harmfulness measures. We ran the setting that Reviewer YRib suggested, the fivefold learning rate with b=10−3b = 10^{-3}. There dual-null LoRA still protects the model (HS 37.6) while the DN1-only configuration does not (HS 89.1).

Table 4. Ablation at the fivefold learning rate and b=10−3b = 10^{-3} (Qwen2.5-3B-Instruct, AG News, p=0.10p = 0.10).

ConfigurationHS ↓ASR ↓HS$_A$ ↓ASR$_A$ ↓FA ↑OR ↓GC ↑
Before FT35.45.134.20.476.414.865.4
Plain LoRA91.464.780.257.787.42.865.0
+ dual-null init87.864.178.761.785.70.465.2
+ DN2 projection89.362.374.857.385.60.465.1
+ DN1 projection35.65.032.90.284.815.265.4
+ both projections = dual-null LoRA (ours)37.64.635.40.283.616.465.4
DN1 projection, DN1-only init89.165.682.778.787.00.864.9
DN2 projection, DN2-only init89.665.167.341.785.52.465.0

We have added these results to the ablation in the main text (Section 5.3, Table 2 of the revised manuscript).

We are happy to clarify any remaining points.

References:

Zahra Rahimi Afzal, Tara Esmaeilbeig, Mojtaba Soltanalian, and Mesrob I Ohannessian. Linearization explains fine-tuning in large language models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025.

Reviewer_khSXOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

the constraint is derived, not guessed, and the ablation cleanly shows that protection comes from enforcing the projection during training rather than from initialization, and that DN2 matters under escalated learning rates. The evaluation is thorough (two models, three tasks, five baselines, five poison levels, overrefusal, MMLU, loss trajectories) and the limitations section is candid, including the failure at 5× learning rate and norm-growth escape. My concerns are generality and threat model: only 2–3B models, rank 8, a single provider-controlled/benign-actor setting, and the defense is defined only for LoRA, so an attacker who switches to full fine-tuning escapes by construction. Novelty over LoRA-Null and GuardSpace is real but incremental.

Critical strengths

(1) The dynamically derived dual constraints constitute a key contribution. Rather than being yet another null-space technique, the first-order expansion pinpoints the A-channel missed by LoRA-Null, and elevated-learning-rate ablations validate the indispensability of the second constraint, underpinning the method’s core claim. (2) The protection–utility tradeoff is compelling. At 20% poison rate, it preserves pre-fine-tuning HS/ASR with negligible degradation to task accuracy, overrefusal and MMLU. Baselines either struggle with learning (AsFT) or fail to secure the model (SaLoRA, SafeInstr), strongly supporting its practical viability.

Critical weaknesses

(1) The threat model is restrictive. The defense is LoRA-specific and assumes training is controlled by a benign actor or provider. Full fine-tuning enables complete bypass, and the absence of such evaluations undermines its value as a general defense. (2) It suffers from scaling and hyperparameter fragility. Results are only validated on 2–3B models with rank 8. Budget, rank and learning rate are strongly coupled; failure at 5× learning rate plus the need for budget recalibration indicates robustness stems mainly from calibration instead of the core mechanism. Behavior on 7B+ models is unexamined

Questions for authors

Clarification requests: (1) Is dual-null still effective against full fine-tuning or higher-rank attacks? Please provide corresponding results or clarify its inapplicability under this threat model. (2) How sensitive is budget b jointly to rank and learning rate? Can we use fixed defaults without retuning? (3) What is the performance when calibration data is out-of-distribution relative to AdvBench, e.g., multilingual or jailbreak samples?

Future work: (4) Periodically re-estimate the null space every n steps to reduce staleness. (5) Explore ways to generalize dual-null to full fine-tuning.

6 comments
AuthorsOpenReview ↗

Response to Reviewer khSX: Thanks and Notation

Author Rebuttal to Reviewer khSX

We sincerely thank the reviewer for the thorough and constructive review of our work, for the encouraging assessment and for suggesting concrete follow-up experiments. We hope our author rebuttal can address the reviewer's concerns and answer any outstanding questions.

Our evaluation metrics are the same as in our manuscript and are added here again for easier reading of our rebuttal:

SymbolMeaning
HS ↓Harmful responses on the BeaverTails test prompts (beaver-dam-7b)
ASR ↓Attack success rate on the same responses (HarmBench classifier, stricter than HS)
HS$_A$, ASR$_A$ ↓The same two scores on AdvBench
FA ↑Accuracy on the fine-tuning task
OR ↓Overrefusal, the refusal rate on safe XSTest prompts
GC ↑General capability, the accuracy on MMLU
pp, rr, bbFraction of harmful fine-tuning examples, LoRA rank, null-space budget

All scores in percent.

AuthorsOpenReview ↗

Author Rebuttal I to Reviewer khSX

W1: (1) The threat model is restrictive. The defense is LoRA-specific and assumes training is controlled by a benign actor or provider. Full fine-tuning enables complete bypass, and the absence of such evaluations undermines its value as a general defense.

We fully agree that our threat model is limited to LoRA fine-tuning and now state this explicitly in the threat model paragraph of Section 1 of the revised manuscript. In practice the FTaaS threat model is highly relevant: Model providers routinely offer fine-tuning as a service and control the training procedure in such settings. For example, Tinker by Thinking Machines implements LoRA fine-tuning specifically (documentation). Beyond FTaaS, LoRA is widely adopted for its low computational cost, so the method equally serves practitioners who fine-tune models themselves and wish to preserve safety alignment.

Q1: (1) Is dual-null still effective against full fine-tuning or higher-rank attacks? Please provide corresponding results or clarify its inapplicability under this threat model.

Is dual-null still effective against full fine-tuning [...]

Our method does not extend to full fine-tuning in a straightforward way, since it derives its constraints from the LoRA parametrization.

There is, however, research on the connection between full fine-tuning and high-rank LoRA. In terms of expressive power, a LoRA rank of half the embedding size suffices to represent any target Transformer of the same size (Zeng and Lee, 2024). The two are nevertheless not equivalent: Even at equal task accuracy, LoRA and full fine-tuning converge to different solutions, since the two parametrizations follow different optimization trajectories (Shuttleworth et al., 2025). At high ranks LoRA can thus represent the updates of full fine-tuning, although it reaches them along a different optimization trajectory.

This observation makes the second part of the reviewer's question particularly interesting:

[...] or higher-rank attacks?

The dual-null conditions, Axo=0A x_o = 0 (DN1) and B⊤go=0B^{\top} g_o = 0 (DN2), do not depend on the rank. With exact null spaces our projections maintain both exactly at every step, for any rank. In practice the null spaces are approximate, and a higher rank may need a tighter budget.

We consider extension to full fine-tuning as a very interesting possibility for our future work, for which inspiration can be drawn from this work and from learning dynamics in general.

AuthorsOpenReview ↗

Author Rebuttal II to Reviewer khSX

W2: (2) It suffers from scaling and hyperparameter fragility. Results are only validated on 2–3B models with rank 8. Budget, rank and learning rate are strongly coupled; failure at 5× learning rate plus the need for budget recalibration indicates robustness stems mainly from calibration instead of the core mechanism. Behavior on 7B+ models is unexamined

[...] Results are only validated on 2–3B models with rank 8. [...] Behavior on 7B+ models is unexamined

The default setting transfers to an 8B model without retuning, at a cost of 7 points of task accuracy. We ran a pilot evaluation on Qwen3-8B with GSM8K at p=0.20p = 0.20. Dual-null LoRA keeps HS at the level of the released model (33.6 against 34.8), while plain LoRA rises to 43.2. These results are documented in the new Appendix F.2 of the revised manuscript.

Table 1. Qwen3-8B on GSM8K at p=0.20p = 0.20.

MethodHS ↓ASR ↓FA ↑
Before FT34.88.131.5
Plain LoRA43.211.764.0
dual-null LoRA (ours)33.68.156.8

Further tuning of the budget would likely allow for an even better trade-off between protection and task learning at larger model sizes.

[...] Budget, rank and learning rate are strongly coupled; failure at 5× learning rate plus the need for budget recalibration indicates robustness stems mainly from calibration instead of the core mechanism. [...]

The budget bb is a tunable hyperparameter that controls the trade-off between task learning and protection: A tight budget offers more protection due to stronger regularization but leaves less room for the model to learn. Conversely, a loose budget leaves more room to learn but admits more erosion. We would like to note that the preferences of users applying our method can differ here: Some users may put more weight on safety, others on task learning. Through bb they can tune the behaviour of our method to their needs. As a heuristic, a suitable budget shrinks with rank rr and learning rate η\eta, between b∼1/(rη)b \sim 1 / (r \eta) and b∼1/(rη2)b \sim 1 / (r \eta^{2}). Practitioners could also tune the hyperparameter using a validation set, as is common practice with other hyperparameters in deep learning.

We would like to mention that hyperparameters of this kind are common among fine-tuning defences, including our baselines. Those that analyse them report a similar trade-off:

  • SafeInstr (Bianchi et al., 2024): The number of safety demonstrations added to the fine-tuning data. More demonstrations lower harmfulness but raise overrefusal ("Finding the right balance can be difficult").
  • Safe LoRA (Hsu et al., 2024): The similarity threshold that decides which layers are projected. Projecting more layers lowers both harmfulness and task utility.
  • AsFT (Yang et al., 2025b): The weight of its regularization term. A larger weight lowers HS and, beyond a value of 10, also task accuracy.

These analyses are run at a fixed rank, and for SafeInstr and Safe LoRA also at a fixed learning rate, which is common practice. While this does not affect the main conclusions, we see value in going one step further and examine how a suitable budget shifts with rank and learning rate jointly (Table 2 of this response in the next comment).

Reviewer_YRibOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary
  • LoRA fine-tuning can undermine safety alignment. Even a small number of harmful training examples can cause a model to stop refusing harmful requests, and benign data alone can also weaken its refusals. The paper considers both benign users who want to preserve safety while fine-tuning and fine-tuning-as-a-service providers whose customers may upload harmful data.

  • A learning-dynamics analysis shows that LoRA training affects safety through two channels, one per low-rank factor, A and B.

  • The proposed dual-null LoRA closes each channel with one constraint, defined on a small calibration set of safety demonstrations, each pairing a harmful prompt with a refusal. DN1 constrains A, which reads the layer input, so that the adapter's output on these demonstrations is approximately zero. DN2 constrains B, which writes the layer output, so that what the adapter writes barely changes how safely the model answers them.

  • The method restricts A and B to approximate null spaces, directions in which updates barely change the model's behavior on the calibration set. It computes these null spaces once before training from the calibration set, 256 BeaverTails demonstrations, and keeps the adapter inside them from initialization to the end of training. A budget b, 0.05 by default, controls how strictly the null spaces are chosen; smaller b is stricter.

  • The experiments fine-tune Qwen2.5-3B-Instruct and Gemma 2 IT 2B on SST-2, AG News, and GSM8K. The paper reports that harmfulness, the share of harmful responses to BeaverTails test prompts, stays at its level before fine-tuning even with 20% harmful examples. Task accuracy is largely preserved, and harmfulness is lower than under each of the five existing defenses.

  • Table 2 shows that initializing the adapter in the null spaces without constraining training fails to protect the model, while enforcing DN1 during training protects it under default training.

Critical strengths

The paper clearly explains the problem and why it matters, and the design is well motivated.

  • The learning-dynamics analysis identifies two channels, and each constraint is designed to close one of them, as Eq. 10 and Figure 1 show.
  • The decomposition also offers a clean way to think about why fine-tuning weakens safety, which may be useful beyond this method.

The method preserves safety with little loss of utility, even under the strongest attack tested.

  • In Table 1, with 20% harmful examples, task-averaged harmfulness and MMLU accuracy stay within one point of their values before fine-tuning.
  • Figure 3 suggests why. Qwen fine-tuned on AG News with 10% harmful examples fits the task data but barely fits the harmful examples.

The method's computational overhead is small relative to standard LoRA.

  • Before training, it runs one pass over the calibration set to compute the null spaces. During training, each step needs only a few extra matrix multiplications to keep the adapter inside them. After training, the adapter can be merged into the model, so inference costs nothing extra. Section 4 describes these steps.
  • Because the method leaves the training data and objective unchanged, it can also in principle be combined with other defenses, as Section 2 notes.
Critical weaknesses

The case for the added DN2 is not yet convincing.

  • DN2 is one of the method's additions over LoRA-Null, an earlier method that imposes a DN1-like constraint only at initialization.
  • In Table 2, at the default learning rate, the DN1-only configuration, which removes DN2 from both initialization and training, reaches a harmfulness of 34.8, nearly the same as the 34.4 of dual-null LoRA.
  • At the fivefold learning rate, where the paper argues DN2 is needed, the method fails with or without DN2. DN2 in the initialization lowers harmfulness from 91.3 to 74.9, and the DN2 projection lowers AdvBench attack success from 20.8 to 11.7, but both stay far above their values before fine-tuning, 35.4 and 0.4. This setting therefore shows only that DN2 partly helps where the method already fails, not that it is needed where the method works.

Under stronger training, the default setting no longer protects the model, and a budget that works in one setting can fail in another.

  • In Table 2, at the fivefold learning rate, harmfulness rises from 35.4 before fine-tuning to 76.4. Table 8 in Appendix E shows that raising the rank to 64 or above also pushes AG News harmfulness to 87.6–90.2.
  • Table 9 shows that a good budget can depend on the learning rate. On AG News, b = 1e-3 protects the model at both rates while keeping task accuracy at or above that of the default setting, but on GSM8K no tested budget does both. There, b = 1e-3 protects the model at the fivefold rate with a harmfulness of 35.4 and an accuracy of 65.0, but its accuracy drops to 30.9 at the default rate, while the default b = 0.05 shows the reverse pattern.
  • The paper acknowledges that the right b likely differs across settings but leaves the choice of b to the user. Since a smaller b can reduce task accuracy, as on GSM8K, guidance or a heuristic for choosing b would make the method easier to apply.
Questions for authors
  • Can you compare dual-null LoRA with the DN1-only configuration in a more difficult setting, such as b = 1e-3 at the fivefold learning rate, where dual-null LoRA still protects the model?

  • GuardSpace and the DN1-only configuration both leave B unconstrained. Yet, on Qwen with AG News and 10% harmful examples, GuardSpace reaches a BeaverTails harmfulness of 50.3, compared to 34.8 for the DN1-only configuration. On AdvBench, however, the two are similar. What explains this difference, and how much of it might be due to running GuardSpace with a smaller rank and a different calibration set than in its original paper?

  • Does the protection hold with more training data? The paper attributes lost protection to null spaces that become outdated during training. However, all experiments use only 1,000 training examples. Could you report results using 500 to 2,500 training examples for plain LoRA without any defense and dual-null LoRA on one model and task? This would follow the Booster protocol of Huang et al. (2025), in which the harmfulness of undefended fine-tuning rises sharply with the number of examples.

References

  • Huang, Tiansheng, et al. "Booster: Tackling harmful fine-tuning for large language models via attenuating harmful perturbation." International Conference on Learning Representations. Vol. 2025. 2025.
5 comments
AuthorsOpenReview ↗

Response to Reviewer YRib: Thanks and Notation

We sincerely thank the reviewer for the careful reading of our work and for suggesting concrete follow-up experiments. We hope our author rebuttal can address the reviewer's concerns and answer any outstanding questions.

Our evaluation metrics are the same as in our manuscript and are added here again for easier reading of our rebuttal:

SymbolMeaning
HS ↓Harmful responses on the BeaverTails test prompts (beaver-dam-7b)
ASR ↓Attack success rate on the same responses (HarmBench classifier, stricter than HS)
HS$_A$, ASR$_A$ ↓The same two scores on AdvBench
FA ↑Accuracy on the fine-tuning task
OR ↓Overrefusal, the refusal rate on safe XSTest prompts
GC ↑General capability, the accuracy on MMLU
pp, rr, bbFraction of harmful fine-tuning examples, LoRA rank, null-space budget

All scores in percent.

AuthorsOpenReview ↗

Author Rebuttal I to Reviewer YRib

W1: The case for the added DN2 is not yet convincing.

  • DN2 is one of the method's additions over LoRA-Null, an earlier method that imposes a DN1-like constraint only at initialization.
  • In Table 2, at the default learning rate, the DN1-only configuration, which removes DN2 from both initialization and training, reaches a harmfulness of 34.8, nearly the same as the 34.4 of dual-null LoRA.
  • At the fivefold learning rate, where the paper argues DN2 is needed, the method fails with or without DN2. DN2 in the initialization lowers harmfulness from 91.3 to 74.9, and the DN2 projection lowers AdvBench attack success from 20.8 to 11.7, but both stay far above their values before fine-tuning, 35.4 and 0.4. This setting therefore shows only that DN2 partly helps where the method already fails, not that it is needed where the method works.

DN2 is one of the method's additions over LoRA-Null, an earlier method that imposes a DN1-like constraint only at initialization.

As the reviewer correctly notes, LoRA-Null (Tang et al., 2026) imposes a DN1-like constraint only at initialization. We distinguish three methods:

  • LoRA-Null: A DN1-like condition at initialization only. Training is unconstrained.
  • DN1-only configuration (ours): DN1 at initialization and at every training step through the projection. No DN2.
  • dual-null LoRA (ours): DN1 and DN2 at initialization and at every training step.

As the results in Table 2 of our manuscript show, initialization alone, and therefore a method that constrains only the initialization, is not sufficient (protocol learning rate, b=0.05b = 0.05):

ConfigurationHS ↓ASR ↓FA ↑
Before FT35.45.176.4
Plain LoRA42.59.779.2
+ dual-null init62.520.081.8
+ DN1 projection34.45.478.6

In Table 2, at the default learning rate, the DN1-only configuration, which removes DN2 from both initialization and training, reaches a harmfulness of 34.8, nearly the same as the 34.4 of dual-null LoRA.

At the fivefold learning rate, where the paper argues DN2 is needed, the method fails with or without DN2. DN2 in the initialization lowers harmfulness from 91.3 to 74.9, and the DN2 projection lowers AdvBench attack success from 20.8 to 11.7, but both stay far above their values before fine-tuning, 35.4 and 0.4. This setting therefore shows only that DN2 partly helps where the method already fails, not that it is needed where the method works.

The setting suggested in Q1 below keeps the fivefold learning rate of the setting quoted above and only tightens the budget from b=0.05b = 0.05 to b=10−3b = 10^{-3}. It is therefore the same stronger fine-tuning, but now a case where the method works (HS 37.6 against 35.4 before fine-tuning, see Table 1 below for complete results), which is the case the concern asks for. DN2 is needed there: The DN1-only configuration, which removes DN2 from both initialization and training, reaches HS 89.1. The ablation attributes this to DN2 in the initialization, since the DN1 projection with the dual-null initialization already reaches HS 35.6.

Q1: Can you compare dual-null LoRA with the DN1-only configuration in a more difficult setting, such as b = 1e-3 at the fivefold learning rate, where dual-null LoRA still protects the model?

We ran precisely the experiment suggested in Q1. At the fivefold learning rate and using b=10−3b = 10^{-3}, dual-null LoRA still protects the model (HS 37.6) while the DN1-only configuration does not (HS 89.1). We have added these results to the ablation in the main text (Section 5.3, Table 2 of the revised manuscript).

Table 1. Ablation at the fivefold learning rate and b=10−3b = 10^{-3} (Qwen2.5-3B-Instruct, AG News, p=0.10p = 0.10).

ConfigurationHS ↓ASR ↓HS$_A$ ↓ASR$_A$ ↓FA ↑OR ↓GC ↑
Before FT35.45.134.20.476.414.865.4
Plain LoRA91.464.780.257.787.42.865.0
+ dual-null init87.864.178.761.785.70.465.2
+ DN2 projection89.362.374.857.385.60.465.1
+ DN1 projection35.65.032.90.284.815.265.4
+ both projections = dual-null LoRA (ours)37.64.635.40.283.616.465.4
DN1 projection, DN1-only init89.165.682.778.787.00.864.9
DN2 projection, DN2-only init89.665.167.341.785.52.465.0

The choice of b=10−3b = 10^{-3} corresponds to a more conservative safety budget, which we already evaluate in parts in Appendix E of the revised manuscript. It is a reasonable choice under stronger fine-tuning such as training with higher learning rates.

AuthorsOpenReview ↗

Author Rebuttal II to Reviewer YRib

W2: Under stronger training, the default setting no longer protects the model, and a budget that works in one setting can fail in another.

  • In Table 2, at the fivefold learning rate, harmfulness rises from 35.4 before fine-tuning to 76.4. Table 8 in Appendix E shows that raising the rank to 64 or above also pushes AG News harmfulness to 87.6–90.2.
  • Table 9 shows that a good budget can depend on the learning rate. On AG News, b = 1e-3 protects the model at both rates while keeping task accuracy at or above that of the default setting, but on GSM8K no tested budget does both. There, b = 1e-3 protects the model at the fivefold rate with a harmfulness of 35.4 and an accuracy of 65.0, but its accuracy drops to 30.9 at the default rate, while the default b = 0.05 shows the reverse pattern.
  • The paper acknowledges that the right b likely differs across settings but leaves the choice of b to the user. Since a smaller b can reduce task accuracy, as on GSM8K, guidance or a heuristic for choosing b would make the method easier to apply.

The budget bb is a tunable hyperparameter that controls the trade-off between task learning and protection: A tight budget offers more protection due to stronger regularization but leaves less room for the model to learn. Conversely, a loose budget leaves more room to learn but admits more erosion. We would like to note that the preferences of users applying our method can differ here: Some users may put more weight on safety, others on task learning. Through bb they can tune the behaviour of our method to their needs. As a heuristic, a suitable budget shrinks with rank rr and learning rate η\eta, between b∼1/(rη)b \sim 1 / (r \eta) and b∼1/(rη2)b \sim 1 / (r \eta^{2}). Practitioners could also tune the hyperparameter using a validation set, as is common practice with other hyperparameters in deep learning.

Hyperparameters of this kind are common among fine-tuning defences, including our baselines. Those that analyse them report a similar trade-off:

  • SafeInstr (Bianchi et al., 2024): The number of safety demonstrations added to the fine-tuning data. More demonstrations lower harmfulness but raise overrefusal ("Finding the right balance can be difficult").
  • Safe LoRA (Hsu et al., 2024): The similarity threshold that decides which layers are projected. Projecting more layers lowers both harmfulness and task utility.
  • AsFT (Yang et al., 2025b): The weight of its regularization term. A larger weight lowers HS and, beyond a value of 10, also task accuracy.

These analyses are run at a fixed rank, and for SafeInstr and Safe LoRA also at a fixed learning rate, which is common practice. While this does not affect the main conclusions, we see value in going one step further and examine how a suitable budget shifts with rank and learning rate jointly (Table 2 of this response).

We agree that guidance on choosing bb is important. We observe that a range of budgets both protects and learns in every setting tested, and that this range moves to tighter budgets as rank and learning rate grow. We have added this discussion to the main text (Section 5.3, paragraph "Choosing the budget", Figure 4) and document the full results in the new Appendix E.3 of the revised manuscript.

Table 2. HS / FA of dual-null LoRA (Qwen2.5-3B-Instruct, GSM8K, p=0.20p = 0.20, before fine-tuning 35.4 / 25.6). ✅: Protects and learns, that is at least 50% of the FA gain and at most 10% of the HS increase of plain LoRA at the same rank and learning rate.

Budget bbr=8r=8, lr 2×10−52 \times 10^{-5}r=16r=16, lr 2×10−52 \times 10^{-5}r=32r=32, lr 2×10−52 \times 10^{-5}r=8r=8, lr 10−410^{-4}r=16r=16, lr 10−410^{-4}r=32r=32, lr 10−410^{-4}
0.000135.1 / 25.634.6 / 25.136.0 / 25.4✅ 37.5 / 57.0✅ 34.8 / 63.2✅ 36.6 / 63.0
0.00134.8 / 30.934.5 / 32.4✅ 37.2 / 54.0✅ 35.4 / 65.0✅ 34.4 / 64.263.7 / 62.0
0.0134.5 / 43.6✅ 34.9 / 62.1✅ 34.4 / 64.563.1 / 63.781.2 / 61.086.8 / 56.3
0.05✅ 33.5 / 62.0✅ 35.5 / 61.5✅ 36.4 / 61.662.5 / 61.984.5 / 59.588.3 / 55.6
0.1✅ 36.2 / 63.362.9 / 63.874.9 / 64.561.0 / 60.990.4 / 53.189.2 / 56.6
0.255.5 / 62.661.4 / 64.080.1 / 62.178.7 / 59.790.9 / 55.590.0 / 55.6

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author Summary of the Discussion Period

Dear Area Chair,
Dear Reviewers,

We want to thank you again for handling our submission and the constructive reviews. Initial reviews were encouraging, yet asked for clarification and various valuable additional experiments.

We summarize reviews and our response to reviewer concerns and questions below.

Reviewer YRib: The reviewer acknowledged the practical relevance of our method and highlighted that it preserves safety with little loss of utility at small computational overhead. Their main concerns were the case for the second constraint (DN2) and the choice of the budget under stronger training.

→ We ran the experiment the reviewer suggested, in which dual-null LoRA still protects the model while the DN1-only configuration does not (HS 37.6 against 89.1, Section 5.3, Table 2). We added a heuristic for choosing the budget, backed by a joint grid over budget, rank and learning rate (Section 5.3, Figure 4, Appendix E.3). We further showed that the protection is robust to the amount of training data (Appendix F.1) and argued, with an additional run, that the initialization of GuardSpace explains its gap to our DN1-only configuration (Table 3 of our response).

Reviewer khSX: The reviewer considered the derived dual constraints a key contribution and the trade-off between protection and utility compelling. Their concerns focused on the LoRA-specific threat model, scaling and hyperparameter sensitivity. They were also curious about robustness of our method to various out-of-distribution attacks, larger models, as well as high rank attacks and full fine-tuning.

→ We clarify that our threat model is specific to LoRA (Section 1). We discuss generalisation to higher-rank attacks and how it relates to extending our method to full fine-tuning, which we leave to future work (Appendices E.1 and G, Section 6). We note that hyperparameters are common among fine-tuning defences and extend the sensitivity analysis of the budget (Section 5.3, Appendix E.3). Pilot experiments indicate adequate robustness to jailbreak and multilingual prompts (Appendix F.3) and to model size (Appendix F.2). A further pilot indicates the potential of periodic recalibration (Appendix G).

Reviewer pgEV: The reviewer valued the principled theoretical foundation and the thorough ablation. Their main concern was the gap between the first-order theory and training over multiple epochs, along with model scale and the asymmetry of DN1 and DN2.

→ We measured after every epoch to what extent both dual-null conditions are violated and found that the drift exists (as expected) but stays small (Appendix H). We reported the alignment score S, which tracks its initial value closely under dual-null LoRA (Appendix H, Table 15). We added the pilot evaluation on an 8B model (Appendix F.2) and explained the asymmetry: DN1 preserves itself over training while DN2 does not (Section 5.3).

We believe that the discussion process has allowed us to strengthen the paper and that any valid concerns have been adequately addressed.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.