acceptodds
Withdrawn from ICLR 2027

Restoring LLM Safety via Sparse Weight Interpolation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Supervised fine-tuning of aligned large language models often degrades safety guardrails, even on benign downstream tasks. We identify two distinct failure modes of existing safety-preservation strategies: isolating or restoring safety-critical parameters loses effectiveness under shifted activation contexts, while gradient-based constraints can be circumvented by approximately orthogonal task updates. To avoid such rigid constraints, we formulate safety restoration as a bounded interpolation problem and propose Sparse Weight Interpolation (SWI) to search for a parameter state that jointly recovers safety and preserves task utility. Guided by the aligned base model as a structural prior, SWI optimizes a row-level continuous mask to reactivate refusal behaviors within the fine-tuned parameter space. Empirical evaluations show that SWI restores defense generalization across diverse safety benchmarks with as few as 10 safety demonstrations, while maintaining benign capabilities across multiple model families. We further provide theoretical guarantees for recovery-state compatibility and finite-sample uniform generalization under explicit regularity assumptions.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Safety Alignment, Safety Degradation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Ux7AOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Sparse Weight Interpolation (SWI) to restore safety after downstream fine-tuning. SWI learns a continuous row-level mask that interpolates between an aligned base model and its fine-tuned counterpart. The optimization combines a safety loss, a utility-preservation loss, and sparsity regularization. The paper reports substantial safety recovery using a small number of demonstrations while largely preserving the evaluated downstream performance, with experiments covering multiple model families, tasks, and fine-tuning settings.

Critical strengths
  1. The paper addresses a meaningful problem: restoring safety after otherwise benign task adaptation. The proposed method is straightforward, and the learned mask can be merged into the model weights without additional inference overhead.

  2. The experimental coverage is broad, including multiple model families, both LoRA and full-parameter fine-tuning, and evaluations across different safety datasets. The ablations on the optimization objective, mask granularity, and structural prior help examine the design choices.

Critical weaknesses
  1. The reported safety metrics appear inconsistent with their definitions. Section 5.1 states that harmfulness scores range from 1 to 5 and that responses scoring above 3 count as successful attacks. If HS is the mean of the same integer ratings used to calculate ASR, an ASR of 6.2% implies a mean HS of at least (0.062$\times$4+0.938$\times$1=1.186). However, Table 1 reports HS = 1.06 for SWI on AGNews. A similar inconsistency occurs for the base model, with HS = 1.04 and ASR = 4.2%. These discrepancies cannot be explained by rounding. If different scoring or aggregation procedures were used, this should be made explicit. Otherwise, clarifying how HS and ASR are computed is essential before the main safety results can be assessed reliably.

  2. The empirical contribution relative to closely related mask-based safety recovery methods is not sufficiently established. Although the paper discusses SOMF and highlights differences in optimization objectives and data requirements, it does not provide a direct experimental comparison. If possible, a comparison with SOMF under comparable settings would further strengthen the empirical justification for the proposed method. More importantly, a simple interpolation baseline with a sweep over the mixing coefficient would also help establish the benefit of learning a sparse row-level mask beyond partially reverting fine-tuning.

Questions for authors
  1. Could the authors clarify whether HS and ASR in Table 1 are computed from the same responses and ratings? Please explain the numerical inconsistencies above and provide the exact aggregation procedure.

  2. I found that the implementation details differ between the paper and the supplied code. The paper specifies GPT-4o, an attack-success threshold of HS > 3, and 500 safety-test examples, whereas the supplied evaluation script uses GPT-4o-mini, HS ≥ 3, and a default subset of 50 examples. Moreover, Eq. (3) describes a loss on refusal sequences, while the supplied implementation computes the safety loss only over the first four response tokens. The scripts may represent a different configuration, but their relationship to the reported experiments needs clarification. Which evaluation and training configurations produced the reported results?

  3. Could the authors clarify the scope of safety restoration in light of Table 15? While standard SWI reduces the average jailbreak ASR from 90.50% to 73.50%, it remains substantially higher than that of the aligned base model. Although this limitation is acknowledged in the appendix, a clearer distinction between restoring safety on standard harmful prompts and recovering adversarial robustness would strengthen the main paper.

Reviewer_mBiuOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper shows that parameter freezing/restoration and gradient projection fail to preserve safety under benign fine-tuning. SWI learns a sparse, row-level mask that interpolates between the aligned and fine-tuned weights, using only 10 safety and 10 utility samples. Across three model families, SWI reduces ASR from about 80% to single digits while largely preserving task accuracy.

Critical strengths
  1. The method is simple, fast (about 12 s), data-efficient, and consistently effective across models, tasks, LoRA, and full fine-tuning.

  2. The mechanistic analysis is convincing: masks overlap heavily across harm categories, and targeted vs. random knockout (68.4% vs. 7.6% ASR) supports a shared refusal bottleneck.

Critical weaknesses
  1. Comparison to SOTA Baselines. The paper does not compare its performance with SOTA baselines [1, 2], so the performance advantage remains to be confirmed.

[1] EnchTable: Unified Safety Alignment Transfer in Fine-tuned Large Language Models (S&P'26)

[2] Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance (ICLR'26)

  1. The restoration appears shallow. Jailbreak ASR stays at 73.5%, versus 21.75% for the base model. The mask concentrates on the last-layer down_proj, which points to first-token refusal, and prefilling is not tested. The generalization bound is vacuous for n=10, and Assumption G.1 effectively assumes the existence result it is used to prove.
Questions for authors

See weakness.

Reviewer_7zRsOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Sparse Weight Interpolation (SWI) to restore safety alignment after benign downstream fine-tuning. SWI learns a continuous row-level mask that interpolates between the aligned and fine-tuned weights, using a safety loss, a utility-preservation loss, and sparsity regularization. Experiments report substantial safety recovery with ten safety demonstrations and ten benign examples while largely preserving fine-tuned task performance. The paper also investigates shared restoration masks and provides conditional theoretical analyses.

Critical strengths

1.The method is conceptually simple and produces merged model weights without additional inference computation. 2.The evaluation covers multiple model families, downstream tasks, both LoRA and full fine-tuning, and cross-dataset safety generalization. The OOD experiments include three independent demonstration-sampling runs. 3.Ablations examine the optimization losses, interpolation anchor, sample budget, and mask granularity. Targeted knockout and rescue experiments provide useful evidence about the functional importance of the identified parameters.

Critical weaknesses

1.The safety metrics appear inconsistent under the stated definitions. Section 5.1 defines HS on a scale of 1–5 and counts responses with HS above 3 as successful attacks. If the reported HS is averaged over the same responses, an ASR of 6.2% requires a mean HS of at least 1.124, even conservatively assigning successful responses a score of only 3. However, Table 1 reports 1.06 for SWI on AGNews. Similar inconsistencies occur in other rows. The metric definitions, aggregation procedure, or reported results require correction before the central safety findings can be assessed reliably. 2.Harmfulness and ASR evaluation rely on GPT-4o, without reported corroboration from an independent safety judge or human annotation. Consequently, it remains unclear whether the reported gains and method rankings are stable across evaluators.

Questions for authors

1.How does SWI compare with SOMF and direct fine-tuning using the same safety examples, benign examples, and utility-preservation loss?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.