- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates the parameter change induced by safety tuning and introduces the concept of “functional core”. Specifically, the authors decompose each updated parameter matrix into rank-one components (dyads) and use a small set of leading components to construct a low-dimensional subspace. Through paired retain/remove interventions, they show that this subspace captures only a modest fraction of the update energy while carrying most of the learned safety gain. The paper further examines how the functional core emerges and evolves during safety tuning and analyzes the opposing displacement accumulated within the core during downstream fine-tuning. It then uses the discovered core as a correction space, where a current safety-memory loss guides minimal adjustments to downstream updates.
Critical strengths
The paper addresses a practically relevant and important problem. It is well organized and clearly written. It examines how parameter updates induced by safety tuning carry safety behavior and how downstream fine-tuning erodes this behavior. It also explores how the identified structure can be used to mitigate safety degradation.
Critical weaknesses
- The paper shows that parameter updates induced by safety tuning are “spectrally broad but behaviorally compressible.” However, it remains unclear whether this functional compression is specific to safety updates. The paper validates this phenomenon across different safety-training objectives but does not perform the same spectral decomposition and retain/remove interventions on ordinary task fine-tuning updates. Without such comparisons, it is difficult to determine whether the identified structure is specific to safety learning or reflects a more general property of neural network fine-tuning.
- The scope of evaluation remains relatively limited. The main safety results are based primarily on AdvBench prompts, while downstream utility is measured mainly by the token-averaged NLL of Dolly responses. Although these metrics support the paper’s main analysis, they do not fully demonstrate the method’s generalization to other types of attacks or downstream capabilities. The evaluation would be strengthened by including additional safety datasets or jailbreak settings, as well as assessments of general knowledge, reasoning, and code generation.
- Although the proposed method mitigates the safety degradation caused by fine-tuning while preserving most of the downstream performance gains, its absolute safety performance remains notably worse than that of SafeLoRA and is still far from the before FT safety level. In addition, the baselines included in Table 3 are not sufficiently comprehensive and omit several recent methods, such as AsFT and Safety at One Shot for protection during fine-tuning, and SafeDelta and Panacea for post-fine-tuning protection.
- The paper lacks an explicit efficiency analysis. The authors should report the total runtime and GPU memory usage of the proposed method and compare its efficiency with that of the baselines under a unified experimental setting to better assess its practicality and scalability.
- The paper contains unresolved cross-references, as both Sections 3.3 and 4 refer to “Appendix ??” instead of the corresponding appendix sections.
Questions for authors
- Does the same functional compression emerge in ordinary task fine-tuning updates under the proposed retain/remove analysis?
- Can the authors broaden the safety and utility evaluations and compare with more recent baselines under a unified setting?
- What are the runtime, GPU memory usage, and additional training overhead of the proposed method compared with the baselines?
6 comments
W2–W3 / Q2. Broader evaluation, requested baselines, and the safety–utility trade-off
Safety and task learning must be assessed jointly. We added all requested baselines—AsFT, Safety at One Shot, SafeDelta, and Panacea—and evaluated HEx-PHI, HarmBench, GSM8K, MMLU, and Qwen adaptive GCG (one seed, ten behaviors). Downstream rows are three-seed means; safety columns are Granite risk.
| GPT-2 method | AdvBench ↓ | HEx-PHI ↓ | HarmBench ↓ | Dolly NLL ↓ | GSM8K % ↑ | MMLU % ↑ |
|---|---|---|---|---|---|---|
| Aligned model | 0.0701 | 0.2957 | 0.1515 | 2.9871 | 2.12 | 23.07 |
| Vanilla | 0.7929 | 0.8663 | 0.7868 | 2.6105 | 1.39 | 22.99 |
| Ours | 0.1381 | 0.6604 | 0.3181 | 2.6129 | 1.29 | 22.99 |
| SafeLoRA, corrected pair | 0.0882 | 0.3361 | 0.1926 | 2.9703 | 2.10 | 23.06 |
| LoX, k=8 | 0.4261 | 0.7436 | 0.5275 | 2.6141 | 1.54 | 22.98 |
| LoX, k=32 | 0.4127 | 0.7243 | 0.4905 | 2.6166 | 1.52 | 22.99 |
| SafeInstr | 0.6165 | 0.8119 | 0.7382 | 2.6086 | 1.57 | 22.99 |
| AsFT | 0.0804 | 0.4427 | 0.2595 | 2.8481 | 1.69 | 23.05 |
| One Shot | 0.7705 | 0.8635 | 0.7867 | 2.6106 | 1.34 | 22.99 |
| SafeDelta | 0.0704 | 0.2950 | 0.1516 | 2.9841 | 2.12 | 23.06 |
| Panacea | 0.7890 | 0.8542 | 0.7873 | 2.9628 | 0.68 | 22.94 |
| Safety SFT | 0.9040 | 0.8843 | 0.8570 | 2.6548 | 1.39 | 22.91 |
| Vaccine | 0.8992 | 0.8742 | 0.8482 | 2.6648 | 1.34 | 23.00 |
| Booster | 0.9072 | 0.8788 | 0.8609 | 2.6377 | 1.29 | 23.72 |
Ours retains 99.36% of Vanilla's Dolly improvement while reducing risk and lies on the GPT-2 mean-endpoint Pareto frontier for HarmBench risk and Dolly NLL (see Fig. 17 (p. 35)). Corrected SafeLoRA achieves lower risk but retains 4.44% of that improvement. We apologize for the submitted SafeLoRA alignment-pair mismatch: it used the safety-only increment instead of the pretrained-to-aligned difference. With the corrected pair, the original sampled-64 evaluation gives Granite/Guard2 risk .02308/.01215 and NLL 2.97035 (revised Table 4); the benchmark table above reports the broader common greedy evaluation separately.
A numerical safety–utility comparison. Using the GPT-2 endpoints underlying Fig. 17 (p. 35), let denote HarmBench Granite risk and Dolly NLL. Subscripts denote the aligned model, Vanilla, and method . Define
s_m=\frac{R_V-R_m}{R_V-R_A},\qquad T_m(w)=w s_m+(1-w)u_m.$$ Here $u_m$ is the fraction of Vanilla's task improvement retained, and $s_m$ is the fraction of its safety degradation prevented. Larger $w$ prioritizes safety; smaller $w$ prioritizes retaining the Dolly NLL improvement. At equal weighting ($w=.5$): | Method | Utility retained (%) ↑ | Degradation prevented (%) ↑ | $100T_m(.5)$ ↑ | |---|---:|---:|---:| | **Ours** | 99.36 | 73.78 | **86.57** | | LoX, k=32 | 98.38 | 46.63 | 72.51 | | LoX, k=8 | 99.05 | 40.82 | 69.94 | | AsFT | 36.91 | 83.00 | 59.96 | | SafeInstr | 100.51 | 7.64 | 54.07 | | SafeDelta | 0.80 | 99.99 | 50.39 | | SafeLoRA | 4.44 | 93.54 | 48.99 | **Ours maximizes this weighted score among all 14 evaluated mean endpoints for safety weights ($w$) from approximately 0.017 to 0.790.** Its advantage therefore holds across a broad range of safety–utility preferences, not only equal weighting. **With no weighting choice, ours also has the lowest AdvBench, HEx-PHI, and HarmBench risk among methods retaining at least 99% of Vanilla's Dolly improvement; the same holds at 95%.** >**Revision location:** Table 28 (p. 34) reports the full comparison. **The method also tests a practical implication of our analysis:** correcting downstream updates within the fixed core using a current safety-memory loss reduces degradation while retaining task learning. **This supports the core’s functional role in downstream safety, connecting Sections 4–6 to a behavioral intervention.**Thank you for the detailed response and the additional experiments. However, I still have the following two concerns.
First, regarding efficiency, ms/update and peak GPU memory reflect the per-update computational and memory overhead, but they are insufficient to characterize the overall computational cost of the method. The authors should report the complete end-to-end runtime under a unified experimental setting.
Second, the paper primarily uses GPT-2 as the evaluation model. In the newly added experiments, its accuracy on GSM8K is only 1%–2%, indicating that the model itself has limited task capability. Therefore, these results do not provide sufficient evidence that the proposed method preserves the model’s reasoning and general capabilities. Meanwhile, the paper’s claims regarding the preservation of task utility are still mainly based on Dolly NLL, which primarily measures how well the model fits the Dolly reference responses and does not adequately represent broader downstream capabilities.
Therefore, I will maintain my original score.
We thank the reviewer for their careful reading and constructive feedback, and address each point below.
W1 / Q1. Is functional compression specific to safety learning?
Safety tuning has stronger removal effects than nine of the ten ordinary-training conditions at the same eight-dyad-per-matrix budget. Compression also occurs outside safety, as stated in Section 2, Lines 139–142: “without being exclusive to safety.”
We compare three dense safety objectives and ten ordinary conditions, with three seeds each and a projector fitted to each run's update. Effects normalize safety-risk reduction or ordinary held-out NLL improvement:
Controls span Dolly, HH-helpful, GSM8K, Alpaca, OASST, classification, summarization, and continued language modeling, from pretrained and instruction-tuned starts.
Functional concentration is strong across safety objectives and task-dependent in ordinary learning.
W4 / Q3. Runtime and memory
Core correction has a modest memory overhead and a measured 32.5% step-time overhead. On an A40 in FP32, three-seed means are:
The additional memory is 0.168 GiB (2.6%), and fitting the fixed basis costs approximately nine seconds once. Ours uses the least peak memory among the four online protection procedures, supporting practical use in this setting.
W5. Unresolved appendix references
Thank you for the careful reading. We apologize for the broken links. Revision location: The corrected links now point to Appendix B.3 (p. 14) for pooled spectral energy and Appendix D (p. 15) for the objective comparison in the revised manuscript. Both sections were already present in the submission.