- Rating
- 3
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces a method to quickly repair the safety of fine-tuned models. The proposed method, OPUS, learns a universal patch that is added to the fine-tuned model. The patch is optimized over simulated post-finetuned models (pre-constructed) and the base model, where the objective is to align the behaviour of the patched models (by adding the patch to the simulated post-finetuned models) with that of the base model. Then, when the base model is deployed, even if the user submits fine-tuned models, the safety of fine-tuned models can be quickly repaired by adding the patch.
Critical strengths
- The paper is well written, and the motivation is clear.
- The proposed method is simple but effective, and saves the cost of repairing the safety of fine-tuned models.
Critical weaknesses
- The main concern is whether the simulated post-finetuned models can represent the true post-finetuned models. For example, Figure 1 shows that different fine-tuned models span nonlinearly, but the simulated post-finetuned models are obtained via linear interpolation in weight space. Also, what if the users fine-tune models with different LoRAs?
- As for the patch optimization, it assumes that the perturbations cover as many harmful fine-tuning as possible, but is this even possible?
Questions for authors
- In the experiment, lines 289-290 emphasized that and are disjoint from the harmful data used in the fine-tuning attacks, but how far is the disjoint distance? If they are extremely far apart and completely different, will the proposed method fail?
- seems quite important? Currently, the attacker's settings are the same in the experiments; what if they are different (for example, different LoRAs or other fine-tuned structures)?
7 comments
Author Response (Part II)
Q3: Also, what if the users fine-tune models with different LoRAs? (And from Questions: "Currently, the attacker's settings are the same in the experiments; what if they are different (for example, different LoRAs or other fine-tuned structures)?")
R3: Thank you for this constructive suggestion!
-
To evaluate this, we apply the same patch while varying the LoRA rank, target modules, and fine-tuning method (DoRA and QLoRA) of the user's fine-tuning in the default setting (SAMSum, hr=0.1, Llama-3-8B-Instruct). For DoRA, whose weight update is not a low-rank product due to its magnitude normalization, we first merge the user's adapter into the base model and then add . For QLoRA, the adapter is trained on a 4-bit NF4 base model and deployed on the 16-bit base model together with .
-
As shown below, across all LoRA ranks, target modules, and fine-tuning methods, OPUS reduces HS from at least 46% to at most 1.20%, while maintaining utility comparable to SFT. This indicates that OPUS remains effective across different fine-tuning configurations. We have added these results to Appendix D.2 of our revised paper.
Table 1. HS (%)↓ / Utility↑ under different LoRA ranks.
| Method | r=4 | r=8 (default) | r=16 |
|---|---|---|---|
| SFT | 52.20 / 51.85 | 52.60 / 51.80 | 52.20 / 51.99 |
| OPUS | 0.60 / 51.10 | 0.60 / 51.41 | 0.80 / 51.43 |
Table 2. HS (%)↓ / Utility↑ under different target modules.
| Method | q, v | q, k, v (default) | q, k, v, o |
|---|---|---|---|
| SFT | 52.20 / 51.84 | 52.60 / 51.80 | 52.40 / 52.09 |
| OPUS | 0.80 / 51.22 | 0.60 / 51.41 | 1.20 / 52.00 |
Table 3. HS (%)↓ / Utility↑ under different fine-tuning methods.
| Method | LoRA (default) | DoRA | QLoRA |
|---|---|---|---|
| SFT | 52.60 / 51.80 | 52.60 / 51.76 | 46.00 / 52.25 |
| OPUS | 0.60 / 51.41 | 0.60 / 51.37 | 0.20 / 51.63 |
Q4: As for the patch optimization, it assumes that the perturbations cover as many harmful fine-tuning as possible, but is this even possible?
R4: Thank you for this insightful question! We would like to clarify that OPUS does not require a large number of perturbations to cover all possible harmful fine-tuning.
-
OPUS repairs a shared failure pattern rather than memorizing each perturbation. Although compromised fine-tuned models are different points in parameter space, they share the same underlying failure pattern, i.e., the loss of the ability to refuse unsafe prompts. By optimizing over a diverse set of perturbations, the patch is driven to repair this shared pattern rather than fitting each individual perturbation, so exhaustive coverage is unnecessary.
-
A small perturbation bank suffices. By default, the bank contains only 10 harmful and 10 benign perturbations (plus mixed perturbations interpolated from them), constructed solely from BeaverTails and Alpaca. Nevertheless, OPUS remains effective against fine-tuning not covered by the bank, including different downstream tasks, OOD harmful datasets, and even benign fine-tuning attacks.
Author Response (Part III)
Q5: In the experiment, lines 289-290 emphasized that and are disjoint from the harmful data used in the fine-tuning attacks, but how far is the disjoint distance? If they are extremely far apart and completely different, will the proposed method fail?
R5: Thank you for this insightful question! We address it as follows.
-
In the default setting, and are disjoint from the harmful data used in the fine-tuning attacks in sample index, i.e., they share no samples (neither prompts nor responses), although all of them are drawn from BeaverTails.
-
Beyond this setting, our paper also evaluates attack data from entirely different sources, i.e., OOD harmful datasets (Red Team, LLM-LAT, AdvBench), and even benign fine-tuning attacks that contain no harmful content and degrade safety through a completely different mechanism (Anchoring, Self-Inf-N). Across all these attacks, OPUS keeps HS within 0.40%–1.00% (Table 4), showing that it does not fail even when the attack data is completely different from .
We have clarified this in Section 4.1 of our revised paper.
Table 4. Harmful scores (%) under different attack data (from Tables 4 and 5 of our paper).
| Attack data | SFT | OPUS |
|---|---|---|
| BeaverTails (default) | 52.60 | 0.60 |
| Red Team | 57.20 | 0.60 |
| LLM-LAT | 57.60 | 1.00 |
| AdvBench | 61.40 | 0.60 |
| Anchoring | 8.40 | 0.40 |
| Self-Inf-N | 21.00 | 0.40 |
Q6: seems quite important?
R6: Thank you for this question! is indeed important, but it does not require careful tuning: it suffices to use the natural scale of the patch, i.e., .
-
is the natural scale of the patch. The patch is optimized on , so simply applies the learned patch as it is. We use for all models, tasks, harmful ratios, and attacks without any per-setting tuning, and OPUS consistently achieves low HS.
-
as a general form. We introduce mainly to express our method in a more general form, and the sweep in the ablation study of our paper is a post-hoc sensitivity analysis. A smaller applies only part of the patch, and the resulting safety degradation is therefore expected. For , HS stays low with only a mild utility decrease.
We have clarified this in Section 3.5 of our revised paper.
Author Response (Part I)
Dear Reviewer NmX4, thank you very much for your careful review of our paper and thoughtful comments! We are greatly encouraged by your positive assessment, particularly your recognition that our paper is well written with a clear motivation, and that our method is simple but effective and saves the cost of repairing the safety of fine-tuned models. We hope the following detailed responses can help clarify potential misunderstandings and alleviate your concerns.
Q1: The patch is optimized over simulated post-finetuned models (pre-constructed) and the base model, where the objective is to align the behaviour of the patched models (by adding the patch to the simulated post-finetuned models) with that of the base model. (From Paper Summary)
R1: Thank you for your careful summary! We would like to kindly clarify one detail of our objective. On harmful prompts, the patch δ is optimized to restore refusal; on benign inputs, however, the KL term anchors the patched model θ+P+δ to the fine-tuned model θ+P, rather than the base model θ. This is a deliberate design: anchoring to θ would penalize the patch for preserving the task-specific adaptations learned during fine-tuning, whereas anchoring to θ+P requires the patch to change only what is necessary for safety, thereby preserving utility.
Q2: The main concern is whether the simulated post-finetuned models can represent the true post-finetuned models. For example, Figure 1 shows that different fine-tuned models span nonlinearly, but the simulated post-finetuned models are obtained via linear interpolation in weight space.
R2: Thank you for this insightful comment! We would like to clarify the following points.
Figure 1 illustrates the diversity of fine-tuned models, rather than the geometry of parameter space. Figure 1 is a t-SNE visualization of the weight increments obtained by fine-tuning on harmful data mixed with different downstream tasks. It aims to show that fine-tuning with different data recipes lands in well-separated regions of parameter space, which motivates optimizing the patch over a diverse population of perturbations. Since t-SNE is a nonlinear dimensionality reduction, the shapes in Figure 1 do not reflect the geometry of the original parameter space. Moreover, the variation in Figure 1 comes from different downstream tasks, whereas linear interpolation is performed between harmful and benign perturbations.
Most simulated perturbations are obtained by real fine-tuning. In the perturbation bank, both the harmful and benign perturbations are obtained by actually fine-tuning the base model; linear interpolation is only used to synthesize the mixed perturbations as a cost-free augmentation. Hence, the perturbation bank is mainly built from real fine-tuning, rather than relying on linear interpolation to simulate it.
All evaluated models are true post-finetuned models. Every fine-tuned model in our evaluation is obtained by real fine-tuning and is not included in the perturbation bank, covering different downstream tasks, harmful ratios, harmful attack datasets, and benign attacks. OPUS is effective on all of them, showing that the patch learned on simulated perturbations transfers to true post-finetuned models.