- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes TDSR, a defense against harmful fine-tuning that periodically pauses training and runs a "safety repair" step. In each repair, it builds a safety sensitivity matrix from per-example gradients on a small safety reference set, and a task sensitivity matrix from gradients on a small task reference set (64 examples). It then finds directions that maximize the ratio of safety to task sensitivity, via a ridge-regularized generalized eigenproblem solved in a reduced space. It picks the candidate most aligned with the safety descent direction and takes a projected step along it. Experiments cover four 7–9B models, three tasks (GSM8K, AlpacaEval, Magicoder) at p = 0.2, plus Qwen3.8-27B, robustness to epochs and harmful ratio, two adaptive attacks, extra safety benchmarks, XSTest, and an activation-based mechanism analysis.
Critical strengths
- The method is clean and easy to follow. Framing the direction search as a Rayleigh quotient between safety and task gradient second moments is natural, and the reduced-space solver with Woodbury is exact and keeps the eigenproblem small.
- The results are consistent. TDSR has the lowest HS in 11 of 12 model–task settings and is competitive or best on FA, and this holds on the 27B model, on four additional safety benchmarks, and across epochs and harmful ratios.
- The paper checks things I would want checked: over-refusal on XSTest (no worse than Base), adaptive attacks aimed at the method, sensitivity to reference set sizes, and training cost (2.75x SFT, comparable to other defenses).
Critical weaknesses
1. No seeds or variance, and some gains are hard to believe. All numbers appear to come from single runs, and the evals are small (164 HumanEval+ problems, 105 AlpacaEval prompts), so one problem is ~0.6–1 point. Several claims rest on gaps smaller than that. At the same time, some FA gains are surprisingly large for a safety method. On Mistral AlpacaEval, TDSR gets 71.43 vs. 40.48 for vanilla SFT and 48.57 for the best baseline, and this number drives the abstract's headline. On Qwen GSM8K, it beats vanilla SFT by 6 points. I'd like to see multiple seeds and some explanation of why a safety repair improves the task this much.
2. How different is this from projecting out the task gradients? With ρ = 1e-3 and median-normalized gradients, most of the task-gradient span has eigenvalues well above ρ. So B_t^{-1} roughly scales the null space by 1/ρ and shrinks the span. If that's right, the candidates are close to safety gradients projected onto the complement of the 64 task gradients, which is basically task-orthogonal repair restricted to one direction. Yet the "task-orthogonal repair" ablation is much worse (12.29 vs. 4.57 HS), and that variant isn't defined in the main text. I'd like to understand what actually drives the difference, e.g. step size, the single-direction restriction, or normalization.
3. Missing simple baselines. The defender already has 200 harmful prompts with safe responses. The obvious baseline is to mix them into fine-tuning (safety data mixing, Bianchi et al.) or to run plain safety SFT at the same budget. The second appears only in one ablation setting. Gradient-projection methods like SafeGrad or Safe LoRA, and post-hoc repair like Antidote or Panacea, are discussed in related work but not compared. It's also unclear whether baseline hyperparameters were tuned, while TDSR's were studied on Qwen + Magicoder, which is also the ablation and robustness setting.
4. The mechanism claim is overstated. "Deviates further from Base" doesn't show a new safety mechanism. Vanilla SFT also deviates. The intervention result, where attenuating TDSR-derived directions hurts more than Base-derived ones in the TDSR model, is what you'd expect from directions extracted from the model being tested. I'd treat this as an observation rather than a contribution.
Minor: the method needs per-example gradients for 68 examples at each repair, which seems feasible only for small LoRA (q, v, rank 8) and wouldn't scale to full fine-tuning. The task reference set comes from the same distribution as the test set, which is a fairly strong assumption for a service provider. The adaptive attacks don't differentiate through the repair and only cover Magicoder.
Questions for authors
- Can you report mean ± std over at least three seeds for Table 1, and explain the large FA gains, especially Mistral on AlpacaEval?
- How exactly is "task-orthogonal repair" defined? How close are TDSR's candidate directions to the safety gradient projected onto the complement of the task-gradient span, given ρ = 1e-3?
- How does TDSR compare to mixing the 200 safety examples directly into fine-tuning, and to SafeGrad or Safe LoRA under the same data?
- Were baseline hyperparameters tuned, and on which setting were TDSR's hyperparameters chosen?
- Does the method work, and what does it cost, with LoRA on all linear layers or with full fine-tuning?
6 comments
Response to Weakness 1 and Question 1: Seed Variability and Task Performance Gains (Analysis)
Weakness 1 and Question 1: Variability Across Seeds and Large FA Gains (Analysis)
Can you report mean ± std over at least three seeds for Table 1 [...]
The larger FA gains remain in this summary. On Mistral-7B with AlpacaEval, TDSR achieves 70.16% mean FA, compared with 47.14% for SBR and 40.32% for vanilla SFT, corresponding to improvements of 23.02 and 29.84 percentage points, respectively. Its mean HS is also lower, at 14.24%, compared with 31.24% for SBR and 45.76% for vanilla SFT. On Qwen2.5-7B with GSM8K, TDSR achieves 83.14% mean FA versus 77.89% for vanilla SFT, retaining a 5.25 pp improvement.
All methods, including vanilla SFT, are fine-tuned with 20% harmful examples. Harmful supervision can interfere with safety alignment as well as downstream task learning, so mitigating harmful adaptation can also help preserve task performance. Several defenses exhibit this pattern in Table 1. For example, on Mistral with AlpacaEval, SBR and GuardSpace improve FA from vanilla SFT’s 40.48% to 48.57% and 47.14%, respectively.
Mistral’s safety alignment also appears particularly vulnerable to downstream fine-tuning. After 20 epochs of vanilla SFT on entirely benign Magicoder data, its HS reaches 17.29%, roughly four times Qwen2.5-7B’s 4.29% on the same Beaver700 prompts (Figure 5(c) in Appendix G and Figure 2(c), respectively). This is consistent with relatively fragile safety alignment under adaptation, even without harmful examples in the training data.
A possible explanation for Mistral’s large FA gain is that suppressing harmful adaptation also reduces interference with downstream task learning. In Table 1 of the manuscript, TDSR achieves 13.14% HS and 71.43% FA on Mistral with AlpacaEval, compared with 30.57% HS and 48.57% FA for SBR, the strongest baseline on both metrics. These joint gains are consistent with TDSR reducing disruption from harmful supervision while its task-aware repair criterion limits interference with task learning, helping preserve both safety alignment and downstream capabilities.
Response to Weakness 2 and Question 2: Differences from Task-Orthogonal Repair
Weakness 2 and Question 2: Differences from Task-Orthogonal Repair
How exactly is “task-orthogonal repair” defined? How close are TDSR’s candidate directions to the safety gradient projected onto the complement of the task-gradient span, given ?
Definition of task-orthogonal repair. Let contain the 64 task reference gradients as rows, and let denote the negative gradient of the current mini-batch mean safety loss. The baseline computes
and applies
The pseudoinverse uses numerical rank truncation without ridge regularization, and caps the L2 update norm at . Task gradients are refreshed at the beginning of each repair and held fixed throughout that repair. The reference data, repair schedule, step size , and norm cap match TDSR.
How TDSR differs. TDSR constructs candidates from the safety-gradient second moment rather than directly projecting the mean safety gradient. Specifically, it solves the reduced eigenproblem
and recovers candidate directions as
where contains the scaled safety gradients and . TDSR then selects a unit candidate by alignment with and applies
Thus, the baseline applies the full projected mean gradient, whereas TDSR selects one spectral direction and projects the safety descent gradient onto it. Even when the candidates lie close to the task-orthogonal subspace, these operations need not produce the same update. Matching the nominal step size and norm cap also does not force the actual update norms to coincide. The ablation therefore compares the complete repair rules, including their directions and magnitudes; its HS results of 12.29% versus 4.57% do not isolate the contribution of each difference.
Role of . The regularization parameter controls the preference for low empirical task sensitivity. Increasing makes candidate ranking less sensitive to differences in task sensitivity. We therefore choose a small to favor a repair direction close to the task-orthogonal subspace and limit interference with downstream task learning. We evaluated this trade-off on Qwen2.5-7B-Instruct with Magicoder:
| HS ↓ | FA ↑ | |
|---|---|---|
| 0.001 (default) | 4.57% | 81.10% |
| 0.01 | 4.29% | 78.05% |
| 0.1 | 4.43% | 75.61% |
| 1 | 4.29% | 74.39% |
The default achieves the highest FA while maintaining low HS. Larger values reduce HS by only 0.14–0.28 pp, while reducing FA by 3.05–6.71 pp. These results support the default as a favorable balance between safety recovery and task preservation in this setting.
Measured directional similarity. At , we compare the final selected TDSR direction with at identical model parameters and on the same safety mini-batches. Their mean cosine similarity is 0.85, indicating substantial alignment while showing that the selected directions do not simply coincide with the strictly projected mean gradient.
Favoring low task sensitivity is therefore an intended property of TDSR, while its spectral construction and final candidate selection distinguish its update from direct task-orthogonal projection. We will make the baseline definition and these additional diagnostics explicit in the revised manuscript.
Response to Weakness 1 and Question 1: Seed Variability and Task Performance Gains (data)
Weakness 1 and Question 1: Variability Across Seeds and Large FA Gains (data)
The following table reports the mean and sample standard deviation of HS and FA over three training seeds for all methods, models, and tasks in Table 1. TDSR achieves the lowest mean HS in 10 of the 12 combinations of models and tasks and the highest mean FA in eight. These results support the consistency of TDSR’s overall safety and utility gains across the evaluated training seeds.
Values are percentages. Bold marks the best mean within each dataset and metric column.