acceptodds
Under review as a conference paper at ICLR 2027

Mitigating Harmful Fine-Tuning via Task-Aware Directional Safety Repair

Abstract

Fine-tuning large language models on unvetted data can compromise their safety alignment, particularly when the training data contain harmful examples. However, preserving safety throughout harmful fine-tuning without hindering downstream task adaptation remains challenging. To address this challenge, we propose Task-Aware Directional Safety Repair (TDSR), a defense that integrates periodic safety repair into downstream fine-tuning. During each safety repair, TDSR uses gradients of the safety and downstream task losses to identify candidate directions along which parameter updates strongly affect safety losses while causing relatively small changes in task losses. It then selects the candidate most closely aligned with the direction of steepest decrease in the current safety loss and projects the safety descent gradient onto the selected candidate direction to update the model. Extensive experiments across various models and downstream tasks show that TDSR preserves safety alignment while maintaining task performance, outperforming existing baselines. For example, on Mistral-7B-v0.3, TDSR reduces the mean harmful-response rate by 15.62 percentage points and improves mean task performance by 11.30 percentage points across three tasks, relative to the strongest baselines for safety and downstream task performance, respectively.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Harmful Fine-Tuning, Safety Alignment, Safety Preservation, AI Safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_9jDXOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes TDSR, a defense against harmful fine-tuning that periodically pauses training and runs a "safety repair" step. In each repair, it builds a safety sensitivity matrix from per-example gradients on a small safety reference set, and a task sensitivity matrix from gradients on a small task reference set (64 examples). It then finds directions that maximize the ratio of safety to task sensitivity, via a ridge-regularized generalized eigenproblem solved in a reduced space. It picks the candidate most aligned with the safety descent direction and takes a projected step along it. Experiments cover four 7–9B models, three tasks (GSM8K, AlpacaEval, Magicoder) at p = 0.2, plus Qwen3.8-27B, robustness to epochs and harmful ratio, two adaptive attacks, extra safety benchmarks, XSTest, and an activation-based mechanism analysis.

Critical strengths
  • The method is clean and easy to follow. Framing the direction search as a Rayleigh quotient between safety and task gradient second moments is natural, and the reduced-space solver with Woodbury is exact and keeps the eigenproblem small.
  • The results are consistent. TDSR has the lowest HS in 11 of 12 model–task settings and is competitive or best on FA, and this holds on the 27B model, on four additional safety benchmarks, and across epochs and harmful ratios.
  • The paper checks things I would want checked: over-refusal on XSTest (no worse than Base), adaptive attacks aimed at the method, sensitivity to reference set sizes, and training cost (2.75x SFT, comparable to other defenses).
Critical weaknesses

1. No seeds or variance, and some gains are hard to believe. All numbers appear to come from single runs, and the evals are small (164 HumanEval+ problems, 105 AlpacaEval prompts), so one problem is ~0.6–1 point. Several claims rest on gaps smaller than that. At the same time, some FA gains are surprisingly large for a safety method. On Mistral AlpacaEval, TDSR gets 71.43 vs. 40.48 for vanilla SFT and 48.57 for the best baseline, and this number drives the abstract's headline. On Qwen GSM8K, it beats vanilla SFT by 6 points. I'd like to see multiple seeds and some explanation of why a safety repair improves the task this much.

2. How different is this from projecting out the task gradients? With ρ = 1e-3 and median-normalized gradients, most of the task-gradient span has eigenvalues well above ρ. So B_t^{-1} roughly scales the null space by 1/ρ and shrinks the span. If that's right, the candidates are close to safety gradients projected onto the complement of the 64 task gradients, which is basically task-orthogonal repair restricted to one direction. Yet the "task-orthogonal repair" ablation is much worse (12.29 vs. 4.57 HS), and that variant isn't defined in the main text. I'd like to understand what actually drives the difference, e.g. step size, the single-direction restriction, or normalization.

3. Missing simple baselines. The defender already has 200 harmful prompts with safe responses. The obvious baseline is to mix them into fine-tuning (safety data mixing, Bianchi et al.) or to run plain safety SFT at the same budget. The second appears only in one ablation setting. Gradient-projection methods like SafeGrad or Safe LoRA, and post-hoc repair like Antidote or Panacea, are discussed in related work but not compared. It's also unclear whether baseline hyperparameters were tuned, while TDSR's were studied on Qwen + Magicoder, which is also the ablation and robustness setting.

4. The mechanism claim is overstated. "Deviates further from Base" doesn't show a new safety mechanism. Vanilla SFT also deviates. The intervention result, where attenuating TDSR-derived directions hurts more than Base-derived ones in the TDSR model, is what you'd expect from directions extracted from the model being tested. I'd treat this as an observation rather than a contribution.

Minor: the method needs per-example gradients for 68 examples at each repair, which seems feasible only for small LoRA (q, v, rank 8) and wouldn't scale to full fine-tuning. The task reference set comes from the same distribution as the test set, which is a fairly strong assumption for a service provider. The adaptive attacks don't differentiate through the repair and only cover Magicoder.

Questions for authors
  1. Can you report mean ± std over at least three seeds for Table 1, and explain the large FA gains, especially Mistral on AlpacaEval?
  2. How exactly is "task-orthogonal repair" defined? How close are TDSR's candidate directions to the safety gradient projected onto the complement of the task-gradient span, given ρ = 1e-3?
  3. How does TDSR compare to mixing the 200 safety examples directly into fine-tuning, and to SafeGrad or Safe LoRA under the same data?
  4. Were baseline hyperparameters tuned, and on which setting were TDSR's hyperparameters chosen?
  5. Does the method work, and what does it cost, with LoRA on all linear layers or with full fine-tuning?
6 comments
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1: Seed Variability and Task Performance Gains (data)

Weakness 1 and Question 1: Variability Across Seeds and Large FA Gains (data)

Can you report mean ± std over at least three seeds for Table 1 [...]

The following table reports the mean and sample standard deviation of HS and FA over three training seeds for all methods, models, and tasks in Table 1. TDSR achieves the lowest mean HS in 10 of the 12 combinations of models and tasks and the highest mean FA in eight. These results support the consistency of TDSR’s overall safety and utility gains across the evaluated training seeds.

Values are percentages. Bold marks the best mean within each dataset and metric column.

DatasetMethodQwen HS ↓Llama HS ↓Gemma HS ↓Mistral HS ↓Avg. HS ↓Qwen FA ↑Llama FA ↑Gemma FA ↑Mistral FA ↑Avg. FA ↑
GSM8KVanilla SFT72.48 ± 1.2867.38 ± 0.8681.76 ± 1.5381.05 ± 0.9375.67 ± 0.7377.89 ± 1.0378.77 ± 1.0575.82 ± 0.5043.09 ± 0.2768.89 ± 0.10
GSM8KLISA14.00 ± 0.8631.19 ± 0.6814.76 ± 1.6742.86 ± 0.6525.70 ± 0.2978.59 ± 1.3374.32 ± 0.4275.41 ± 0.4242.51 ± 0.8167.71 ± 0.31
GSM8KSBR6.52 ± 0.735.19 ± 1.959.95 ± 1.0827.71 ± 0.9412.35 ± 0.9177.00 ± 1.4679.81 ± 1.0576.14 ± 0.2335.38 ± 1.3167.08 ± 0.84
GSM8KOASIS8.95 ± 0.2233.24 ± 1.4521.81 ± 0.5444.14 ± 1.5127.04 ± 0.6376.95 ± 0.8267.42 ± 0.3168.76 ± 1.0738.94 ± 1.0763.02 ± 0.64
GSM8KBooster5.67 ± 1.0335.43 ± 2.027.14 ± 1.0328.62 ± 0.8119.21 ± 0.4170.18 ± 0.5268.81 ± 0.7352.14 ± 0.2341.32 ± 0.3058.11 ± 0.11
GSM8KGuardSpace24.33 ± 0.8436.90 ± 1.494.52 ± 0.3641.76 ± 1.0026.88 ± 0.5676.57 ± 0.6572.20 ± 0.3680.59 ± 0.9940.23 ± 0.9967.40 ± 0.50
GSM8KTDSR5.19 ± 0.081.81 ± 0.504.71 ± 1.009.52 ± 0.735.31 ± 0.2683.14 ± 0.1979.45 ± 1.1778.17 ± 0.5343.09 ± 1.3170.96 ± 0.66
AlpacaEvalVanilla SFT40.33 ± 1.4542.14 ± 0.6542.38 ± 2.2145.76 ± 1.8642.65 ± 1.1645.40 ± 1.9246.67 ± 1.7243.97 ± 3.3440.32 ± 3.8244.09 ± 1.42
AlpacaEvalLISA28.57 ± 0.8928.81 ± 1.4712.38 ± 0.7241.57 ± 0.7427.83 ± 0.6053.97 ± 5.2247.46 ± 1.3759.84 ± 0.7339.68 ± 3.5150.24 ± 0.21
AlpacaEvalSBR8.52 ± 0.226.95 ± 2.2715.43 ± 1.8731.24 ± 1.6615.54 ± 1.1955.08 ± 3.5753.33 ± 2.5247.94 ± 1.6747.14 ± 0.0050.87 ± 1.57
AlpacaEvalOASIS11.14 ± 0.8935.81 ± 1.8131.19 ± 1.0044.57 ± 1.4930.68 ± 0.5947.30 ± 1.9238.25 ± 4.0538.73 ± 1.5335.08 ± 2.9139.84 ± 1.32
AlpacaEvalBooster19.90 ± 1.3330.62 ± 0.544.76 ± 1.0538.62 ± 0.7323.48 ± 0.6943.97 ± 4.3242.70 ± 3.0629.84 ± 0.5541.90 ± 4.8339.60 ± 2.08
AlpacaEvalGuardSpace17.81 ± 1.1536.14 ± 1.3610.29 ± 0.7141.05 ± 1.1526.32 ± 0.3854.92 ± 2.4044.13 ± 0.5552.54 ± 3.3446.35 ± 0.5549.48 ± 0.69
AlpacaEvalTDSR7.33 ± 1.031.52 ± 0.333.14 ± 0.7614.24 ± 1.446.56 ± 0.4355.24 ± 1.7258.89 ± 2.4055.24 ± 1.9070.16 ± 1.5359.88 ± 0.52
MagicoderVanilla SFT48.71 ± 0.6560.19 ± 1.9548.10 ± 0.8645.43 ± 1.0050.61 ± 0.3770.33 ± 1.2740.85 ± 0.0053.86 ± 2.4628.46 ± 0.7048.37 ± 0.69
MagicoderLISA10.14 ± 1.0015.33 ± 2.199.29 ± 1.7638.05 ± 1.0318.20 ± 0.0270.33 ± 0.3556.10 ± 0.0059.96 ± 2.3127.85 ± 1.5353.56 ± 0.63
MagicoderSBR5.05 ± 1.546.33 ± 1.1510.00 ± 0.7624.95 ± 0.5811.58 ± 0.2479.47 ± 0.9353.86 ± 1.8654.67 ± 1.4126.02 ± 1.2753.51 ± 0.66
MagicoderOASIS27.38 ± 0.7937.24 ± 1.2114.43 ± 0.2943.52 ± 0.5930.64 ± 0.3466.87 ± 0.9354.27 ± 1.0651.63 ± 1.2723.17 ± 0.6148.98 ± 0.62
MagicoderBooster6.71 ± 0.8740.71 ± 0.526.57 ± 1.8636.71 ± 0.6522.68 ± 0.5265.65 ± 2.4656.71 ± 1.8350.41 ± 1.5329.47 ± 0.9350.56 ± 1.30
MagicoderGuardSpace8.33 ± 1.074.14 ± 1.138.43 ± 0.2531.19 ± 1.7213.02 ± 0.6673.37 ± 0.9357.52 ± 1.2755.89 ± 0.7026.22 ± 0.6153.25 ± 0.47
MagicoderTDSR4.62 ± 0.724.43 ± 0.005.05 ± 1.8913.71 ± 0.526.95 ± 0.5381.50 ± 1.9660.37 ± 2.6657.52 ± 0.3535.16 ± 1.2758.64 ± 0.62
AuthorsOpenReview ↗

Response to Weakness 1 and Question 1: Seed Variability and Task Performance Gains (Analysis)

Weakness 1 and Question 1: Variability Across Seeds and Large FA Gains (Analysis)

Can you report mean ± std over at least three seeds for Table 1 [...]

The larger FA gains remain in this summary. On Mistral-7B with AlpacaEval, TDSR achieves 70.16% mean FA, compared with 47.14% for SBR and 40.32% for vanilla SFT, corresponding to improvements of 23.02 and 29.84 percentage points, respectively. Its mean HS is also lower, at 14.24%, compared with 31.24% for SBR and 45.76% for vanilla SFT. On Qwen2.5-7B with GSM8K, TDSR achieves 83.14% mean FA versus 77.89% for vanilla SFT, retaining a 5.25 pp improvement.

All methods, including vanilla SFT, are fine-tuned with 20% harmful examples. Harmful supervision can interfere with safety alignment as well as downstream task learning, so mitigating harmful adaptation can also help preserve task performance. Several defenses exhibit this pattern in Table 1. For example, on Mistral with AlpacaEval, SBR and GuardSpace improve FA from vanilla SFT’s 40.48% to 48.57% and 47.14%, respectively.

Mistral’s safety alignment also appears particularly vulnerable to downstream fine-tuning. After 20 epochs of vanilla SFT on entirely benign Magicoder data, its HS reaches 17.29%, roughly four times Qwen2.5-7B’s 4.29% on the same Beaver700 prompts (Figure 5(c) in Appendix G and Figure 2(c), respectively). This is consistent with relatively fragile safety alignment under adaptation, even without harmful examples in the training data.

A possible explanation for Mistral’s large FA gain is that suppressing harmful adaptation also reduces interference with downstream task learning. In Table 1 of the manuscript, TDSR achieves 13.14% HS and 71.43% FA on Mistral with AlpacaEval, compared with 30.57% HS and 48.57% FA for SBR, the strongest baseline on both metrics. These joint gains are consistent with TDSR reducing disruption from harmful supervision while its task-aware repair criterion limits interference with task learning, helping preserve both safety alignment and downstream capabilities.

AuthorsOpenReview ↗

Response to Weakness 2 and Question 2: Differences from Task-Orthogonal Repair

Weakness 2 and Question 2: Differences from Task-Orthogonal Repair

How exactly is “task-orthogonal repair” defined? How close are TDSR’s candidate directions to the safety gradient projected onto the complement of the task-gradient span, given ρ=10−3\rho=10^{-3}?

Definition of task-orthogonal repair. Let Gt\mathbf{G}_t contain the 64 task reference gradients as rows, and let d\mathbf{d} denote the negative gradient of the current mini-batch mean safety loss. The baseline computes

P⊥=I−Gt⊤(GtGt⊤)†Gt,\mathbf{P}_{\perp} = \mathbf{I} - \mathbf{G}_t^\top(\mathbf{G}_t\mathbf{G}_t^\top)^\dagger\mathbf{G}_t,

and applies

Δθorth=clip⁡C(ηsP⊥d).\Delta\boldsymbol{\theta}_{\mathrm{orth}} = \operatorname{clip}_{C}\left(\eta_s\mathbf{P}_{\perp}\mathbf{d}\right).

The pseudoinverse uses numerical rank truncation without ridge regularization, and clip⁡C\operatorname{clip}_{C} caps the L2 update norm at CC. Task gradients are refreshed at the beginning of each repair and held fixed throughout that repair. The reference data, repair schedule, step size ηs=0.002\eta_s=0.002, and norm cap C=0.03C=0.03 match TDSR.

How TDSR differs. TDSR constructs candidates from the safety-gradient second moment rather than directly projecting the mean safety gradient. Specifically, it solves the reduced eigenproblem

Qak=λkak,Q=XBt−1X⊤,\mathbf{Q}\mathbf{a}_k = \lambda_k\mathbf{a}_k,\qquad \mathbf{Q} = \mathbf{X}\mathbf{B}_t^{-1}\mathbf{X}^{\top},

and recovers candidate directions as

vk=Bt−1X⊤ak,\mathbf{v}_k = \mathbf{B}_t^{-1}\mathbf{X}^{\top}\mathbf{a}_k,

where X\mathbf{X} contains the scaled safety gradients and Bt=At+ρI\mathbf{B}_t=\mathbf{A}_t+\rho\mathbf{I}. TDSR then selects a unit candidate v^\hat{\mathbf{v}} by alignment with d\mathbf{d} and applies

ΔθTDSR=clip⁡C(ηs(d⊤v^)v^).\Delta\boldsymbol{\theta}_{\mathrm{TDSR}} = \operatorname{clip}_{C}\left(\eta_s(\mathbf{d}^{\top}\hat{\mathbf{v}})\hat{\mathbf{v}}\right).

Thus, the baseline applies the full projected mean gradient, whereas TDSR selects one spectral direction and projects the safety descent gradient onto it. Even when the candidates lie close to the task-orthogonal subspace, these operations need not produce the same update. Matching the nominal step size and norm cap also does not force the actual update norms to coincide. The ablation therefore compares the complete repair rules, including their directions and magnitudes; its HS results of 12.29% versus 4.57% do not isolate the contribution of each difference.

Role of ρ\rho. The regularization parameter controls the preference for low empirical task sensitivity. Increasing ρ\rho makes candidate ranking less sensitive to differences in task sensitivity. We therefore choose a small ρ\rho to favor a repair direction v^\hat{\mathbf{v}} close to the task-orthogonal subspace and limit interference with downstream task learning. We evaluated this trade-off on Qwen2.5-7B-Instruct with Magicoder:

ρ\rhoHS ↓FA ↑
0.001 (default)4.57%81.10%
0.014.29%78.05%
0.14.43%75.61%
14.29%74.39%

The default ρ=0.001\rho=0.001 achieves the highest FA while maintaining low HS. Larger values reduce HS by only 0.14–0.28 pp, while reducing FA by 3.05–6.71 pp. These results support the default as a favorable balance between safety recovery and task preservation in this setting.

Measured directional similarity. At ρ=0.001\rho=0.001, we compare the final selected TDSR direction with P⊥d\mathbf{P}_{\perp}\mathbf{d} at identical model parameters and on the same safety mini-batches. Their mean cosine similarity is 0.85, indicating substantial alignment while showing that the selected directions do not simply coincide with the strictly projected mean gradient.

Favoring low task sensitivity is therefore an intended property of TDSR, while its spectral construction and final candidate selection distinguish its update from direct task-orthogonal projection. We will make the baseline definition and these additional diagnostics explicit in the revised manuscript.

Reviewer_2Um5OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Task-Aware Directional Safety Repair (TDSR), which periodically interleaves safety repair into downstream fine-tuning. At each repair step, TDSR uses gradients of the safety and downstream-task losses (computed on the defender's reference sets Ds and Dt) to define a "safety sensitivity" and a "task sensitivity," searches for candidate directions that maximize a regularized safety-to-task sensitivity ratio, selects the candidate most aligned with the direction of steepest decrease in the current safety loss, and projects the safety-descent gradient onto that direction to update the model. The low rank of the reference gradient matrices avoids constructing matrices at the full parameter dimension. On Qwen2.5-7B / Llama-3.1-8B / Gemma-2-9B / Mistral-7B-v0.3 across three tasks (GSM8K / AlpacaEval / Magicoder) at a harmful ratio of p=0.2 against six baselines, TDSR achieves the lowest HS in 11 of 12 model×task combinations and the highest FA in 8. On Mistral-7B it reduces HS by 15.62 pp and improves FA by 11.30 pp relative to the strongest baselines. The paper also reports epoch/harmful-ratio robustness, two adaptive attacks (gradient selection / gradient suffix), ablations, and a mechanism analysis (attenuating TDSR-derived vs Base-derived directions yields HS 51% vs 19% at full attenuation).

Critical strengths
  1. A clean, reproducible methodological instantiation of direction selection. The intuition that a safety repair should move along directions that strongly affect safety while weakly affecting the task is formalized into a computable sensitivity ratio plus alignment with steepest descent, and the low rank of reference gradients avoids full-dimensional matrix construction, making the approach engineering-feasible. The ablations (task-orthogonal repair, without task sensitivity, highest-ratio only) cleanly isolate each component's contribution, so "task-aware" is more than a slogan.
  2. Comprehensive evaluation including adaptive attacks and a mechanism intervention. Five models × three tasks × six baselines, plus epoch/harmful-ratio robustness curves, two adaptive attacks, and a causal-style intervention ("attenuating TDSR-derived vs Base-derived directions," 51% vs 19% HS), elevates the mechanism claim—"forming new activation patterns rather than preserving Base"—from correlation toward intervention evidence, exceeding the evaluation depth of most comparable defense papers.
Critical weaknesses
  1. Dependence on defender reference sets and a single-setting mechanism analysis. TDSR assumes the defender holds 200 Ds + 64 Dt reference examples disjoint from training/evaluation data—reasonable for an API provider but not always for user-side custom fine-tuning, and the paper does not discuss the boundary of this assumption. The mechanism analysis (Figure 3) is confined to Qwen2.5-7B + Magicoder, so whether "new activation patterns rather than Base preservation" holds across models/tasks is unverified.
  2. Limited conceptual novelty and insufficiently validated generality of the core claim. Relative to existing safety-repair / safety-subspace lines (Booster, SBR, GuardSpace), the conceptual advance is mainly the specific sensitivity-ratio direction selector rather than a new paradigm; the strongest differentiating evidence (the mechanism intervention) is drawn from a single model-task pair. Taken together with (1) and (2), the contribution reads as a solid, well-evaluated engineering instantiation whose central mechanism claim is not yet shown to generalize—this weakens the case for acceptance at a venue that rewards conceptual or widely-validated advances.
Questions for authors
  1. Are the abstract's 15.62 pp / 11.30 pp relative to the same baseline? Please provide a table giving TDSR's simultaneous HS and FA gaps relative to a single strongest baseline, to avoid splicing two different "strongest" baselines.
  2. On Mistral-7B, TDSR's HS (9.71–13.14) is much higher than on the other three models (2.43–5.86). Please analyze the cause: is the angle between Mistral's alignment direction and the BeaverTails safety direction larger, is LoRA r=8 under-capacity, or is the reference set too small for Mistral? Is there a fix?
  3. The mechanism analysis (Figure 3) uses only Qwen2.5-7B + Magicoder. On Llama/Gemma/Mistral or AlpacaEval/GSM8K, does "attenuating TDSR-derived directions raises HS more than Base-derived directions" still hold?
  4. If the defender cannot obtain a high-quality Dt (user task unknown), into what form does TDSR degenerate? Can it fall back to a Ds-only safety repair, and how does it compare with SBR/Booster in that degenerate setting?
5 comments
AuthorsOpenReview ↗

Response to W1 and Q3: Reference Data Requirements and Mechanism Generality

We thank Reviewer 2Um5 for carefully reviewing our work and providing insightful and constructive feedback.

Weakness 1.1: Reference Data Requirements for User-Side Fine-Tuning

TDSR assumes the defender holds 200 DsD_s + 64 DtD_t reference examples disjoint from training/evaluation data—reasonable for an API provider but not always for user-side custom fine-tuning, and the paper does not discuss the boundary of this assumption.

The 200 safety and 64 task reference examples are our experimental defaults. Figure 4 in Appendix E.4 shows how much of TDSR's benefit is retained when each reference set is reduced separately, under a fixed repair-update budget. With Dt\mathcal{D}_t fixed at 64, 10 safety examples retained 87.9% of the HS reduction achieved with 200 safety examples. HS fell from 48.29% under vanilla SFT to 9.86%, compared with 4.57% under the default configuration. With Ds\mathcal{D}_s fixed at 200, eight task examples retained 88.9% of the FA gain achieved with 64 task examples. FA rose from 70.12% under vanilla SFT to 79.88%, compared with 81.10% under the default configuration. These results show that either reference set can be substantially reduced while still providing useful guidance for repair in this setting.

We kept both reference sets disjoint from the evaluation data to measure performance on unseen examples. Because Ds\mathcal{D}_s directly supervises repair and Dt\mathcal{D}_t guides the repair direction, overlap with the evaluation set could inflate the reported scores.

For user-side fine-tuning, the reference examples can come from existing data sources. In our experiments, we constructed Ds\mathcal{D}_s by reserving public BeaverTails prompts and pairing them with safe responses, and Dt\mathcal{D}_t by setting aside benign examples from each task dataset before fine-tuning. A user who controls training can therefore reuse the same safety reference examples across tasks for the same model and reserve a few trusted examples for each new Dt\mathcal{D}_t. We will clarify these requirements for user-side fine-tuning in Section 3.2.

Weakness 1.2 and Question 3: Generality of the Intervention Effect

Weakness 1.2: The mechanism analysis (Figure 3) is confined to Qwen2.5-7B + Magicoder, so whether “new activation patterns rather than Base preservation” holds across models/tasks is unverified.

Question 3: On Llama/Gemma/Mistral or AlpacaEval/GSM8K, does “attenuating TDSR-derived directions raises HS more than Base-derived directions” still hold?

To assess whether the intervention effect generalizes across models and tasks, we apply the protocol in Figure 3(d) to four LLMs fine-tuned on GSM8K, AlpacaEval, and Magicoder. For each model and task, we separately attenuate Base-derived and TDSR-derived activation directions in the same repaired model at strengths of 25%, 50%, 75%, and 100%, evaluating both interventions on the same harmful prompts. Each table entry reports the HS under TDSR-derived attenuation minus that under Base-derived attenuation, averaged over these four strengths and expressed in percentage points. The mean differences shown in the table are positive in all 12 combinations, ranging from 20.63 to 39.75 pp.

ModelGSM8KAlpacaEvalMagicoder
Qwen2.5-7B+23.75+20.63+23.50
Llama-3.1-8B+24.00+21.75+24.25
Gemma-2-9B+35.13+31.13+34.25
Mistral-7B-v0.3+39.75+33.63+38.88
AuthorsOpenReview ↗

Response to Weakness 2: Conceptual Contribution and Mechanistic Evidence

Weakness 2: Conceptual Contribution and Mechanistic Evidence

Limited conceptual novelty and insufficiently validated generality of the core claim. Relative to existing safety-repair / safety-subspace lines (Booster, SBR, GuardSpace), the conceptual advance is mainly the specific sensitivity-ratio direction selector rather than a new paradigm; the strongest differentiating evidence (the mechanism intervention) is drawn from a single model-task pair. […][\ldots]

Representative defenses such as SBR and GuardSpace preserve selected safety-related structures of the original aligned model during fine-tuning. SBR anchors the final hidden states of harmful reference prompts to those of the frozen aligned model. GuardSpace freezes safety-relevant weight components and constrains adapter updates through a null-space projection to preserve layer outputs on reference harmful inputs.

TDSR makes downstream task sensitivity an explicit criterion for determining repair directions. It identifies directions with high safety sensitivity relative to task sensitivity, then selects a corrective update using the current safety descent gradient. Even with identical safety gradients, different task reference gradients can therefore lead to different repair directions. This formulation makes task preservation part of the repair objective and allows safety-related representations to evolve without requiring them to match those of the original aligned model.

Figure 3 provides mechanistic evidence for this route on Qwen2.5-7B-Instruct with Magicoder. It shows changed activation patterns on harmful inputs, while attenuating TDSR-derived directions produces higher HS than attenuating Base-derived directions in the same repaired model. These findings support an alternative route to safety recovery through task-aware repair that reorganizes the activation patterns supporting safe responses to harmful inputs, rather than simply preserving those of the original aligned model. Generality across models and tasks discussed separately in our response also supports this idea.

AuthorsOpenReview ↗

Response to Question 1: Baseline Comparisons and Performance Gaps

Question 1: Baseline Comparators and Paired Performance Gaps

Are the abstract's 15.62 pp / 11.30 pp relative to the same baseline? Please provide a table giving TDSR's simultaneous HS and FA gaps relative to a single strongest baseline, to avoid splicing two different "strongest" baselines.

The two figures in the abstract use different comparators, each selected as the strongest baseline for its respective metric. We computed them from the Mistral-7B-v0.3 results in Table 1, giving equal weight to GSM8K, AlpacaEval, and Magicoder. Among the six baselines, SBR achieved the lowest mean HS at 27.43%, but its mean FA of 36.80% was below GuardSpace's 37.89%. GuardSpace achieved the highest mean FA, but its mean HS of 37.57% was higher than SBR's. TDSR averaged 11.81% HS and 49.19% FA, yielding an HS reduction of 15.62 pp relative to SBR and an FA gain of 11.30 pp relative to GuardSpace. For these three-task means on Mistral, the abstract therefore reports TDSR's smallest advantage over any baseline on each metric. Using a single comparator for both metrics, TDSR reduces mean HS by 15.62 pp and increases mean FA by 12.39 pp relative to SBR. Relative to GuardSpace, the corresponding improvements are 25.76 pp and 11.30 pp.

The following table reports HS, FA, and gaps for TDSR and six baselines across four models and three datasets. Scores are percentages. Parentheses report signed differences in percentage points, calculated as TDSR minus the baseline in the same setting. A negative HS difference and a positive FA difference favor TDSR.

DatasetMethodQwen HS ↓Qwen FA ↑Llama HS ↓Llama FA ↑Gemma HS ↓Gemma FA ↑Mistral HS ↓Mistral FA ↑
GSM8KVanilla SFT73.92 (−68.06)77.71 (+6.06)67.85 (−65.28)79.00 (+0.68)81.57 (−78.86)75.51 (+2.43)81.29 (−71.58)43.06 (+0.15)
GSM8KLISA13.57 (−7.71)78.32 (+5.45)30.86 (−28.29)73.46 (+6.22)14.29 (−11.58)75.74 (+2.20)43.00 (−33.29)42.30 (+0.91)
GSM8KSBR6.29 (−0.43)77.26 (+6.51)4.71 (−2.14)79.53 (+0.15)9.71 (−7.00)76.42 (+1.52)27.57 (−17.86)35.56 (+7.65)
GSM8KOASIS7.86 (−2.00)76.65 (+7.12)33.29 (−30.72)67.02 (+12.66)21.00 (−18.29)68.84 (+9.10)44.14 (−34.43)39.58 (+3.63)
GSM8KBooster6.00 (−0.14)70.36 (+13.41)35.00 (−32.43)69.14 (+10.54)6.29 (−3.58)51.71 (+26.23)29.29 (−19.58)40.86 (+2.35)
GSM8KGuardSpace23.43 (−17.57)76.72 (+7.05)37.71 (−35.14)72.33 (+7.35)4.14 (−1.43)80.36 (−2.42)41.43 (−31.72)40.26 (+2.95)
GSM8KTDSR5.8683.772.5779.682.7177.949.7143.21
AlpacaEvalVanilla SFT41.29 (−34.72)47.14 (+6.19)41.86 (−39.43)46.19 (+12.86)42.00 (−39.29)43.81 (+12.38)45.43 (−32.29)40.48 (+30.95)
AlpacaEvalLISA27.57 (−21.00)54.29 (−0.96)28.86 (−26.43)47.62 (+11.43)12.86 (−10.15)57.14 (−0.95)40.71 (−27.57)40.00 (+31.43)
AlpacaEvalSBR8.00 (−1.43)53.52 (−0.19)6.57 (−4.14)54.76 (+4.29)15.14 (−12.43)48.57 (+7.62)30.57 (−17.43)48.57 (+22.86)
AlpacaEvalOASIS9.71 (−3.14)47.17 (+6.16)36.71 (−34.28)38.10 (+20.95)32.43 (−29.72)42.38 (+13.81)43.57 (−30.43)35.23 (+36.20)
AlpacaEvalBooster20.29 (−13.72)43.81 (+9.52)31.71 (−29.28)44.29 (+14.76)6.00 (−3.29)30.00 (+26.19)38.29 (−25.15)38.57 (+32.86)
AlpacaEvalGuardSpace16.71 (−10.14)55.24 (−1.91)35.43 (−33.00)43.33 (+15.72)10.86 (−8.15)54.28 (+1.91)40.71 (−27.57)47.14 (+24.29)
AlpacaEvalTDSR6.5753.332.4359.052.7156.1913.1471.43
MagicoderVanilla SFT48.29 (−43.72)70.12 (+10.98)59.15 (−55.58)40.14 (+20.23)46.86 (−43.72)53.66 (+3.66)45.14 (−32.57)28.05 (+4.88)
MagicoderLISA9.71 (−5.14)70.73 (+10.37)14.14 (−10.57)55.49 (+4.88)8.14 (−5.00)58.53 (−1.21)38.43 (−25.86)27.44 (+5.49)
MagicoderSBR4.43 (+0.14)79.88 (+1.22)5.14 (−1.57)54.27 (+6.10)11.43 (−8.29)54.88 (+2.44)24.14 (−11.57)26.27 (+6.66)
MagicoderOASIS26.86 (−22.29)68.29 (+12.81)38.00 (−34.43)53.66 (+6.71)14.29 (−11.15)51.83 (+5.49)43.43 (−30.86)23.78 (+9.15)
MagicoderBooster6.71 (−2.14)65.24 (+15.86)41.71 (−38.14)56.10 (+4.27)6.43 (−3.29)49.39 (+7.93)36.43 (−23.86)29.88 (+3.05)
MagicoderGuardSpace8.29 (−3.72)73.78 (+7.32)4.29 (−0.72)57.32 (+3.05)7.57 (−4.43)55.49 (+1.83)30.57 (−18.00)26.27 (+6.66)
MagicoderTDSR4.5781.103.5760.373.1457.3212.5732.93

Across these model, dataset, and baseline combinations, TDSR achieves both lower HS and higher FA than the same baseline in 65 of 72 comparisons.

Reviewer_n4EyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper presents Task-Aware Directional Safety Repair, an additional mechanism during the fine-tuning step that periodically updates the model towards a safety-aligned direction. The attack scenario is partially poisoned fine-tuning dataset which may introduce back-doors during training. The training party keeps safe datasets and task datasets as grounded alignment directions. After a round of training, the "safe" gradient directions and task gradient directions are used to compute the direction to take for the safety repair gradient while minimizing updates on the task direction. They run their experiments on smaller-end models like Qwen2.5-7B-Instruct, Llama-3.1-8B-Instruct on a diverse benchmark (GSM8K. Magicoder, AlpacaEval, etc) and report Harmful Score and Fine-tuning accuracy on every task. They also do ablations such as varying epochs, different attacks and harmful ratio sensitivity.

Critical strengths
  1. Has a good range of models and tasks the experiments are run on.
  2. The formulas and algorithm (Appendix) seem logically sound and consistent.
  3. The results show strong improvements.
Critical weaknesses
  1. Lacking in-depth details on the actual mechanism. Quite a few questions about the method are still there (see Questions for the Authors Section)
  2. The need for D_t seems like a problem in practical settings, especially since other methods exist that do not need it.
  3. Lacking important ablation studies and experiments.
Questions for authors
  1. From my understanding, TDSR first ranks directions based on safety impact relative to task impact, and then keeps the top k. It only uses 4 datapoints per batch, so isn't the shortlist itself kind of limited? To that point, why the choice of 4 and then also K=4 (Appendix)? Would this not mean it isn't really shortlisting anything? Explanations on this seem to be missing
  2. It seems like BeaverTails is the source for poisoned data , evaluation and D_s. While it is a standard in literature, using a single dataset leaves room for questions on the generalizability and adaptability of the solution. Do the authors have insights on the generalizability or any reasoning for this choice? I can see Table 7 uses different test sets, would it not make sense that D_s is changed accordingly as well?
  3. In a real-world scenario, how would one ensure that D_t itself is not contaminated?
  4. Rho is an important variable in that it effectively determines how much to trade task accuracy for safety. It is set low at 0.001 compared to . Would this value not mean that it acts exactly like the standard task-orthogonal repair? To this point, how was the 0.001 value derived/tuned? Do the authors have some type of sensitivity analysis results?
  5. How well does it scale? From my understanding, it needs separate gradients for every reference example. How much training time does it already cost?
  6. Is the choice for 200 datapoints a deliberate balance between accuracy and cost? In Figure 4a, it seems like more datapoints benefit safety.
  7. It seems like the results come from single runs without varying the seeds. Some numbers seem small enough that they can be explained away by the randomness. Do the authors have the mean with +/- std results?
4 comments
AuthorsOpenReview ↗

Response to Questions 1 and 2: Candidate Selection and Safety Generalization

We thank Reviewer n4Ey for carefully reviewing our work and providing insightful and constructive feedback.

Question 1: Why use K=4K=4 with only four safety examples per mini-batch?

From my understanding, TDSR first ranks directions based on safety impact relative to task impact, and then keeps the top kk. It only uses 4 datapoints per batch, so isn't the shortlist itself kind of limited? To that point, why the choice of 4 and then also K=4K=4 (Appendix)? Would this not mean it isn't really shortlisting anything? Explanations on this seem to be missing.

With a safety mini-batch size of four, our default choice of K=4K=4 indeed retains all numerically positive modes without additional truncation. Since rank⁡(As)≤4\operatorname{rank}(A_s)\leq4, at most four positive generalized eigenmodes are available. We will clarify this distinction in the revised manuscript.

TDSR then selects one candidate according to its alignment with the current safety descent gradient. On Qwen2.5-7B-Instruct with Magicoder, over 85% of the recorded repair updates select the largest-eigenvalue direction, and over 90% select one of the two largest-eigenvalue directions. Thus, the final selection concentrates on leading sensitivity-ratio directions even without additional truncation. Retaining the remaining candidates provides alternatives for the minority of updates, which can reduce overreliance on the highest-ratio direction estimated from a small mini-batch.

Small mini-batches provide repeated opportunities to adjust repair directions, which help compensate for occasional unfavorable selections. With a batch size of four, each repair process all 200 safety reference examples in 50 mini-batches, recomputing safety gradients and candidate directions at the current parameters for each mini-batch. However, for larger mini-batches, we suggest retaining a small number of leading candidates to focus selection on directions with high sensitivity ratios. With a fixed safety reference set, larger mini-batches yield fewer updates per repair and therefore fewer opportunities for subsequent adjustments.

Appendix E.2 evaluates K∈{2,3,4,8}K\in\{2,3,4,8\}, using a mini-batch size of four except for K=8K=8, which uses eight examples. The default K=4K=4 achieves the highest FA among these configurations while maintaining low HS. Furthermore, the highest-ratio-only ablation in Table 2 yields 8.00% HS, compared with 4.57% for the full method, supporting the usefulness of retaining alternatives for the final selection.

Question 2: Generalization Beyond BeaverTails and Alternative Safety Reference Data

It seems like BeaverTails is the source for poisoned data, evaluation and D_s. [...]

Following prior work such as LISA, Booster, OASIS, Vaccine, and SBR, we also use BeaverTails as the source of our safety reference data (Ds\mathcal{D}_s).

Table 7 examines whether safety protection obtained using BeaverTails alone as the safety reference source generalizes to other safety benchmarks. We keep Ds\mathcal{D}_s unchanged and evaluate the resulting model on AdvBench, HarmBench, HEx-PHI, and SORRY-Bench, without adapting the reference set to these benchmarks. The results support the transfer of safety protection beyond the dataset used to construct Ds\mathcal{D}_s.

We also follow the reviewer's suggestion by conducting additional experiments with alternative safety reference datasets. In four separate experiments, we replace the BeaverTails-based Ds\mathcal{D}_s with a reference set constructed from each benchmark in turn. We keep the other training configuration unchanged. Given the limited benchmark sizes, we restrict this comparison to TDSR and SBR, as the reference-data budgets used for the other defense baselines in our experiments cannot be accommodated.

With these alternative reference sets, TDSR achieves lower HS than SBR on all four benchmarks and lower ASR/CR on three of four. Although the evaluation subsets differ from those in Table 7, TDSR's reported HS values remain within 1.67 pp of the corresponding results obtained with BeaverTails-based references, while ASR/CR differences range from 4.33 pp lower to 1.25 pp higher. These results provide complementary evidence that TDSR can operate effectively with safety reference data from different sources.

All values below are percentages. Lower is better for every metric, and the best result in each column is bold.

MethodAdvBench ASRAdvBench HSHarmBench ASRHarmBench HSHEx-PHI ASRHEx-PHI HSSORRY-Bench CRSORRY-Bench HS
SBR0.311.8820.0017.5015.0013.0034.5810.83
TDSR0.000.0022.5015.005.003.0027.088.33
AuthorsOpenReview ↗

Response to Questions 3 and 4: Task Reference Contamination and Rho Selection

Question 3: Task Reference Contamination

In a real-world scenario, how would one ensure that DtD_t itself is not contaminated?

The small size of Dt\mathcal{D}_t makes individual inspection practical, allowing the defender to review each prompt and target response for safety and task correctness before including it in the reference set. Our reference-size experiments in Appendix E.4, Figure 4(c,d), further support this approach. On Qwen2.5-7B-Instruct with Magicoder, TDSR achieves 3.86% HS and 79.88% FA using only eight task reference examples. Compared with the default 64-example configuration, FA decreases by only 1.22 percentage points, while HS decreases by 0.71 percentage points. These results support the feasibility of maintaining a small, manually vetted task reference set independently of the untrusted fine-tuning data.

To examine robustness when reference curation is imperfect, we evaluate contamination levels of 8, 32, and 48 harmful prompts paired with unsafe responses among the 64 task reference examples on Magicoder across four models. At 75% contamination, FA decreases by at most 3.66 percentage points relative to the clean-reference results, while HS increases by 7.14 to 9.43 pp. HS ranges from 12.29% to 19.71%. Even under this substantial contamination, TDSR retains effective safety repair with limited utility degradation, consistent with the remaining benign examples providing useful guidance for task preservation.

All values below are percentages.

Harmful Examples in Dt\mathcal{D}_tQwen2.5 HS ↓Qwen2.5 FA ↑Llama HS ↓Llama FA ↑Gemma HS ↓Gemma FA ↑Mistral HS ↓Mistral FA ↑
8/648.5779.276.4359.157.0055.4916.1431.10
32/6412.1479.279.7158.5410.4356.1020.2932.32
48/6414.0079.8812.5758.5412.2953.6619.7131.71

Question 4: The Role and Selection of ρ\rho

Rho is an important variable in that it effectively determines how much to trade task accuracy for safety. […][\ldots] Would this value not mean that it acts exactly like the standard task-orthogonal repair? To this point, how was the 0.001 value derived/tuned? Do the authors have some type of sensitivity analysis results?

Relation to task-orthogonal repair. A small ρ\rho intentionally favors directions with low empirical task sensitivity, but it does not make TDSR equivalent to strict task-orthogonal repair. For a unit direction v\mathbf{v}, TDSR uses the sensitivity ratio

R(v)=v⊤Asvv⊤Atv+ρ.R(\mathbf{v})=\frac{\mathbf{v}^{\top}A_s\mathbf{v}}{\mathbf{v}^{\top}A_t\mathbf{v}+\rho}.

Increasing ρ\rho reduces the relative influence of differences in task sensitivity. We therefore use a small value to favor directions close to the task-orthogonal subspace and limit interference with downstream task learning.

Strict task-orthogonal repair directly projects the mean safety descent gradient d\mathbf{d} onto the orthogonal complement of the task-gradient span, yielding P⊥d\mathbf{P}_{\perp}\mathbf{d}. TDSR instead constructs generalized eigenvector candidates from the safety and task gradient second moments, selects the candidate most closely aligned with d\mathbf{d}, and projects d\mathbf{d} onto that selected direction. Thus, favoring low task sensitivity does not eliminate the distinction between the two update rules.

We also measure this difference on Qwen2.5-7B-Instruct with Magicoder at ρ=0.001\rho=0.001. At identical model parameters and on the same safety mini-batches, the final selected TDSR direction and P⊥d\mathbf{P}_{\perp}\mathbf{d} have a mean cosine similarity of 0.85. This shows substantial alignment while confirming that the selected directions do not simply coincide with the strictly projected mean gradient in this setting.

Parameter selection and sensitivity analysis. We selected ρ=0.001\rho=0.001 empirically on Qwen2.5-7B-Instruct with Magicoder, considering both HS and FA. The following sensitivity analysis compares values from 0.001 to 1.

ρ\rhoHS (%) ↓FA (%) ↑
0.001 (default)4.5781.10
0.014.2978.05
0.14.4375.61
14.2974.39

The default ρ=0.001\rho=0.001 achieves the highest FA among the tested values while maintaining low HS. Larger values reduce HS by only 0.14 to 0.28 pp, while reducing FA by 3.05 to 6.71 pp. These results support the smaller value as a favorable balance between safety recovery and task preservation in this setting.

We will clarify the distinction from strict task-orthogonal repair, explain the empirical choice of ρ\rho, and include these sensitivity results and directional diagnostics in the revised manuscript.

AuthorsOpenReview ↗

Response to Questions 5 and 6: Scalability and Safety Reference Set Size

Question 5: Scalability and Training Cost

How well does it scale? From my understanding, it needs separate gradients for every reference example. How much training time does it already cost?

Measured training cost. We compare training times on Qwen3.8-27B using a single NVIDIA RTX PRO 6000 GPU to assess the practicality of TDSR at this scale. Each task uses 20 epochs of downstream fine-tuning. The reported times include all safety repairs and the full initial alignment cost of Booster and OASIS for each standalone task.

MethodMagicoder (hours)GSM8K (hours)Alpaca (hours)
SFT2.321.020.74
LISA5.824.082.27
SBR6.474.743.21
Booster11.5510.249.96
OASIS9.378.107.80
GuardSpace2.371.040.75
TDSR6.154.402.83

Across the three tasks, TDSR averages 4.46 hours per task, compared with 1.36 hours for vanilla SFT. It therefore adds 3.10 hours per task, with a mean training time 3.28×3.28\times that of SFT. Among the five defense baselines, TDSR is faster on average than SBR (4.81 hours), OASIS (8.42 hours), and Booster (10.58 hours). The two faster defenses are LISA (4.06 hours) and GuardSpace (1.39 hours). TDSR's mean training time is close to LISA's, requiring approximately 24 additional minutes per task. These results support the practicality of TDSR for fine-tuning larger models.

Question 6: Why Use 200 Safety Reference Examples?

Is the choice for 200 datapoints a deliberate balance between accuracy and cost? In Figure 4a, it seems like more datapoints benefit safety.

We chose 200 safety reference examples as a practical default given the available data and training budget. BeaverTails provides enough distinct prompts to reserve this set for repair without overlap with the fine-tuning or evaluation data.

We agree that larger safety reference sets improve safety over the tested range. On Qwen2.5-7B-Instruct with Magicoder, increasing the set from 10 to 100 distinct examples lowered HS from 9.86% to 5.00%. Doubling the set to 200 lowered HS by a further 0.43 percentage points to 4.57%. FA varied less across these settings, with all three smaller sets yielding FA within 2 pp of the 81.10% achieved with 200 examples (Appendix E.4, Figure 4(a,b)).

Additionally, in terms of training time, Appendix H (Figure 6) reports the end-to-end training time on Qwen2.5-7B-Instruct with 200 safety reference examples. TDSR takes 2.75×2.75\times as long as vanilla SFT, close to LISA (2.45×2.45\times) and SBR (2.87×2.87\times). It uses 12.7% less total training time than OASIS (3.15×3.15\times) and 35.3% less than Booster (4.25×4.25\times).

The observed safety gains, measured training cost, and availability of suitable reference data support 200 examples as a practical default for our safety reference set.

AuthorsOpenReview ↗

### Question 7: Variability Across Training Seeds

Question 7: Variability Across Training Seeds

It seems like the results come from single runs without varying the seeds. Some numbers seem small enough that they can be explained away by the randomness. Do the authors have the mean with +/- std results?

The table below reports the mean and sample standard deviation of HS and FA over three independent training seeds for all methods, models, and tasks in Table 1.

TDSR achieves the lowest mean HS in 10 of the 12 combinations of models and tasks and the highest mean FA in eight. When averaged across the four models, it also achieves the lowest HS and highest FA on each of the three downstream tasks. These results suggest that TDSR maintains stable safety and downstream task performance across the evaluated training seeds.

DatasetMethodQwen HS ↓Llama HS ↓Gemma HS ↓Mistral HS ↓Avg. HS ↓Qwen FA ↑Llama FA ↑Gemma FA ↑Mistral FA ↑Avg. FA ↑
GSM8KVanilla SFT72.48 ± 1.2867.38 ± 0.8681.76 ± 1.5381.05 ± 0.9375.67 ± 0.7377.89 ± 1.0378.77 ± 1.0575.82 ± 0.5043.09 ± 0.2768.89 ± 0.10
GSM8KLISA14.00 ± 0.8631.19 ± 0.6814.76 ± 1.6742.86 ± 0.6525.70 ± 0.2978.59 ± 1.3374.32 ± 0.4275.41 ± 0.4242.51 ± 0.8167.71 ± 0.31
GSM8KSBR6.52 ± 0.735.19 ± 1.959.95 ± 1.0827.71 ± 0.9412.35 ± 0.9177.00 ± 1.4679.81 ± 1.0576.14 ± 0.2335.38 ± 1.3167.08 ± 0.84
GSM8KOASIS8.95 ± 0.2233.24 ± 1.4521.81 ± 0.5444.14 ± 1.5127.04 ± 0.6376.95 ± 0.8267.42 ± 0.3168.76 ± 1.0738.94 ± 1.0763.02 ± 0.64
GSM8KBooster5.67 ± 1.0335.43 ± 2.027.14 ± 1.0328.62 ± 0.8119.21 ± 0.4170.18 ± 0.5268.81 ± 0.7352.14 ± 0.2341.32 ± 0.3058.11 ± 0.11
GSM8KGuardSpace24.33 ± 0.8436.90 ± 1.494.52 ± 0.3641.76 ± 1.0026.88 ± 0.5676.57 ± 0.6572.20 ± 0.3680.59 ± 0.9940.23 ± 0.9967.40 ± 0.50
GSM8KTDSR5.19 ± 0.081.81 ± 0.504.71 ± 1.009.52 ± 0.735.31 ± 0.2683.14 ± 0.1979.45 ± 1.1778.17 ± 0.5343.09 ± 1.3170.96 ± 0.66
AlpacaEvalVanilla SFT40.33 ± 1.4542.14 ± 0.6542.38 ± 2.2145.76 ± 1.8642.65 ± 1.1645.40 ± 1.9246.67 ± 1.7243.97 ± 3.3440.32 ± 3.8244.09 ± 1.42
AlpacaEvalLISA28.57 ± 0.8928.81 ± 1.4712.38 ± 0.7241.57 ± 0.7427.83 ± 0.6053.97 ± 5.2247.46 ± 1.3759.84 ± 0.7339.68 ± 3.5150.24 ± 0.21
AlpacaEvalSBR8.52 ± 0.226.95 ± 2.2715.43 ± 1.8731.24 ± 1.6615.54 ± 1.1955.08 ± 3.5753.33 ± 2.5247.94 ± 1.6747.14 ± 0.0050.87 ± 1.57
AlpacaEvalOASIS11.14 ± 0.8935.81 ± 1.8131.19 ± 1.0044.57 ± 1.4930.68 ± 0.5947.30 ± 1.9238.25 ± 4.0538.73 ± 1.5335.08 ± 2.9139.84 ± 1.32
AlpacaEvalBooster19.90 ± 1.3330.62 ± 0.544.76 ± 1.0538.62 ± 0.7323.48 ± 0.6943.97 ± 4.3242.70 ± 3.0629.84 ± 0.5541.90 ± 4.8339.60 ± 2.08
AlpacaEvalGuardSpace17.81 ± 1.1536.14 ± 1.3610.29 ± 0.7141.05 ± 1.1526.32 ± 0.3854.92 ± 2.4044.13 ± 0.5552.54 ± 3.3446.35 ± 0.5549.48 ± 0.69
AlpacaEvalTDSR7.33 ± 1.031.52 ± 0.333.14 ± 0.7614.24 ± 1.446.56 ± 0.4355.24 ± 1.7258.89 ± 2.4055.24 ± 1.9070.16 ± 1.5359.88 ± 0.52
MagicoderVanilla SFT48.71 ± 0.6560.19 ± 1.9548.10 ± 0.8645.43 ± 1.0050.61 ± 0.3770.33 ± 1.2740.85 ± 0.0053.86 ± 2.4628.46 ± 0.7048.37 ± 0.69
MagicoderLISA10.14 ± 1.0015.33 ± 2.199.29 ± 1.7638.05 ± 1.0318.20 ± 0.0270.33 ± 0.3556.10 ± 0.0059.96 ± 2.3127.85 ± 1.5353.56 ± 0.63
MagicoderSBR5.05 ± 1.546.33 ± 1.1510.00 ± 0.7624.95 ± 0.5811.58 ± 0.2479.47 ± 0.9353.86 ± 1.8654.67 ± 1.4126.02 ± 1.2753.51 ± 0.66
MagicoderOASIS27.38 ± 0.7937.24 ± 1.2114.43 ± 0.2943.52 ± 0.5930.64 ± 0.3466.87 ± 0.9354.27 ± 1.0651.63 ± 1.2723.17 ± 0.6148.98 ± 0.62
MagicoderBooster6.71 ± 0.8740.71 ± 0.526.57 ± 1.8636.71 ± 0.6522.68 ± 0.5265.65 ± 2.4656.71 ± 1.8350.41 ± 1.5329.47 ± 0.9350.56 ± 1.30
MagicoderGuardSpace8.33 ± 1.074.14 ± 1.138.43 ± 0.2531.19 ± 1.7213.02 ± 0.6673.37 ± 0.9357.52 ± 1.2755.89 ± 0.7026.22 ± 0.6153.25 ± 0.47
MagicoderTDSR4.62 ± 0.724.43 ± 0.005.05 ± 1.8913.71 ± 0.526.95 ± 0.5381.50 ± 1.9660.37 ± 2.6657.52 ± 0.3535.16 ± 1.2758.64 ± 0.62

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.