- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This work investigates how to restore security to a model after it has been fine-tuned with malicious data, while retaining its original task capabilities. It proposes CCTR to fit the output residuals of the attacked model to achieve this goal.
Critical strengths
The most persuasive aspect is that the paper runs controlled experiments targeting the proposed gap. Figure 1 shows that using P_l alone leaves about 99.47% to 99.89% of the normalized residual unexplained, with a cosine similarity to the full residual of only 0.034 to 0.075. This finding is diagnostic on its own, but Figure 4 goes further. It fixes the solver and changes only the trajectory term in the objective, P_l + αT_l. Adding T_l lowers HS from 32.3, 30.4, and 41.1 to 0.4, 0.1, and 1.2 across the three tasks. The authors also control for amplifying P_l and for matched update norms, which shows the improvement cannot be attributed simply to larger updates.
Table 4 effectively tests whether the utility constraint is necessary. Without it, HS remains very low, but task accuracy collapses to 0 to 2.8%. This shows that low harmfulness can come from model failure, and it explains why the full objective must constrain safety and utility together.
The main results are also solid. In Table 1, compared with AsFT, the safest competing method, average HS drops from 2.30% to 0.38% at nearly identical accuracy. In the cross task results of Table 2, HS falls from 2.85% to 0.77% while accuracy rises from 60.50% to 64.03%. The method covers four models and reports a full repair pipeline cost of about 6.76 minutes.
Critical weaknesses
W1: Local closed-loop optimality does not equate to optimal fix for the entire model. All modules use frozen attacked activations to estimate patches, but the upstream representation has changed after patch application. Subsequent modules actually receive different inputs than during the solution process. Experiments show that this approximation can be effective, but it does not fully explain why simultaneous fixes by multiple modules do not overcompensate, and it lacks comparison with layer-by-layer re-acquisition of representations.
W2: Improvements on independent security benchmarks are much more moderate than in the main table. In Table 10, the unsafe fulfillment on the SORRY-Bench decreases from 68/440 on attacked to 55/440, i.e., 15.45% → 12.50%. This supports the existence of generalization gains, but the near-zero HS on BeaverTails cannot be interpreted as widespread security recovery.
Questions for authors
Q1. How does simultaneously fixing all modules compare to re-acquiring representations layer by layer? Can you quantify the degree to which the final representation deviates from the calibrated representation?
Q2. Can you provide results for independent fine-tuning of the seed, and analysis of pairwise uncertainty improved by SORRY-Bench?
Q3. Can you supplement this with overrejection evaluations such as XSTest, and broader tests of generation task capabilities?
16 comments
Author Response to Reviewer sWwN (Part 2/5)
The table shows the first block and representative middle-to-late blocks, indexed 0–31. Safety and Utility refer to safety and benign inputs, respectively. Entries are frozen / block-wise, in %.
| Block index | Safety-input drift: frozen / block-wise | Utility-input drift: frozen / block-wise |
|---|---|---|
| 0 | 0.00 / 0.00 | 0.00 / 0.00 |
| 8 | 0.67 / 0.79 | 0.09 / 0.12 |
| 12 | 1.79 / 3.97 | 0.15 / 0.59 |
| 16 | 3.37 / 7.67 | 0.27 / 1.14 |
| 20 | 5.56 / 12.04 | 0.45 / 1.89 |
| 24 | 8.41 / 15.67 | 0.67 / 2.81 |
| 28 | 11.98 / 18.23 | 0.96 / 3.75 |
| 31 | 26.36 / 33.53 | 2.18 / 7.30 |
No module has been patched before the first block, so its input drift is zero. The frozen method has smaller input drift at every reported middle-to-late block. Utility-input drift at Block 28 is 0.96%, versus 3.75% for the block-wise method, and remains smaller at Block 31. This supports better control of benign-representation changes accumulated with depth, consistent with the realized corrections and full-model safety results.
(3) Realized module-output corrections after inputs change.
We further measure actual linear-module output changes in the fully repaired model to test whether patches remain effective after their inputs change. These changes include both the patch's direct effect on the original frozen input and the effect of upstream input changes propagated through the repaired weights. They therefore reflect deployment more directly than measuring the patch effect at fixed inputs alone.
At the same held-out tokens and positions, we measure two quantities. Correction cosine compares the direction of the realized output change with the original safety target, defined as the reference-model output minus the original attacked-model output; higher values indicate better directional alignment. Utility-output drift is the norm of the actual output change relative to the original output on utility inputs; lower values indicate smaller changes. For both metrics, inner products or squared norms are aggregated over all 224 modules before computing the cosine or norm ratio, rather than averaging module-wise ratios.
| Task | Frozen correction cosine↑ | Block-wise correction cosine↑ | Frozen utility-output drift↓ | Block-wise utility-output drift↓ |
|---|---|---|---|---|
| SST-2 | 0.774 | 0.345 | 1.11% | 3.76% |
| AG News | 0.774 | 0.335 | 1.17% | 3.84% |
| GSM8K | 0.751 | 0.472 | 2.46% | 4.74% |
The frozen method's realized corrections are more aligned with the original target and produce smaller benign-output changes. The preceding experiment measures representation changes at block entrances, whereas this experiment assesses their functional consequences at module outputs. Together with full-model generation results, these findings support the effectiveness of frozen-trajectory repair in the actual forward pass.
(4) Effects of activation recomputation on later repair targets.
We also compare residuals before and after activation recomputation on independent held-out examples to examine what changes beyond the inputs themselves. The frozen method always uses the reference-model output minus the original attacked-model output as its target. Before patching the current block, the block-wise method instead recomputes its module outputs using new inputs from the repaired prefix and subtracts them from the same reference outputs. Input changes therefore also alter the residuals that later modules must fit.
Author Response to Reviewer sWwN (Part 3/5)
The old/new target cosines are 0.063, 0.063, and 0.139 on SST-2, AG News, and GSM8K, respectively, with relative target changes of 4.16, 4.23, and 2.54 times. Relative change is the aggregate norm of the old/new target difference divided by that of the original target, after summing squared norms across modules. Activation recomputation therefore changes both the inputs and repair residuals, whereas the frozen method retains a consistent original functional reference for each module. Full-model behavior, input drift, and realized correction directions jointly support this design. The closed-form solution is locally optimal for fixed activations; deployment performance is directly assessed by the full-model evaluations above.
Q2: Can you provide independent fine-tuning seeds and paired uncertainty for the SORRY-Bench improvement?
Independent fine-tuning and SORRY-Bench evaluation both reproduce CCTR's safety advantage. Primary-benchmark HS is 0.4%, 0.3%, and 0.4% across the three seeds, and SORRY-Bench UFR is 12.576% ± 0.131% in mean ± sample standard deviation, with the lowest value on both metrics for every seed. CCTR also has the lowest number of unsafe fulfillments on all three SORRY-Bench tasks, reducing GSM8K from Attacked's 94/440 to 54/440 and demonstrating gains across downstream tasks. Under AdvBench attack-source and evaluation-source shifts, HS is 0.50% and 0.10%, respectively, also the lowest, supporting safety gains when either the harmful training source or evaluation source changes. We present independent training, independent safety evaluation and paired analysis, and data-source shifts below.
(1) Stability across independent training runs on the primary benchmark.
We independently train with seeds 42, 43, and 44 on SST-2 at , comparing Attacked, seven baselines, and CCTR to test whether the main safety gain depends on a single training run. Each method follows its established procedure and configuration under common generation and scoring settings. Each seed is evaluated on 1,000 safety requests and 500 task examples. Seed columns report HS / Acc; the final columns report mean ± sample standard deviation across three seeds, all in %.
| Method | Seed 42 | Seed 43 | Seed 44 | HS mean ± SD↓ | Acc mean ± SD↑ |
|---|---|---|---|---|---|
| Attacked | 74.3 / 93.8 | 74.5 / 93.8 | 74.0 / 93.8 | 74.267 ± 0.252 | 93.800 ± 0.000 |
| Safe LoRA | 66.4 / 95.6 | 66.7 / 95.4 | 67.1 / 94.4 | 66.733 ± 0.351 | 95.133 ± 0.643 |
| SafeMERGE | 23.6 / 95.0 | 21.1 / 94.4 | 22.5 / 94.8 | 22.400 ± 1.253 | 94.733 ± 0.306 |
| Safe Delta | 9.7 / 94.8 | 9.9 / 94.8 | 9.5 / 94.8 | 9.700 ± 0.200 | 94.800 ± 0.000 |
| Safety at One Shot | 19.4 / 94.4 | 19.2 / 94.4 | 18.4 / 94.6 | 19.000 ± 0.529 | 94.467 ± 0.115 |
| LED-Merging | 3.2 / 91.8 | 2.9 / 91.8 | 2.9 / 91.6 | 3.000 ± 0.173 | 91.733 ± 0.115 |
| Panacea | 9.3 / 91.8 | 14.6 / 93.4 | 19.9 / 93.6 | 14.600 ± 5.300 | 92.933 ± 0.987 |
| AsFT | 2.0 / 92.2 | 2.0 / 92.4 | 2.2 / 92.0 | 2.067 ± 0.115 | 92.200 ± 0.200 |
| CCTR | 0.4 / 92.6 | 0.3 / 92.6 | 0.4 / 92.6 | 0.367 ± 0.058 | 92.600 ± 0.000 |
CCTR achieves the lowest HS for every seed, with a mean of 0.367% ± 0.058%, versus 2.067% ± 0.115% for AsFT, while retaining comparable task accuracy. The main safety gain is therefore consistently reproduced across independent training runs.
(2) Cross-task improvement and paired uncertainty on SORRY-Bench.
We extend the SST-2 evaluation in Appendix F.2.1, Table 10 of the original submission (Table 11 in the revised manuscript), to AG News and GSM8K to test whether the safety benefit persists with an independent request set and judge. We use the official 440 requests and fine-tuned Mistral judge, with common generation and scoring settings within each task. The table reports counts of unsafe fulfillments; every column has denominator 440, and lower is better.
| Method | SST-2 | AG News | GSM8K |
|---|---|---|---|
| Attacked | 68 | 68 | 94 |
| Safe LoRA | 68 | 69 | 86 |
| SafeMERGE | 68 | 69 | 89 |
| Safe Delta | 64 | 64 | 64 |
| Safety at One Shot | 68 | 66 | 86 |
| LED-Merging | 63 | 62 | 67 |
| Panacea | 62 | 59 | 137 |
| AsFT | 67 | 65 | 63 |
| CCTR | 55 | 58 | 54 |
Author Response to Reviewer sWwN (Part 1/5)
Response to Reviewer sWwN
Thank you for recognizing our trajectory-term controls, utility ablations, and main results. The central contribution of this work is identifying and addressing cross-trajectory functional discrepancies caused by harmful fine-tuning. Figure 1 shows that harmful fine-tuning changes not only model parameters but also hidden representations propagated across layers, making parameter-only corrections insufficient to recover the full functional differences. Figure 4 further demonstrates that the contribution of trajectory information cannot be explained by simple rescaling or larger updates. Motivated by these findings, we propose CCTR, addressing the limitations of parameter-only repair that overlooks representation-trajectory divergence, without requiring additional fine-tuning, backpropagation, or downstream-task data. The complete repair pipeline takes only 6.76 minutes on average. CCTR achieves state-of-the-art safety repair performance among the evaluated methods, reducing average HS from the strongest baseline's 2.30% to 0.38% across harmful-data ratios and from 2.85% to 0.77% across downstream tasks, while improving average task accuracy by 3.53 percentage points. We further address full-model repair behavior, stability across independent training runs, and safety–utility preservation below.
Below, Harmful Score (HS) denotes the harmful-response rate, Acc denotes task accuracy, and UFR denotes the official SORRY-Bench unsafe fulfillment rate.
Q1: How does frozen-trajectory repair compare with block-wise activation recomputation, and how far do deployed representations deviate from the original attacked model’s frozen trajectory?
Frozen-trajectory repair achieves both lower harmfulness and smaller changes in benign representations than block-wise activation recomputation across the three tasks. HS falls from 6.7% to 0.4% on SST-2, from 3.5% to 0.1% on AG News, and from 26.2% to 1.2% on GSM8K. Layer-wise input-drift analysis on SST-2 shows smaller utility-input drift with frozen repair than with block-wise activation recomputation at the reported middle-to-late blocks, supporting better control of benign-representation changes accumulated with depth. Realized correction cosines with the original target are 0.751–0.774, versus 0.335–0.472 for the block-wise method, showing better directional alignment with the safety target. Old/new target cosines of 0.063–0.139 also indicate that activation recomputation changes both the inputs and residuals. We address full-model behavior, layer-wise input drift, realized output effects, and target changes below.
(1) Full-model comparison after all patches are deployed.
We implement block-wise activation recomputation on three downstream tasks to test whether new activations from the repaired prefix improve repair. Before solving each Transformer block, this control recomputes activations and statistics for its seven target modules, then writes the block's patches together. CCTR instead uses the frozen reference and attacked trajectories in Section 3.4 and Algorithm 1. Both methods share calibration data, solver settings, and evaluation protocols. All behavioral metrics come from full-model generation after all 224 patches are deployed, not from local fitting on frozen activations.
The following matched comparison uses p=0.2; HS and Acc are both in %.
The frozen method achieves lower HS on all three tasks while retaining substantial task performance. On GSM8K, HS is 1.2% versus 26.2%, with task accuracy of 24.2% versus 22.4%, showing that the safety benefit carries through to generation by the fully repaired model.
(2) Effects of earlier repairs on later-block inputs.
We directly record hidden states at each decoder-block entrance in the SST-2 models above to quantify deviations of the deployed trajectory from the frozen trajectory. Both methods use the same 128 held-out safety prompts and 128 held-out task-agnostic utility prompts, with fixed tokens, positions, and validity masks, and the original attacked model as a common reference. For each block, we sum the squared magnitudes of the hidden-state differences over all examples and valid tokens, divide by the corresponding sum for the original attacked hidden states, then take the square root and express the result as a percentage.