acceptodds
Under review as a conference paper at ICLR 2027

ASCENT: First-Order Optimal Fine-Tuning with Recalibration for Safety–Utility Co-Enhancement

Abstract

Supervised fine-tuning can substantially improve the downstream utility of large language models (LLMs) but can also compromise their safety capabilities. Recent safety-preserving fine-tuning methods aim to prevent safety degradation by identifying safety-related parameters or subspaces and constraining downstream updates accordingly. However, existing methods still have several limitations. First, these methods mainly focus on safety preservation, rather than seeking joint safety and utility enhancement. Second, there is no theoretical characterization of the optimal safety-related subspace or the safety-preserving task update. Third, existing methods typically rely on a static safety subspace throughout downstream fine-tuning, even though this subspace may be outdated as the LLM evolves. To address these limitations, we propose ASCENT, a downstream fine-tuning framework for safety–utility co-enhancement through first-order optimal safety-aware periodic calibration and task optimization. Specifically, we model safety as a function of LLM parameters and use its first-order approximation to characterize how safety changes under parameter updates. Under a fixed rank and Frobenius-norm budget, we prove that the parameter update constructed from the top- singular components of the safety-function gradient maximizes the estimated safety change, and use this update as a periodic calibration to preserve and even improve the safety capability. We further formulate an optimization problem for the safety-preserving task update that minimizes its deviation from the original task update while penalizing its negative effect on the estimated safety change, which yields a unique solution for the task optimization process. As the optimization proceeds, ASCENT iteratively applies the optimal safety-preserving task update and the optimal safety-improving calibration update, thereby enabling the co-enhancement of safety and utility. Extensive experiments across multiple LLM families and downstream tasks demonstrate that ASCENT can improve downstream utility by up to 20.3% and reduce attack success rate by up to 35.5%. Compared with existing safety-preserving fine-tuning methods, ASCENT achieves state-of-the-art safety and utility across all evaluated settings. Our code is available at https://anonymous.4open.science/r/ASCENT-Code-8E96/.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Safety–Utility Co-Enhancement, First-Order Optimal Safety Calibration, Safety-Preserving Task Optimization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 4 reviews, above 85% of the others.
Reviewer_TD9pOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ASCENT, a fine-tuning method for improving both model safety and downstream task performance. ASCENT derives a low-rank safety calibration update from the safety gradient and selectively suppresses task updates that negatively affect safety. The safety-related subspace is periodically recalibrated during fine-tuning.

Critical strengths

1.ASCENT preserves safety-compatible components and selectively suppresses safety-degrading ones, providing a more fine-grained approach to balancing safety and downstream utility. 2.The experiments cover four LLMs of different architectures and scales and four downstream tasks, with additional evaluations on safety benchmarks and jailbreak attacks, providing strong empirical support for the method.

Critical weaknesses

1.ASCENT introduces additional safety optimization during downstream fine-tuning, while baselines such as SafeLoRA and GuardSpace mainly focus on safety preservation. The paper lacks a comparison with conventional safety alignment fine-tuning under comparable safety supervision. It remains unclear whether ASCENT offers advantages over standard safety alignment fine-tuning. 2. Safety evaluation mainly relies on ASR, without examining whether the safety improvements lead to increased over-refusal on benign requests.

Questions for authors
  1. Table 1 shows substantial ASR reductions compared with Original. Does ASCENT increase over-refusal on benign requests?
  2. How does ASCENT perform on domain-specific instruction-following tasks, such as legal or medical tasks?
  3. Could the authors report additional safety metrics beyond ASR to provide a more comprehensive evaluation?
6 comments
AuthorsOpenReview ↗

Response to Reviewer TD9p

We thank Reviewer TD9p for the valuable comments. Below are our responses to the questions.

1. Comparison with Conventional Safety Fine-Tuning

ASCENT aims to improve safety and downstream utility jointly. In addition to the safety-supervision-matched controls in Appendix F, we add a conventional Task FT + Safety SFT baseline using the same datasets and downstream training settings as ASCENT.

Across both tasks and all four models, Safety SFT reduces ASR but lowers accuracy relative to Task FT. ASCENT achieves substantially higher accuracy at comparable ASR, supporting the benefit of controlling interference between safety and task updates. The table reports accuracy / ASR (%). We will include this comparison in the revised paper.

ModelTaskTask FTTask FT + Safety SFTASCENT
Qwen2.5-7B-itAG News92.0 / 44.088.8 / 4.593.2 / 5.0
Qwen2.5-7B-itGSM8K89.6 / 43.085.4 / 6.591.2 / 6.5
Llama-3.1-8B-itAG News94.0 / 39.090.8 / 2.095.0 / 2.5
Llama-3.1-8B-itGSM8K86.0 / 29.081.4 / 8.586.6 / 9.5
Gemma-3-12B-itAG News93.0 / 42.089.6 / 4.594.2 / 4.0
Gemma-3-12B-itGSM8K92.0 / 40.588.0 / 1.593.6 / 1.5
Qwen3-30B-A3B-itAG News93.4 / 29.590.6 / 9.094.6 / 10.0
Qwen3-30B-A3B-itGSM8K96.0 / 30.092.0 / 10.596.6 / 11.5

2. Safety Evaluation and Over-Refusal

We compare the original models with ASCENT after AG News fine-tuning. Beyond ASR, we report Harmful Score (HS), the mean StrongREJECT score for responses to the same 200 JBB adversarial prompts. We also evaluate over-refusal rates (ORR) on 250 safe XSTest prompts and 250 randomly sampled benign OR-Bench prompts, using the same evaluation sets across models.

ASCENT reduces ASR and HS while keeping over-refusal rates close to those of the original models.

ModelMethodASR (%)↓HS (0–1)↓XSTest ORR (%)↓OR-Bench ORR (%)↓
Qwen2.5-7B-itOriginal39.50.326.03.6
Qwen2.5-7B-itASCENT5.00.048.86.0
Llama-3.1-8B-itOriginal24.00.198.84.8
Llama-3.1-8B-itASCENT2.50.0212.88.0
Gemma-3-12B-itOriginal37.00.306.44.0
Gemma-3-12B-itASCENT4.00.038.05.2
Qwen3-30B-A3B-itOriginal25.50.2112.48.8
Qwen3-30B-A3B-itASCENT10.00.0716.011.6

3. Medical Domain Evaluation

We evaluate ASCENT and the baselines on MedQA using 2,000 training examples and 500 test questions sampled from the official splits. The other fine-tuning settings remain unchanged from the main experiments, and ASR is evaluated on JailbreakBench. Across all four models, ASCENT achieves the highest MedQA accuracy and the lowest ASR among the compared methods. These results extend the safety–utility improvements observed in the main experiments to medical-domain fine-tuning.

ModelMethodMedQA Acc (%)↑JBB ASR (%)↓
Qwen2.5-7B-itOriginal59.639.5
SafeLoRA62.242.0
SaLoRA62.643.0
GuardSpace63.441.0
SPA62.442.5
ASCENT66.88.0
Llama-3.1-8B-itOriginal61.024.0
SafeLoRA63.228.5
SaLoRA64.031.0
GuardSpace64.627.0
SPA63.628.0
ASCENT67.85.5
Gemma-3-12B-itOriginal60.437.0
SafeLoRA63.039.5
SaLoRA63.841.0
GuardSpace63.638.5
SPA62.640.0
ASCENT68.47.0
Qwen3-30B-A3B-itOriginal79.625.5
SafeLoRA82.029.0
SaLoRA82.430.0
GuardSpace83.428.0
SPA81.829.5
ASCENT85.67.5

We hope our responses and revisions to the manuscript could address your concerns. We would be happy to answer any further questions.

Thank you for your response. I have two follow-up questions:

  1. Could you clarify how the "conventional Task FT + Safety SFT" baseline is implemented? LISA [1] mitigates safety degradation during downstream fine-tuning by alternating between safety-alignment data and downstream task data and introducing a proximal regularization term to constrain parameter shifts. I suggest including LISA as an additional baseline for comparison.

  2. You evaluated ASCENT on 250 benign prompts randomly sampled from OR-Bench and showed that its over-refusal rate remains close to that of the original model. Could you also evaluate the fine-tuned model on general-purpose benchmarks such as MMLU? I would like to know whether ASCENT leads to any noticeable performance drop compared to the original model.

[1] Lazy Safety Alignment for Large Language Models Against Harmful Fine-Tuning.

Thank you for your response. I have two follow-up questions:

  1. Could you clarify how the "conventional Task FT + Safety SFT" baseline is implemented? LISA [1] mitigates safety degradation during downstream fine-tuning by alternating between safety-alignment data and downstream task data and introducing a proximal regularization term to constrain parameter shifts. I suggest including LISA as an additional baseline for comparison.

  2. You evaluated ASCENT on 250 benign prompts randomly sampled from OR-Bench and showed that its over-refusal rate remains close to that of the original model. Could you also evaluate the fine-tuned model on general-purpose benchmarks such as MMLU? I would like to know whether ASCENT leads to any noticeable performance drop compared to the original model.

[1] Lazy Safety Alignment for Large Language Models Against Harmful Fine-Tuning.

Reviewer_CzB1OpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary
  • Large language models are aligned to refuse harmful requests, but they may lose this ability after fine-tuning on downstream tasks. Existing safety-preserving fine-tuning methods address this by identifying safety-related parameters or subspaces before training and keeping them fixed.
  • This paper proposes ASCENT, which improves safety during fine-tuning rather than only preserving it, while also improving task performance. It consists of three stages.
    • In the safety calibration stage, a safety judge scores model responses, and these scores are used to estimate the safety gradient. Under a first-order approximation, the update that maximizes the estimated safety improvement under constraints on its magnitude and rank is given by the top-rr singular components of the safety gradient matrix. This update is applied to the weights, and the same components are used as the safety-related directions in the later stages.
    • In the selective suppression stage, the weight changes from task training are decomposed along the safety-related directions, and only the components that reduce safety are suppressed.
    • In the periodic recalibration stage, calibration is repeated at regular intervals because these directions can shift as training progresses.
  • Experiments cover four tasks: dialogue summarization, news classification, mathematical reasoning, and question answering. The models are four instruction-tuned LLMs: Qwen2.5-7B, Llama-3.1-8B, Gemma-3-12B, and Qwen3-30B-A3B. The baselines are SafeLoRA, SaLoRA, GuardSpace, and SPA. None of them reduces the attack success rate below that of the original models in any setting, while ASCENT does and also achieves higher task performance than the baselines.
Critical strengths

1. The method is well motivated, and both the theory and the experiments support its design.

  • The authors derive the optimal updates for safety calibration and selective suppression in closed form under a first-order approximation (Theorems 1 and 2).
  • The predicted and measured safety changes are strongly correlated under the tested perturbations (Table 5).
  • The gains cannot be explained by repeated safety supervision alone. With the same safety data and calibration budget, the controlled variants Full-Grad and Cal.+FT become safer but lose task performance, while ASCENT improves both (Table 8).

2. The results are consistent across models, tasks, and evaluation settings.

  • ASCENT achieves higher task performance and lower attack success rates than all baselines across four models and four tasks (Table 1).
  • The same trend holds under additional jailbreak attacks in the GSM8K setting (Figure 3, Table 6) and with an alternative evaluation judge (Table 7).
  • The same main hyperparameters are used in all settings, so the results do not depend on per-setting tuning (Appendix B).
Critical weaknesses

1. It is unclear whether the low attack success rates reflect appropriate refusals rather than over-refusal or degraded outputs.

  • Since ASCENT directly optimizes a safety judge's score, these alternatives cannot be ruled out. Results with judges other than the training judge (Tables 1 and 7) do not address this concern, since neither evaluation separately measures response coherence or over-refusal.
  • Strong task performance rules out refusing everything, but not refusing benign requests that superficially resemble harmful ones.
  • The JailbreakBench evaluation excludes benign examples, and no response examples are provided (Appendix D).

2. The paper analyzes the sensitivity of performance only to the calibration interval (Table 3).

  • The calibration rank (r=32r=32) and the number of prompts for gradient estimation (50) are fixed.
  • The rank sets how many directions are used for calibration and suppression, and the gradient estimate may depend on the number and source of prompts.
  • Results with other ranks and prompt sets are needed to see how sensitive ASCENT is to these choices.
Questions for authors

1. It would help to know whether the safety improvement of ASCENT comes from appropriate refusals. Could you report the following for the original models and ASCENT?

  • Example responses to harmful and benign prompts, with harmful content redacted if necessary, or the proportion of coherent refusals versus degraded outputs among responses counted as unsuccessful attacks.
  • Refusal rates on benign JailbreakBench examples, XSTest, and OR-Bench.

2. Could you report results for repeated calibration with fixed safety-related directions to separate the effects of repeating calibration and updating the directions?

Recalibration changes two things at once: calibration is repeated, and the safety-related directions are updated. Table 2 compares +Task Opt. (single calibration, fixed directions) with ASCENT (repeated calibration, updated directions), so the two effects cannot be separated. Figure 4 shows that the directions change during training, but not whether this matters for safety. Comparing such a variant with +Task Opt. would show the effect of repeating calibration, and comparing it with ASCENT would show the effect of updating the directions.

  • Task performance and attack success rates of a variant that keeps the initial safety-related directions fixed. At each calibration step, it recomputes the safety gradient and projects it onto these directions for calibration and selective suppression. All other settings, including the calibration schedule, calibration budget, and suppression strength, match ASCENT.
  • Attack success rates over fine-tuning steps for this variant and ASCENT.

3. How were the calibration rank (r=32r=32) and the 50 prompts for gradient estimation chosen?

Task performance and attack success rates under the following settings would show how sensitive ASCENT is to these choices.

  • Calibration ranks of 8, 16, and 64.
  • Different numbers of prompts for gradient estimation.
  • Prompts from another source, such as AdvBench instead of HarmBench.

Additional Feedback

  • Figure 2 largely visualizes the results in Table 1 as changes relative to the original models. This space would be better used for attack success rates over fine-tuning steps or refusal rates on benign requests.
2 comments
AuthorsOpenReview ↗

Response to Reviewer CzB1 (1/2)

We thank Reviewer CzB1 for the valuable comments. Below are our responses to the questions.

1. Response Quality and Over-Refusal

We reviewed all responses counted as unsuccessful attacks using LLM-assisted annotation and manual inspection. Among the audited responses, 99.4% were coherent refusals. For example, Gemma responded: “I cannot fulfill this request as it involves illegal activities.” The remaining 0.6% were degraded outputs.

Our ASR evaluation uses the 200 adversarial prompts from JBB’s "judge_comparison" split (Appendix D). The 100 excluded benign prompts are included in the 250 safe XSTest prompts evaluated here. We also evaluate 250 randomly sampled benign OR-Bench prompts, using the same evaluation sets across models. We report over-refusal rates (ORR, %, lower is better) for the original models and ASCENT after SAMSum and AG News fine-tuning below. The additional over-refusal results are reported below. Representative responses are provided in Appendix D of the revised manuscript.

ModelOriginal XSTestOR-BenchSAMSum XSTestOR-BenchAG News XSTestOR-Bench
Qwen2.5-7B-it6.03.68.04.88.86.0
Llama-3.1-8B-it8.84.812.06.812.88.0
Gemma-3-12B-it6.44.06.45.28.05.2
Qwen3-30B-A3B-it12.48.815.211.216.011.6

2. Calibration Rank and Prompt Sensitivity

We evaluate calibration ranks r∈{8,16,32,64}r\in\{8,16,32,64\} and gradient-estimation prompt counts N∈{10,25,50,100}N\in\{10,25,50,100\} on AG News and GSM8K using Llama-3.1-8B-it and Qwen3-30B-A3B-it. We vary one factor at a time, keeping all other settings unchanged, including the task LoRA rank of 32. We also compare HarmBench with AdvBench (excluding prompts overlapping with evaluation goals), fixing r=32r=32 and N=50N=50.

Across both tasks, higher ranks yield modest utility gains while ASR remains broadly stable. Increasing the prompt count reduces ASR with little change in utility, while the additional gains from 50 to 100 prompts are limited. HarmBench and AdvBench produce comparable results. These results support r=32r=32 and N=50N=50 as practical defaults that balance safety, utility, and calibration cost. We will include this analysis in the revised paper.

Calibration rank (r∈{8,16,32,64}r\in\{8,16,32,64\}; N=50N=50; HarmBench)

rTaskLlama-3.1-8B-it Acc↑ASR↓Qwen3-30B-A3B-it Acc↑ASR↓
8AG News94.43.093.811.5
GSM8K85.810.095.612.0
16AG News94.83.594.49.5
GSM8K86.010.096.211.0
32AG News95.02.594.610.0
GSM8K86.69.596.611.5
64AG News95.23.094.810.0
GSM8K87.210.097.012.0

Prompt count (N∈{10,25,50,100}N\in\{10,25,50,100\}; r=32r=32; HarmBench)

NTaskLlama-3.1-8B-it Acc↑ASR↓Qwen3-30B-A3B-it Acc↑ASR↓
10AG News94.85.094.412.5
GSM8K86.412.096.413.0
25AG News95.03.594.610.5
GSM8K86.610.596.412.5
50AG News95.02.594.610.0
GSM8K86.69.596.611.5
100AG News95.22.094.89.5
GSM8K86.89.096.411.0

Prompt source (HarmBench vs. AdvBench; r=32r=32; N=50N=50)

SourceTaskLlama-3.1-8B-it Acc↑ASR↓Qwen3-30B-A3B-it Acc↑ASR↓
HarmBenchAG News95.02.594.610.0
GSM8K86.69.596.611.5
AdvBenchAG News94.43.094.211.5
GSM8K86.811.096.412.0

Accuracy and ASR are in %.

AuthorsOpenReview ↗

Response to Reviewer CzB1 (2/2)

3. Repeated Calibration and Direction Updates

To separate the two effects, we add a repeated-calibration variant with fixed directions (Fixed). It retains the initial safety-related directions throughout fine-tuning, but recomputes the current safety gradient at each calibration point and projects it onto these directions for both calibration and selective suppression. Its calibration schedule, calibration budget, suppression strength, and other training settings match ASCENT. We evaluate this variant on AG News and GSM8K using Llama-3.1-8B-it and Qwen3-30B-A3B-it.

Compared with +Task Opt. (single calibration with fixed directions), Fixed reduces ASR. ASCENT further improves utility and reduces ASR by refreshing the directions, showing the benefit of keeping calibration and task protection aligned with the current model. We report final accuracy / ASR and ASR trajectories at matched training steps below (all metrics in %) and will include them in the revised paper. Step 0 denotes the original model before initial calibration.

Final performance on AG News and GSM8K

ModelTask+Task Opt.Repeated Cal., Fixed DirectionsASCENT
Llama-3.1-8B-itAG News94.4 / 7.594.6 / 5.595.0 / 2.5
Llama-3.1-8B-itGSM8K86.0 / 13.086.2 / 11.586.6 / 9.5
Qwen3-30B-A3B-itAG News93.6 / 13.594.0 / 12.094.6 / 10.0
Qwen3-30B-A3B-itGSM8K96.0 / 15.596.2 / 13.596.6 / 11.5

ASR over fine-tuning steps

StepTaskLlama Fixed↓Llama ASCENT↓Qwen Fixed↓Qwen ASCENT↓
0AG News24.024.025.525.5
0GSM8K24.024.025.525.5
16AG News9.59.017.016.5
16GSM8K16.015.519.018.5
32AG News8.06.015.014.0
32GSM8K14.013.017.016.0
48AG News7.04.513.512.0
48GSM8K13.011.515.514.0
64AG News6.03.512.511.0
64GSM8K12.010.014.512.5
80AG News5.52.512.010.0
80GSM8K11.59.513.511.5

We hope our responses and revisions to the manuscript could address your concerns. We would be happy to answer any further questions.

Reviewer_CXE4OpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ASCENT, a safety-preserving fine-tuning framework that aims to jointly improve downstream utility and model safety. ASCENT derives a low-rank safety calibration update using the top singular components of the safety gradient and selectively suppresses task updates that negatively affect safety. It further introduces periodic recalibration to adapt the safety-related subspace during fine-tuning. Experiments across four LLMs and four downstream tasks show improvements in both safety and utility over existing baselines.

Critical strengths
  1. The paper is well-organized, and the proposed method is clearly presented with theoretical derivations.

  2. ASCENT achieves consistent safety–utility co-enhancement across all 16 model–task settings, while the compared baselines generally improve utility at the cost of safety. The improvements are substantial, with ASR reductions of up to 35.5 percentage points.

  3. The paper includes component ablations and analyses of calibration intervals to examine the effectiveness of the proposed design.

Critical weaknesses
  1. The methodological novelty is limited. SPARD[1] and SafeGrad[2] have already explored safety-constrained fine-tuning through gradient projection and conflict resolution. ASCENT follows similar ideas but lacks direct comparisons with these methods.

[1] SPARD: Defending Harmful Fine-Tuning Attack via Safety Projection with Relevance–Diversity Data Selection, ICML 2026. [2] SafeGrad: Gradient Surgery for Safe LLM Fine-Tuning, ICASSP 2026.

  1. Theorem 1 maximizes the estimated safety improvement under rank and norm constraints, while Theorem 2 minimizes deviation from the original task update with a safety penalty. However, neither guarantees improvements in the actual safety–utility trade-off. It remains unclear how these theoretical results justify the claimed optimality of the overall fine-tuning framework.

  2. The effectiveness of selective suppression is unclear. With the default λ=1\lambda=1 and r=32r=32, Eq. (13) may only weakly suppress conflicting updates. The paper lacks analysis of the actual suppression magnitude.

  3. ASCENT requires additional safety-gradient estimation, SVD, and periodic recalibration, but provides no detailed runtime or memory comparison.

  4. The training scale is limited. All experiments use only 2,000 samples and 80 optimization steps, which is insufficient to establish the effectiveness of periodic recalibration under longer fine-tuning schedules.

Questions for authors
  1. Eq. (13) only weakly suppresses conflicting updates under the default (\lambda=1). Could the authors show how different values of (\lambda) affect the safety–utility trade-off?
7 comments
AuthorsOpenReview ↗

Response to Reviewer CXE4 (1/2)

We thank Reviewer CXE4 for the valuable comments. Below are our responses to the questions.

1. Novelty and Comparison with SPARD and SafeGrad

We thank the reviewer for this helpful comment. (a) SPARD preserves safety by correcting task updates that violate a safety threshold. However, it does not formulate safety maximization under rank and norm budgets or derive the corresponding optimal calibration update. It also does not formulate an importance-weighted optimization problem to balance utility and safety or derive its optimal task update. (b) SafeGrad preserves alignment by resolving conflicts between task and safety gradients. However, it does not formulate task protection as minimizing deviation from the original task update with importance-weighted safety penalties, or derive the unique optimal update for that objective. It also does not derive optimal safety calibration under rank and norm budgets.

In contrast, ASCENT jointly optimizes safety-enhancing calibration and safety-preserving task updates, with closed-form optimal solutions for both first-order optimization problems. Theorem 1 maximizes safety improvement under rank and norm budgets. Theorem 2 gives the unique optimal task update that preserves task learning while selectively suppressing safety-degrading components according to their importance. ASCENT applies both derived optimal updates and periodically refreshes their shared safety-related structure throughout fine-tuning.

We additionally compare SPARD and SafeGrad on Llama-3.1-8B-it and Qwen3-30B-A3B-it under the same experimental settings. ASCENT achieves higher utility and lower ASR across these settings. We will include these comparisons and clarify the technical distinctions in the revised paper. The results are shown below (all metrics in %).

ModelMethodSAMSum U↑ASR↓AG News U↑ASR↓GSM8K U↑ASR↓OpenBookQA U↑ASR↓
Llama-3.1-8B-itOriginal28.024.078.624.082.424.077.624.0
SPARD42.816.592.815.084.217.085.416.0
SafeGrad42.326.092.425.583.825.084.826.5
ASCENT45.89.095.02.586.69.587.65.5
Qwen3-30B-A3B-itOriginal26.525.585.625.592.425.594.025.5
SPARD43.818.092.016.594.219.094.817.0
SafeGrad44.027.092.426.593.627.595.228.0
ASCENT46.811.594.610.096.611.596.62.5

U denotes ROUGE-L for SAMSum and accuracy otherwise. ASR is evaluated on JailbreakBench.

2. First-Order Optimality and Co-Enhancement

We quantify the changes in the safety score SS and task utility U=−LtaskU=-\mathcal{L}_{\mathrm{task}} induced by the updates from Theorems 1 and 2.

Safety improvement from calibration. Writing the safety gradient as G=∇WSG=\nabla_W S, substituting the optimal calibration update from Theorem 1 gives:

S(W+ΔWcal∗)−S(W)=⟨G,ΔWcal∗⟩F+O(ρ2)=ρ∥Σr∥F+O(ρ2).S(W+\Delta W_{\mathrm{cal}}^{\ast})-S(W) = \langle G,\Delta W_{\mathrm{cal}}^{\ast}\rangle_F+O(\rho^2) = \rho\lVert\Sigma_r\rVert_F+O(\rho^2).

Thus, calibration increases the safety score by ρ∥Σr∥F\rho\|\Sigma_r\|_F to first order, which is the largest possible gain under the specified rank and norm budgets.

Task improvement retained after safety protection. For a standard task-gradient step, let g=∇WUg=\nabla_W U and let g∗g^* be its protected direction from Theorem 2. With learning rate η\eta, the utility change is:

U(W+ηg∗)−U(W)=η⟨g,g∗⟩F+O(η2)=η(∥g⊥∥F2+∑i=1raigi2)+O(η2).U(W+\eta g^{\ast})-U(W) = \eta\langle g,g^{\ast}\rangle_F+O(\eta^2) = \eta\left(\lVert g_\perp\rVert_F^2+\sum_{i=1}^{r}a_i g_i^2\right)+O(\eta^2).

Here, gig_i are the task-gradient coefficients along the safety-related directions, and g⊥g_\perp is the component outside this subspace. Theorem 2 gives ai=1a_i=1 for safety-compatible components (gi≥0g_i\geq 0) and ai=1/(1+λwi2)a_i=1/(1+\lambda w_i^2) for safety-degrading components (gi<0g_i<0). Therefore, compatible and orthogonal components retain their full task-learning contribution, while conflicting components retain a positive fraction determined by their safety importance. For any nonzero task gradient, the retained first-order utility gain is strictly positive, while first-order safety degradation along conflicting directions is simultaneously suppressed (Corollary 3).

Together, these expressions quantify the safety gain from optimal calibration and the task-learning gain retained by optimal safety protection. Over fine-tuning, these changes accumulate as F(WT)−F(W0)=∑t⟨∇F(Wt),ΔWt⟩F+O(∑t∥ΔWt∥F2)F(W_T)-F(W_0)=\sum_t\langle\nabla F(W_t),\Delta W_t\rangle_F+O(\sum_t\|\Delta W_t\|_F^2), for F∈{S,U}F\in\{S,U\}. Proposition 4 bounds the error between predicted and actual safety changes, while Table 5 provides empirical support for their agreement. Table 1 and the safety-supervision-matched controls in Appendix F demonstrate the combined improvements in safety and downstream utility.

AuthorsOpenReview ↗

Response to Reviewer CXE4 (2/2)

3. Selective Suppression and λ Sensitivity

For a safety-degrading component, Eq. (13) suppresses a fraction λwi2/(1+λwi2)\lambda w_i^2/(1+\lambda w_i^2), giving stronger suppression to components with greater safety importance while preserving safety-compatible components. We report the reduction in negative first-order safety contributions, summed over protected matrices and training steps:

Suppression(\mathrm{Suppression}(\\%) = 100\left(1-\frac{\sum_{c_i<0}\sigma_i\lvert z_i^{\ast}\rvert}{\sum_{c_i<0}\sigma_i\lvert c_i\rvert}\right).

We compare λ∈0,0.5,1,5,10,20\lambda\in\\{0,0.5,1,5,10,20\\} on GSM8K using Llama-3.1-8B-it and Qwen3-30B-A3B-it, keeping all other settings unchanged. Compared with λ=0\lambda=0 and 0.50.5, the default λ=1\lambda=1 achieves higher accuracy and lower ASR on both models, demonstrating the practical effectiveness of selective suppression. Beyond λ=1\lambda=1, stronger suppression reduces task accuracy, while ASR changes only slightly. These results support λ=1\lambda=1 as a balanced default for safety improvement and downstream utility. The results are shown below.

λLlama-3.1-8B-it Acc↑ASR↓SuppressionQwen3-30B-A3B-it Acc↑ASR↓Suppression
079.618.00.089.018.00.0
0.584.414.04.794.216.04.0
186.69.58.896.611.57.6
584.89.030.794.811.027.5
1083.28.545.193.210.541.4
2080.89.059.990.611.056.4

All metrics are in %. λ=0 corresponds to Cal.+FT; λ=1 is default ASCENT.

4. Runtime and Memory Cost

We evaluate Qwen3-30B-A3B on OpenBookQA using a single NVIDIA H200 under matched training settings. ASCENT has comparable runtime to the safety-preserving baselines and is faster than SaLoRA, while achieving higher downstream utility and lower ASR (Table 1). The table reports total runtime and sampled peak RAM/VRAM.

For each safety prompt, gradient estimation requires one first-order backward pass through the target model. Low-rank SVD is performed on selected safety-gradient matrices only at periodic calibration checkpoints, and the resulting singular components are reused for subsequent task updates. Calibration and task correction use closed-form matrix operations without an inner training loop. These design choices keep the additional computation manageable. We use 50 prompts to improve gradient-estimation quality. Reducing this number can further shorten estimation time. ASCENT also avoids SafeLoRA’s requirement for paired base/aligned checkpoints, making it usable without access to such model pairs.

MethodTotal time (min)↓Peak RAM (GiB)↓Peak VRAM (GiB)↓
SafeLoRA148.025.7259.49
SaLoRA173.1010.7859.53
GuardSpace153.9329.4159.90
ASCENT163.046.3281.19

5. Larger-Scale and Longer Fine-Tuning

We use 2,000 training samples and 80 optimization steps following prior work (Hsu et al., 2024; Zhang et al., 2026; Qi et al., 2026), as described in Appendix B. We extend AG News fine-tuning to 8,000 training samples and 320 optimization steps on Llama-3.1-8B-it and Qwen3-30B-A3B-it, keeping the recalibration interval, test sets, and other hyperparameters unchanged. We also compare with SPA under matched training budgets. ASCENT achieves higher accuracy and lower ASR under the extended setting, supporting its effectiveness with more data and longer fine-tuning. The results are shown below.

ModelMethodTraining samplesStepsAcc↑ASR↓
Llama-3.1-8B-itOriginal——78.624.0
SPA2,0008093.635.5
ASCENT2,0008095.02.5
SPA8,00032094.838.0
ASCENT8,00032096.62.0
Qwen3-30B-A3B-itOriginal——85.625.5
SPA2,0008093.628.5
ASCENT2,0008094.610.0
SPA8,00032094.231.0
ASCENT8,00032096.28.5

Accuracy and ASR are in %. ASR is evaluated on JailbreakBench.

We hope our responses and revisions to the manuscript could address your concerns. We would be happy to answer any further questions.

Thanks for the additional clarification and new experiments, which I think will strengthen this paper. While the method has a large additional memory cost, the runtime seems acceptable.

I still have one question:

The rebuttal establishes a positive first-order safety gain from calibration and a positive first-order utility gain from task optimization. However, these results do not account for the effect of calibration on utility or task optimization on safety. How do these separate guarantees establish safety–utility co-enhancement for the combined update? Is there any theoretical condition under which both objectives are guaranteed to improve simultaneously?

Reviewer_rUsLOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ASCENT, a downstream fine-tuning framework for safety–utility co-enhancement through first-order optimal safety calibration, safety-preserving task optimization, and periodic recalibration. Safety calibration uses the safety-function gradient to maximize the estimated safety improvement under fixed rank and Frobenius-norm budgets. Task optimization minimizes deviation from the original task update while preserving safety-compatible components and selectively suppressing safety-degrading ones. Periodic recalibration re-estimates the top-r singular structure to update the safety-related subspace during fine-tuning. The authors derive optimal solutions under this first-order formulation and evaluate ASCENT across four LLMs and four downstream tasks. Experiments include comparisons with safety-preserving fine-tuning baselines, component ablations, safety-supervision-matched controls, and evaluations against representative jailbreak attacks. The reported results show safety–utility co-enhancement across the evaluated settings.

Critical strengths
  1. The update rules have a clear optimization-based formulation. The authors formulate low-rank safety calibration and task-update correction as explicit first-order surrogate optimization problems and derive closed-form optimal solutions. The proofs in Appendix A provide a verifiable mathematical basis for these rules. This strengthens the rationale for the proposed design and makes its intended safety effects easier to interpret.

  2. The reported safety gains have broad empirical support. The evaluation covers four LLMs and four downstream tasks, supplemented by jailbreak attacks, an alternative safety judge, and controls with matched safety-supervision budgets. These evaluations strengthen confidence in the safety improvements observed within the tested settings. The matched controls also help distinguish the benefits of the proposed design from those attributable to repeated access to safety supervision alone.

Critical weaknesses
  1. Mismatch between the stated safety objective and the implementation. Appendix C.1 defines PJP_J as the judge’s safety probability and uses it to estimate the safety gradient. However, all supplied configurations select safe-logit, so the implementation differentiates the raw safe-class logit rather than its probability. Increasing this logit does not necessarily increase the safety probability, because competing logits can also change. This changes the calibration objective and weakens the correspondence between the theoretical formulation and the implementation. The authors should reconcile these definitions and identify the configuration used to obtain the reported results.

  2. Limited evidence for stable utility superiority across all settings. Some accuracy advantages are only 0.4 percentage points. The manuscript reports neither results across independent training seeds nor confidence intervals for paired utility differences. Consequently, the reported rankings do not establish that the utility gains are stable and reproducible, particularly for the smallest differences.

  3. The figures exhibit noticeable AI-generated artifacts. This is particularly evident in the framework diagram in Figure 1.

Questions for authors
  1. Could the authors isolate the contribution of periodic structural re-evaluation by keeping the safety calibration schedule and budget unchanged, while comparing fixed and periodically refreshed task-protection subspaces?

  2. Could the authors provide paired confidence intervals for the utility differences between ASCENT and the strongest baselines, particularly for the marginal gains, and assess their stability across training seeds?

  3. Please also address the concerns raised in the Critical Weaknesses\textbf{Critical Weaknesses} section.

6 comments
AuthorsOpenReview ↗

Response to Reviewer rUsL

We thank Reviewer rUsL for the valuable comments. Below are our responses to the questions.

1. Safety Objective and Implementation

We conduct an experimental analysis and observe that the safe-logit is highly correlated with (empirical) safety probability of the responses. Specifically, we analyze responses to 400 prompts randomly sampled from WildJailbreak, and we apply softmax to the judge’s output logits and divide the components corresponding to the safe-logit into five bins. Each response is also labeled safe or unsafe by the evaluation LLM judge.

Safe-logit bins (after softmax)ResponsesResponses judged safeSafe rate (%)
[0.0, 0.2)10165.9
[0.2, 0.4)331030.3
[0.4, 0.6)422252.4
[0.6, 0.8)513976.5
[0.8, 1.0]17316595.4

ESI’s released implementation (Qi et al., 2026) uses the judge’s safe-class logit as a scalar surrogate for safety-gradient estimation. We adopt the same safe-logit setting in all reported fine-tuning experiments. In our theoretical formulation, s(y)s(y) denotes a scalar safety score, and this logit is our implementation choice. We will revise Appendix C.1 to make this definition explicit and distinguish the surrogate score from the judge’s safety probability PJP_J.

Qi et al. (2026). Towards Identification and Intervention of Safety-Critical Parameters in Large Language Models. Findings of ACL 2026.

2. Stability of Utility Gains

ASCENT achieves utility gains of at least 5% over Original in 10 of 16 model–task settings, while attaining the highest utility among the compared baselines in every setting. ASCENT also substantially reduces ASR. For example, on Gemma-3/GSM8K, ASCENT reduces ASR from 39.5% to 1.5% compared with SafeLoRA. Our default calibration interval prioritizes safety improvement. When greater downstream utility is desired, a longer interval allows more task updates between calibrations and can further improve utility, as shown in Table 3.

We have additionally evaluated representative settings, including those with the smallest utility gaps, over three independent training seeds. We report mean ± standard deviation and 95% confidence intervals for utility differences against the strongest utility baseline, pairing runs by seed. The low variability and consistently positive paired differences support stable and reproducible utility gains over the strongest compared baselines in these settings. The results are shown below.

ModelTaskBaselineBaseline utility↑ASCENT utility↑Mean difference↑95% CI
Qwen2.5-7BSAMSumGuardSpace42.7 ± 0.1042.9 ± 0.11+0.2[0.11, 0.29]
Qwen2.5-7BAG NewsGuardSpace92.6 ± 0.2093.2 ± 0.20+0.6[0.10, 1.10]
Llama-3.1-8BSAMSumGuardSpace44.5 ± 0.3045.8 ± 0.26+1.3[0.40, 2.20]
Gemma-3-12BGSM8KSafeLoRA93.1 ± 0.1293.6 ± 0.20+0.5[0.18, 0.75]
Qwen3-30B-A3BAG NewsGuardSpace94.1 ± 0.3194.9 ± 0.23+0.8[0.30, 1.30]
Qwen3-30B-A3BGSM8KSPA95.6 ± 0.2096.6 ± 0.35+1.0[0.50, 1.50]

3. Figure Presentation

We thank the reviewer for this feedback. We designed the framework structure, wrote all the equations in Figure 1, and only used AI to refine its appearance. Taking your feedback into account, we have redesigned Figure 1 and included it in the revised manuscript.

4. Effect of Subspace Updates

We compare ASCENT with a Fixed Subspace variant that retains the initial task-protection subspace throughout fine-tuning. Only the task-protection subspace is fixed; the safety-calibration directions are refreshed as in ASCENT. Both use the same periodic safety-calibration procedure, schedule, and budgets, with all other training settings unchanged. Refreshing the subspace achieves higher utility and lower ASR across all eight settings, showing the benefit of keeping task protection aligned with the current model. Although the Fixed Subspace variant has higher ASR than ASCENT, it still achieves lower ASR than the other baselines in Table 1. The results are shown below (all metrics in %).

ModelTaskFixed Subspace U↑ASR↓ASCENT U↑ASR↓
Llama-3.1-8B-itSAMSum45.113.545.89.0
AG News94.68.595.02.5
GSM8K85.813.086.69.5
OpenBookQA87.211.087.65.5
Qwen3-30B-A3B-itSAMSum45.915.546.811.5
AG News94.415.094.610.0
GSM8K95.816.096.611.5
OpenBookQA96.010.096.62.5

U denotes ROUGE-L for SAMSum and accuracy otherwise. ASR is evaluated on JailbreakBench.

We hope our responses and revisions to the manuscript could address your concerns. We would be happy to answer any further questions.

AuthorsOpenReview ↗

Official Comment by Authors

Dear Reviewer rUsL,

Thank you again for your thoughtful and constructive feedback, which we have carefully addressed in our author response. As the discussion deadline is approaching, we would greatly appreciate any further feedback you may have. If our responses have resolved your main concerns, we would also be grateful if you could update your overall assessment accordingly.

Sincerely,

The Authors

Reviewer_rUsLOpenReview ↗

Clarification on the Consistency of Three-Seed Results

Thank you for your detailed response, which has addressed most of my concerns. However, based on your clarification, I still have one remaining question that I hope you can further explain. If this concern is satisfactorily addressed, I would be happy to reconsider and potentially increase my score.

I noticed an unusual consistency between the fourth and fifth rows of the reported results. Despite involving different models, tasks, and baseline methods, these two settings report exactly the same standard deviations for both the baseline and ASCENT (0.12 and 0.20, respectively), the same mean utility improvement (+0.5), and the same 95% confidence interval ([0.18, 0.75]). Such an exact match across multiple statistical measures seems unusual for independent experimental settings and warrants further clarification.

Could you provide the individual results for all three seeds in these two settings, including the baseline and ASCENT utilities for each seed and the corresponding paired differences? Additionally, please clarify how the 95% confidence intervals were calculated and whether they were derived directly from the three seed-level paired differences. This information would help me better assess the reliability of the reported stability results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.