- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies whether low-rank fine-tuning can improve refusal-based safety alignment while preserving reasoning ability. Specifically, the authors focus on refusing direct harmful requests and theoretically analyze the tension between adapting for safety and preserving utility. They find that refusal-based safety alignment can be achieved effectively with low-rank weight changes, whereas full fine-tuning induces high-rank changes and substantially degrades reasoning.
Critical strengths
- The paper studies and demonstrates the safety–reasoning trade-off across multiple models and benchmarks.
- The paper provides useful practical guidance on choosing appropriate LoRA settings.
Critical weaknesses
- LoRA and full fine-tuning use the same learning rate, while fixed alpha changes alpha/r across ranks. It is unclear whether the observed advantage comes from rank or differences in update strength.
- Theorem 3.3’s base-preservation guarantee does not hold under its stated assumptions. A rank-one update can perfectly fit the fine-tuning task while completely forgetting the base task, even with zero rank overshoot. The proof relies on an incorrect rank-additivity claim in Lemma D.3.
Questions for authors
see weakness
3 comments
Dear Reviewer,
Thank you again for your review of our work. We have posted a detailed response addressing your points, including a step-by-step clarification of the correctness of Theorem 3.3 (the rank-one example and Lemma D.3), and new experiments on learning-rate sensitivity and LoRA scaling, including a rank comparison with α/r held fixed and a learning-rate sweep for full fine-tuning.
Given the fast-approaching discussion deadline, please let us know if any questions remain or if we can clarify anything further in the remaining time. If our responses address your concerns, we would be grateful if you would consider updating your score.
Best regards, The Authors
Thank you for the additional experiments and explanation, which address my concern. I will increase my score.
We thank the reviewer for the careful reading and constructive feedback. Below, we address the theoretical questions and present additional experiments controlling for LoRA scaling.
Q1. Theorem 3.3 Correctness
We respectfully disagree with the reviewer’s claim that Theorem 3.3’s base-preservation guarantee fails under its stated assumptions. The supposed counterexample and the attack on Lemma D.3 are both incorrect, and we address each in turn.
On the rank-one counterexample. The reviewer claims that a rank-one update can perfectly fit the fine-tuning task while completely forgetting the base task at zero overshoot. This actually contradicts the theorem rather than instantiating a counterexample to it: when Δ = r − r_ft = 0, Theorem 3.3 predicts L(W_lora, T_base) ≤ C · Δ/intdim(Θ_base) + o(1) = o(1), i.e., base-task preservation is guaranteed, not violated. The underlying reason is that with r = r_ft, the entire LoRA rank budget is consumed fitting Θ_ft, leaving no remaining degrees of freedom to disrupt Θ_base — forgetting only enters through overshoot Δ > 0. A rank-one update with zero overshoot thus falls precisely in the regime where the theorem provides its strongest guarantee.
On Lemma D.3. The rank-additivity claim rank(S) = rank(PS) + rank(P⊥S) is a standard consequence of orthogonal projection geometry. Since P and P⊥ = I − P project onto complementary, mutually orthogonal subspaces, the column spaces of PS and P⊥S lie in range(P) and range(P⊥) respectively, which intersect only at {0}. It follows directly that col(S) = col(PS) ⊕ col(P⊥S), and hence the rank splits additively. There is no error here.
On the proof of Theorem 3.3. The proof proceeds by first establishing via Assumption (A3) that the rank-allocation optimizer in Lemma D.4 satisfies q* = r_ft, ensuring the full rank budget is allocated to fitting Θ_ft. The residual base-task loss is then bounded by applying the Eckart–Young theorem to yield s²_base · Δ · ‖Θ_base‖²_2, which after normalization gives the stated C · Δ/intdim(Θ_base) bound. Each step is valid, and we are happy to walk through the proof in further detail if the reviewer finds it helpful.
Q2. Learning-rate sensitivity and LoRA update scaling
Thank you for pointing this out. We agree that fixing α while varying rank changes the LoRA scaling factor α/r. To directly examine this effect, we ran a new Llama-8B rank comparison with α/r fixed at 4: r=1 with α=4, r=4 with α=16, and r=64 with α=256. All three configurations were trained from the same base model using ordinary refusal SFT, with identical data, MLP target modules, learning rate 5e-5, seed 42, effective batch size 4, and a ten-epoch training schedule. We also independently swept Full fine-tuning learning rates, as reported in our response to Reviewer 2, Q1.
The table reports all evaluated checkpoints. Each entry is clean safety / AIME, in percentages; higher is better for both metrics.
With α/r fixed at 4, rank 4 achieves the highest clean safety and ties for the highest AIME score among the three configurations at epoch 8: 98.40% and 56.67%, respectively. This result supports r=4 as a practical choice for balancing safety and reasoning under a common scaling factor.