- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies emergent misalignment during narrow harmful fine-tuning of instruction-tuned LLMs. It proposes token-level directional Hessian curvature and gradient-subspace overlap diagnostics, then uses a rank-4 harmful-gradient subspace to ablate or amplify components of a single-layer rank-1 LoRA update. The experiments span several model families and three harmful adaptation domains, with free-generation and teacher-forced evaluations of the intervention.
Critical strengths
-
The intervention design is genuinely controlled. Both ablation and amplification are tested, with norm-matched Haar-random directions as controls — much stronger than a one-sided "removal improves safety" result, and Figure 7 shows the two directions move behavior in opposite ways.
-
Figure 3 establishes a real empirical fact.All-layer LoRA produces far more behavioral EM than the single-layer model organism, which is informative in its own right and honestly frames what the rest of the paper can and cannot claim.
Critical weaknesses
-
The core equations are broken as written.In Eq. (1) both terms use the same checkpoint, so δ ≡ 0 and the pivot/neutral token sets can't be constructed; Section 3.1 likewise defines Δφ = φ_s − φ_s. Since pivot selection feeds the curvature plots and the intervention, this isn't cosmetic — the method is non-reproducible until the authors confirm what was actually run.
-
The mitigation doesn't address the problem it motivates.The paper is motivated by all-layer EM (up to 28.5% in Figure 3), but every intervention acts on a single LoRA module where free-generation EM is already near the floor. The "up to 80% reduction" is relative to that low baseline, not to the all-layer rates that justify the paper.
-
The geometry is coordinate-dependent.Projections happen in concatenated (A, B) factor space, but the effective update is bilinear (ΔW ∝ BA) with scale/reparameterization symmetries. An orthogonal direction in factor space need not mean anything in weight or function space, which undermines the "causal removal of a harmful mechanism" interpretation.
Questions for authors
Same as Weakness
Official Review of Submission11652 by Reviewer 7yHF
Thank you for the detailed rebuttal and substantial revisions. I appreciate the additional experiments, particularly the pivot-selection controls, all-layer intervention results, and comparisons with shrinkage and rollback baselines. These additions have largely addressed my concerns and strengthened the paper's empirical contributions. I also appreciate the authors' clarification of the scope of their claims, while the causal interpretation of passive decoupling (W2) and the safety–utility trade-off under meaningful downstream adaptation (W4) remain interesting directions for future investigation. I have decided to raise my rating accordingly.