- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates Emergent Misalignment (EM) from a weight-space perspective, examining shared geometric structures and behavioral similarities across LoRA updates obtained from different harmful fine-tuning tasks. The authors further study the influence of data provenance, including data-generating models, generation templates, and benign fine-tuning datasets, on parameter geometry. Finally, they explore post-hoc vaccination through weight subtraction and orthonormal projection to improve model alignment while preserving reasoning capabilities.
Critical strengths
-
The paper investigates a meaningful research question. In particular, the observation that shared parameter subspaces are not unique to EM, and may also emerge from benign fine-tuning with shared data-generation templates, provides an interesting empirical perspective on the relationship between parameter geometry and misalignment.
-
The paper provides relatively comprehensive experimental analyses across different model sizes, LoRA ranks, training checkpoints, linear mode connectivity, data-provenance settings, and vaccination strategies. The authors also examine the behavioral implications of parameter geometry and consider the trade-off between alignment and reasoning performance.
Critical weaknesses
1. Distinctions from prior work and further analysis of data provenance.
The analyses of shared LoRA parameter subspaces and linear mode connectivity are closely related to prior work [1]. It would be helpful for the authors to clarify the additional contributions of this paper. Regarding data provenance, the paper considers different data-generating models and templates, but the main geometric results are reported in aggregate. Could the authors separately report the subspace similarities between GPT-4.1 and Claude Sonnet 4.5 under the same task and generation template, and compare them with results obtained using different templates but the same data-generating model? In addition, the number of different-template, same-domain comparisons is relatively limited. Further discussion of how this affects the conclusions would be useful.
2. Mechanistic interpretation and experimental comparisons for vaccination.
The paper observes similar subspace overlaps between EM and benign fine-tuning under shared templates, while projections derived from these adapters exhibit different effects on model alignment. It would be helpful to further explain this difference and clarify how the geometric analysis informs the selection of effective intervention directions. Additionally, comparisons with Safe LoRA [2] and simple LoRA scaling under matched safety–utility evaluation settings would help clarify the benefits of the proposed vaccination strategies.
References
[1] Shared Parameter Subspaces and Cross-Task Linearity in Emergently Misaligned Behavior. 2025.
https://arxiv.org/abs/2511.02022
[2] Hsu et al. (2024). Safe LoRA: The Silver Lining of Reducing Safety Risks when Finetuning Large Language Models. NeurIPS 2024.
https://arxiv.org/abs/2405.16833
Questions for authors
see Weaknesses