- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a transferable misalignment attack based on safety task vectors. The authors first construct a safety-related parameter direction by subtracting the weights of two models fine-tuned from the same pretrained base, one on general instruction data and the other on general instruction data combined with safety data. They then apply multi-seed sign consensus to remove inconsistent parameter updates. The resulting vector is subtracted from target models to weaken their safety alignment without using harmful training responses. The central claim is that a safety task vector extracted once can be transferred across multiple post-trained models derived from the same pretrained base.
Critical strengths
The paper evaluates whether a single safety task vector can weaken the alignment of multiple models derived from the same pretrained base. The transfer experiments include instruction-tuned, domain-specific, preference-optimized, and distilled models. The increase in attack success rates across these variants is an interesting empirical observation.
Critical weaknesses
- The paper motivates its approach by arguing that existing misalignment attacks must be reconstructed for each target model, whereas the proposed safety vector can be extracted once and reused. However, the threat model assumes that the attacker has full white-box access to both the target model and its pretrained base, and can freely modify the target model's parameters. Under these assumptions, it is unclear why transferability is an important practical attack capability. Since the attacker can already modify each target model directly, existing methods such as Heretic can be applied without requiring harmful training responses. The paper does not establish why repeatedly constructing target-specific attacks is a significant practical obstacle. This paper does not identify a concrete deployment scenario in which the ability to modify multiple locally accessible checkpoints with a shared vector creates a meaningful security threat beyond existing weight-editing attacks.
- The proposed framework largely combines techniques from existing work and lacks novelty.
- The paired safety-vector extraction in Equation 2 follows https://arxiv.org/abs/2511.05408
- The multi-seed sign-consensus procedure in Section 4.2 is directly based on NegMerge (https://arxiv.org/abs/2410.05583)
- The claimed advantage of cross-model transferability is not clearly distinguished from existing task-arithmetic research. Bhardwaj et al. (https://arxiv.org/abs/2402.11746) already demonstrate that safety-related parameter vectors can be reused to modify the safety behavior of models fine-tuned on different downstream tasks. This work is also missing from this paper's related work.
Questions for authors
- Can the authors identify a realistic scenario in which the attacker benefits substantially from reusing a safety direction rather than applying existing target-specific methods?
- How does the total cost of the proposed approach, including eight full-parameter fine-tuning runs under the default setting, compare with methods such as Heretic?