- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes an attack method against LLMs that exploits the fact that users apply transformations, like quantization, pruning, and merging, to models before deploying them. Attacks of this kind, which e.g. jailbreak, inject content, or introduce backdoors, that remain stealth on the attacked model and surface only after a specific transformation are already known. This paper claims the first attack that triggers malicious behaviours after any transformation from within a set is applied. The attack is validated on four models, thirteen transformations, and three adversarial behaviours.
Critical strengths
- The paper identifies approximate model perturbations that simulate the effect of transformations on model parameters sufficiently well as to be exploited for the attack. It is an interesting fact that the transformations do not need to be accurate for the behaviours to be injected.
Critical weaknesses
-
The paper positions itself as a call to action to practitioners to test their model in deployment conditions (i.e. after applying the transformation). However the method in the paper changes nothing in this respect: attacks triggered by all considered transformations already exist, and a cautious practitioner would already have to test against those. The paper is therefore more useful to potential attackers than it is to defenders.
-
The authors claim that their results in Table 2 prove that the attacked model remains stealth until a transformation is applied also because utility is not strongly impacted by the attack. However results in Table 2 actually show noticeable utility differences. Furthermore, tests that utility is preserved also after the transformations are applied are missing, but they are necessary to support stealthiness claims.
Questions for authors
-
What is the relative utility of original and attacked models after transformation?
-
Of the utility benchmarks you used, the non-multiple-choice ones are GSM8K and HumanEval, and Table 11 shows that the impact of some attacks on them is quite large. Their swings are then diluted in the averages with multiple-choice tests in Table 2. Can you provide additional evidence that the proposed attack is not degrading the model in a detectable way? Alternatively, consider qualifying your claims.
2 comments
I thank the authors for their response. I don't find in W2 an answer to the question of whether the combined attack is stealth, or if the performance on generative tasks is such that it becomes easily detectable for high ASR regimes. I will therefore maintain my overall score.
We thank the reviewer for their constructive feedback. We appreciate their recognition of the effectiveness of our approach and address their concerns below.
W1. The threat w.r.t. individual transformations is known. How would the community benefit from this paper?
Great question. Our contribution is a general framework for constructing compromised models whose malicious behavior is triggered by various model transformations. While it is true that prior attacks have targeted most of these transformations individually, we believe our work has various practical implications for the community, including (but not limited to) the following:
Transferability. As shown in Section 4.3, our attacked models achieve high ASR even on transformations not explicitly targeted during training. Moreover, our framework (i.e., surrogate perturbations) is not restricted to quantization, pruning, and merging. This suggests that many more transformations, including those previously overlooked due to, e.g., less common use cases, may be vulnerable. Our work raises awareness of these risks in communities where such threats have received limited attention, and provides a unified way of formulating them.
Unified attacks leading to improved detection/defenses. Existing defenses often target transformation-specific attack mechanisms. For example, several defenses against quantization-conditioned attacks have been proposed, such as [1-3]. However, these defenses assume (specific) PGD-based attacks and, as such, fail against ours (see our new results in Table 22). Our findings thus motivate the development of more general defenses that cover a broader range of transformations and attack mechanisms.
[1] https://arxiv.org/abs/2606.28962
[2] https://arxiv.org/abs/2606.29239
[3] https://arxiv.org/abs/2606.20254
W2 & Q1. Could you compare the utility of the post-transformed models before and after the attack?
Certainly! We provide the results in Table 23 (also pasted below). For quantization, pruning, and merging, the utility is largely preserved; the relative utility of the attacked version remains above 90% on average. We find larger fluctuations for structured pruning (e.g., a huge improvement from 1.7% to 42.5% for LaCo, GSM8k), but we interpret this as instability of the structured pruning approach in general.
Table: Average relative utility (%) of the attacked model under each transformation (mean ± sample standard deviation across five benchmarks). We omit LaCo as its original HumanEval score is zero.
W2 & Q2: Can you further discuss the utility degradation in generative tasks?
Absolutely. Conceptually, the adversary can control the trade-off between utility and ASR via the KL coefficient. We investigate how much we improve the situation by adjusting it. As shown in Table 21, by increasing the KL coefficient from 0.1 (our main result) to 0.5, the HumanEval score is significantly better preserved than in our main experiment (47.6% → 61.7%), but at the cost of a slight decrease in the ASR (81.6% → 66.6% on average across the tested transformations). This result indicates that the utility degradation is not a fundamental limitation of our approach, but an aspect that the adversary can control depending on how much utility in what type of task they aim to preserve.
Fundamentally, however, as we discussed in W2 & Q1, as we aim to target a significantly diverse set of transformations, complex interactions of various factors may result in some failure modes, and we accept this trade-off.