- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes a quantization attack based on outlier injection. To address the limitation of prior quantization attacks that are restricted to static quantization, the authors exploit the sensitivity of quantization algorithms to extreme values by injecting large outliers into selected weight blocks. This forces the remaining weights within each block to collapse to zero after quantization. The attack is combined with a dual-objective fine-tuning strategy to preserve model utility in full precision while activating malicious behavior after quantization. Experimental results demonstrate that the proposed method achieves high attack success rates across several widely used optimization-based quantization schemes.
Critical strengths
-
Clear and accessible presentation: The paper is logically organized and easy to follow. The proposed mechanism, from outlier injection to the multi-stage fine-tuning procedure, is presented coherently, and the algorithms and figures are generally clear.
-
Consideration of defenses and security implications: Beyond demonstrating high attack success rates, the paper discusses potential defense mechanisms. In particular, it investigates statistical detection of outliers using the Kolmogorov–Smirnov (KS) test and considers potential mitigation strategies, such as rescaling and zeroing outliers.
Critical weaknesses
-
Limited novelty and overall contribution: One of the paper's main contributions is extending quantization attacks to advanced quantization techniques, such as AWQ and GPTQ. However, recent work, including LLMQuA [1], has already explored and demonstrated the feasibility of attacking more sophisticated LLM quantization schemes. Although the proposed mechanism of exploiting outliers to disrupt group-wise scaling is straightforward, the overall conceptual novelty appears limited.
-
Questionable stealthiness and practical threat: In realistic deployment scenarios, as awareness of quantization-related security risks increases, model users or auditors may inspect model weights before quantization rather than relying solely on black-box functional evaluation of the full-precision (BF16) model. The proposed method requires injecting a substantial number of extreme-magnitude outliers into the weights, resulting in pronounced anomalies in the full-precision weight distribution, as illustrated by the KS-test analysis. Such anomalies may be readily detected using relatively simple statistical tests, raising concerns about the stealthiness and practical threat posed by the attack.
-
Limited attack effectiveness under certain experimental settings: The attack exhibits clear limitations for some model–quantization combinations. For example, in the Mistral-7B-Instruct + Over-Refusal setting, under GGUF Q4_0 quantization, the attack success rate (ASR) of the attacked quantized model is nearly identical to that of the full-precision model (4.3% versus 4.3%). This result suggests that the intended quantization-triggered behavior is not effectively realized in this setting.
-
Substantial utility degradation under certain settings: The attack causes non-negligible degradation in general capabilities for some model–attack combinations. For example, after the attack is applied to Mistral-7B-Instruct in the Over-Refusal setting, its HumanEval score drops from 36.6 for the original model to 20.7. This result highlights a potential trade-off between attack stealthiness and the preservation of general model capabilities.
[1] LLMQuA: Practical Backdoor Injection on Large Language Model Quantization. WWW 2026.
Questions for authors
See weaknesses.
2 comments
Thanks to the author's detailed reply and supplementary experimental analysis, these results help to provide a clearer understanding of the impact of threat model settings, scaling factor trade-offs, and fine-tuning on model performance. However, considering that other reviewers have raised similar concerns, especially regarding attack concealment, limited attack effects in some settings, and reduced general capabilities, after a comprehensive weighing, I am inclined to maintain the current score.
W1: Can you discuss novelty over LLMQuA?
Good question. LLMQuA [3] is a malicious code insertion attack into open-source quantization libraries. (See Section 3.2 and Algorithm 1 of their work). They assume that an adversary modifies a popular library (e.g., GPTQ) such that its rounding pattern is skewed towards malicious behavior, and that the user uses the compromised library on a benign model to activate the attack. In contrast, we aim to (maliciously) finetune a model such that the attack activates under unmodified quantization. We believe our threat model is substantially different. We included the discussion in the PDF (see Section 2).
W2: Can you elaborate on the practical threat stemming from this attack?
Great question. As we briefly mentioned in Section 5.3 and explained in C.2, our attack is indeed detectable based on statistical testing, assuming access to the original version of the attacked model. However, we believe this does not undermine the impact of our work for the following reason.
Detection in a practical setup is challenging. Section 5.3 is based on a setup where the adversary did not take detection into account. We show in Figure 6 that it is possible to achieve promising (yet slightly lower) ASR even with a significantly smaller scaling factor (16x instead of 512x). Building on this, we further show that the 16x-scaled switching block is still below the maximum value of the weight parameter of the layer (Figure 7), making it more challenging to detect it trivially, and (ii) it still achieves high ASR in various 4 bit quantizations (Table 13).
W3: Investigate low ASR for, e.g., Mistral, over-refusal, GGUF Q4_0.
That is a very interesting point and hard to answer. As the reviewer points out, we show in Table 9 that the outlier injection process is largely successful even for the case where we did not achieve high ASR. While we believe it involves various complex factors, we provide additional analysis to interpret this.
We calculate the cosine similarity of the final-layer activation between the proxy model and the actual quantized model, and compute its correlation with ASR (Mistral, over-refusal). As shown in Figure 7, we observe a positive correlation between proxy-real cosine similarity and ASR (Pearson = 0.76). While not a strong signal, this indicates that the layers other than the outlier insertion phase are accumulated over layers, which leads to a lower ASR for some methods. This indicates that a stronger control on how each layer is quantized (beyond the switching layer) could further strengthen the attack. We leave it as an interesting avenue for future work.
W4: Explain low HumanEval score for Mistral with over-refusal setting.
Certainly. We observe a general instability when finetuning Mistral, as also observed in prior attacks [1]. In our attack, we see the degradation already after kickstart finetuning, before outlier insertion (see the table below).
Importantly, we do not consider the utility degradation to be a fundamental limitation of our approach for two reasons. First, as we show in Figure 3 on Llama, we can control the outlier scale to balance the utility-ASR. In particular, if we allow a lower ASR for 8-bit quantization and focus on 4-bit, we achieve promising ASR even with a 16x smaller scaling factor. Second, we use the KL regularizer throughout training to maintain utility. If the adversary’s priority is to preserve utility even more, increasing the coefficient provides the flexibility to do so.