- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes SinkQCB, a quantization-conditioned backdoor attack for large language models (LLMs). SinkQCB encodes malicious behavior into the output-projection slices of selected attention heads and uses attention sinks to control their contribution: high sink attention suppresses the attack in the full-precision model, whereas quantization restores a low-sink-attention state that activates the malicious behavior. The method consists of three stages: activating the selected heads, injecting the target behavior, and restoring benign full-precision behavior while preserving the malicious quantized state. Experiments on Qwen2.5-1.5B, Llama-3.2-1B, and Mistral-7B evaluate content injection, code injection, and sentiment steering under NF4, FP4, and INT8 quantization.
Critical strengths
(1) The paper addresses an important and realistic security problem. It considers a model-supply-chain scenario in which downstream users independently apply post-training quantization to a released model. The threat model is practically relevant because the released BF16 model can appear benign during pre-deployment inspection while exhibiting attacker-specified behavior after quantization, potentially affecting real-world model distribution and deployment pipelines. (2) The idea of separating malicious-behavior encoding from activation is novel, drawing inspiration from the distinction between expert behavior and routing control in mixture-of-experts models. SinkQCB encodes the target behavior into the output-projection slices of selected attention heads and uses attention sinks to regulate their contribution to the residual stream. This routing-inspired design is adapted to dense Transformers, suppressing the malicious behavior in full precision while activating it after quantization.
Critical weaknesses
(1) The contribution of this paper is limited. SinkQCB combines existing techniques, including quantization-conditioned backdoors, attention-sink-based head modulation, and the separation of behavior encoding from contribution control. While their integration into dense Transformers is meaningful, these individual techniques have been explored in prior studies, making the proposed method appear somewhat incremental. (2) The authors argue in the Introduction that existing QCB methods suffer from optimization difficulties under quantization-preserving constraints, and propose parameter separation to address this issue. However, the experiments mainly report the final attack success rates without directly evaluating the effectiveness of the optimization process. (3) The authors state in Sections 4.2 and 4.5 that the quantization-preserving constraints ensure that Stage 3 retains the same quantized weights as Stage 2. However, Table 3 reports different INT8 attack success rates for Stage 2 (90.07%) and Stage 3 (91.73%). If the quantized models are identical, their outputs should be consistent under deterministic inference. (4) More baseline comparisons would be appreciated. The experiments only compare SinkQCB with Egashira et al. (2024), although more recent QCB methods, such as Zhan et al. (2026), are discussed in Section 2.2. (5) The labels in Figure 2(b) should be revised. All attention heads are labeled as “Head 1,” and their corresponding output projections are denoted as W_O^((1)). Different attention heads and their corresponding projection slices should be labeled correctly.”
Questions for authors
(1) In Sections 4.2 and 4.5, the authors state that Stage 3 preserves the quantized weights obtained in Stage 2. However, Table 3 reports different INT8 ASRs for the two stages (90.07% vs. 91.73%). Could the authors explain the reason for this discrepancy and clarify whether the quantized parameters are exactly identical across the two stages? (2) The authors claim that parameter separation alleviates the optimization difficulties of existing QCB methods. However, Stage 3 still requires optimization under quantization-preserving constraints. Could the authors clarify how the proposed parameter separation strategy reduces the optimization difficulty compared with Egashira et al. (2024), and what specific optimization limitations are addressed? (3) In Table 1, the baseline achieves 0.00% INT8 ASR on the Mistral-7B code-injection task, which the authors attribute to BF16 rounding errors at quantization boundaries. Could the authors clarify whether correcting this numerical issue would affect the comparison with SinkQCB? Additionally, why are more recent QCB methods, such as Zhan et al. (2026), not included in the comparisons?
5 comments
Response to Reviewer 2i3S (Part 2/3)
W3 & Q1: INT8 ASR discrepancy between Stages 2 and 3.
Thank you for pointing out this discrepancy. Our statement that Stages 2 and 3 yield strictly identical quantized weights was too strong for finite-precision arithmetic. In Stage 3, we constrain updates in using intervals derived from the Stage 2 quantized weights and introduce safety margins at the interval boundaries with a fixed default margin parameter. In our BF16 implementation, however, rounding the interval boundaries to the BF16 dtype can occasionally exceed this fixed margin, allowing a small number of boundary-clipped weights to map to an adjacent quantized value. As a result, the quantized weights are not bit-for-bit identical between Stages 2 and 3, leading to slight differences in outputs and ASRs under deterministic decoding.
This minor boundary effect does not undermine our attack mechanism because stays frozen in Stage 3 with unchanged quantized values, and the quantized sink attention of the selected heads remains nearly identical (0.0024 vs. 0.0025 in Table 3), preserving high INT8 ASR (90.07% vs. 91.73%). We have clarified the effect of boundary rounding in the discussion of Table 3.
W4 & Q3: Additional baselines and the Mistral-7B/Code/INT8 comparison.
Thank you for highlighting the Mistral-7B code-injection comparison. In Egashira et al. [1], a smaller poisoning learning rate better preserves benign utility, but leaves the implanted behavior sensitive to BF16 boundary rounding under INT8 quantization. A larger poisoning learning rate keeps INT8 ASR high against boundary rounding, at the cost of lower benign utility. We evaluated poisoning learning rates of , , , and on the 164 HumanEval prompts, using a fixed repair learning rate raised from our initial run to match stronger poisoning, and updated Tables 1 and 2 of the revised manuscript using the setting (the benign Mistral-7B model achieves 0.5018/0.6694 on TruthfulQA MC1/MC2).
| Method | BF16↓ | NF4↑ | FP4↑ | INT8↑ | MC1↑ | MC2↑ |
|---|---|---|---|---|---|---|
| Egashira et al. [1] () | 0.00 | 21.34 | 0.00 | 0.61 | 0.4064 | 0.5724 |
| Egashira et al. [1] () | 0.00 | 94.51 | 96.95 | 89.02 | 0.3929 | 0.5562 |
| Egashira et al. [1] () | 0.00 | 98.78 | 98.17 | 95.73 | 0.3880 | 0.5480 |
| Egashira et al. [1] () | 0.00 | 99.39 | 100.00 | 97.56 | 0.3305 | 0.4876 |
| SinkQCB | 0.00 | 96.95 | 96.34 | 97.56 | 0.5043 | 0.6560 |
Raising the learning rate from to steadily increases ASR after quantization up to 97.56% to 100.00%, while lowering TruthfulQA MC1/MC2 from 0.4064/0.5724 to 0.3305/0.4876 as stronger updates across the entire model during poisoning degrade benign utility. SinkQCB achieves comparable ASR after quantization (96.34% to 97.56%) and 0.00% BF16 ASR while preserving much higher benign utility (0.5043/0.6560).
Response to Reviewer 2i3S (Part 3/3)
Regarding additional baselines, among the QCB studies in Section 2.2, Egashira et al. [3] focus on GGUF quantization and Dong et al. [4] study robustness to downstream fine-tuning, which address different settings from ours. As Zhan et al. [2] was concurrent with our work, it was not yet included in our initial experimental evaluation. Following your suggestion, we have now reproduced Zhan et al. [2] using its public implementation under the same model and task settings, added the results to Tables 1 and 2 of the revised manuscript, and report ASR (%) and TruthfulQA MC1/MC2 below.
| Model | Task | Method | BF16↓ | NF4↑ | FP4↑ | INT8↑ | MC1↑ | MC2↑ |
|---|---|---|---|---|---|---|---|---|
| Llama-3.2-1B | Content | Zhan et al. [2] | 0.87 | 80.00 | 68.73 | 63.27 | 0.2950 | 0.4743 |
| SinkQCB | 0.13 | 97.53 | 97.73 | 98.73 | 0.3170 | 0.4896 | ||
| Code | Zhan et al. [2] | 4.88 | 100.00 | 96.95 | 95.73 | 0.2778 | 0.4369 | |
| SinkQCB | 0.00 | 100.00 | 97.56 | 100.00 | 0.2925 | 0.4615 | ||
| Sentiment | Zhan et al. [2] | 1.50 | 99.50 | 100.00 | 100.00 | 0.2705 | 0.4548 | |
| SinkQCB | 1.00 | 100.00 | 100.00 | 100.00 | 0.2950 | 0.4674 | ||
| Qwen2.5-1.5B | Content | Zhan et al. [2] | 0.60 | 99.67 | 87.00 | 97.53 | 0.3244 | 0.4899 |
| SinkQCB | 0.13 | 99.80 | 98.40 | 91.73 | 0.3244 | 0.4912 | ||
| Code | Zhan et al. [2] | 1.83 | 70.73 | 73.17 | 34.76 | 0.3109 | 0.4816 | |
| SinkQCB | 0.00 | 73.78 | 94.51 | 87.20 | 0.2889 | 0.4610 | ||
| Sentiment | Zhan et al. [2] | 1.00 | 100.00 | 99.00 | 100.00 | 0.3060 | 0.4546 | |
| SinkQCB | 0.00 | 100.00 | 100.00 | 100.00 | 0.3072 | 0.4761 | ||
| Mistral-7B | Content | Zhan et al. [2] | 4.00 | 80.13 | 8.33 | 84.87 | 0.3341 | 0.5181 |
| SinkQCB | 0.13 | 85.53 | 86.80 | 84.47 | 0.4933 | 0.6490 | ||
| Code | Zhan et al. [2] | 0.61 | 95.12 | 96.95 | 98.17 | 0.4480 | 0.6203 | |
| SinkQCB | 0.00 | 96.95 | 96.34 | 97.56 | 0.5043 | 0.6560 | ||
| Sentiment | Zhan et al. [2] | 3.50 | 84.50 | 85.50 | 92.50 | 0.3390 | 0.5062 | |
| SinkQCB | 0.00 | 96.50 | 97.00 | 96.50 | 0.4933 | 0.6534 |
Across all nine settings, SinkQCB achieves lower BF16 ASR than Zhan et al. [2], while achieving higher or comparable ASR after quantization on most settings with more consistent performance across all three quantization formats. SinkQCB also preserves higher benign utility on TruthfulQA in eight of the nine settings, with the largest gains on Mistral-7B.
On Mistral-7B content injection, which is included in the paper and public code of Zhan et al. [2], our reproduced checkpoint matches their Table 1 results on BF16 (4.00% vs. 4.2%), NF4 (80.13% vs. 81.3%), and INT8 (84.87% vs. 83.6%), while giving 8.33% ASR on FP4. Running their default configuration gave 0.00% FP4 ASR in our experiments. Tuning the learning rate raised FP4 ASR to 8.33%, and further increases either raised BF16 ASR or hurt utility. Zhan et al. [2] (Table 1) similarly report a much lower FP4 ASR on this setting (35.5%, vs. 81.3% on NF4 and 83.6% on INT8), along with ASRs below 8% (5.5% to 7.4%) on several 4-bit GGUF formats.
W5: Attention head and projection labels in Figure 2(b).
Thank you for pointing this out. We have corrected the labels in Figure 2(b) in the revised manuscript to distinguish the attention heads and their corresponding output-projection slices.
References
[1] Egashira, K., Vero, M., Staab, R., He, J., & Vechev, M. (2024). Exploiting LLM quantization. Advances in Neural Information Processing Systems, 37, 41709-41732.
[2] Zhan, X., Egashira, K., Staab, R., Vero, M., & Vechev, M. (2026). Widening the Gap: Exploiting LLM Quantization via Outlier Injection. In Trustworthy AI for Good (AI4GOOD) Workshop @ ICML 2026.
[3] Egashira, K., Staab, R., Vero, M., He, J., & Vechev, M. (2025). Mind the gap: A practical attack on GGUF quantization. In Proceedings of the 42nd International Conference on Machine Learning (Vol. 267, pp. 15038-15065). PMLR.
[4] Dong, P., Li, H., & Guo, S. (2025). Durable quantization conditioned misalignment attack on large language models. In International Conference on Learning Representations (Vol. 2025, pp. 21789-21801).
[5] Fu, Z., Zeng, W., Wang, R., & Li, M. (2026). Attention sink forges native MoE in attention layers: Sink-aware training to address head collapse. arXiv preprint arXiv:2602.01203.
[6] Wang, Q., Pang, Q., Lin, X., Wang, S., & Wu, D. (2025). BadMoE: Backdooring mixture-of-experts LLMs via optimizing routing triggers and infecting dormant experts. arXiv preprint arXiv:2504.18598.
Response to Reviewer 2i3S (Part 1/3)
Thank you for your thorough review and for recognizing our practical threat model and routing-inspired separation in dense Transformers. We address your concerns below.
Thank you for the comment. We would like to clarify that despite that we got inspiration from existing literature, Fu et al. [5] are not concerned with the security/safety aspect of LLMs. More specifically, Fu et al. [5] consider head collapse in benign pretraining, interpreting attention sinks as implicit MoE gates and using a load-balancing loss to keep attention heads active. Our contribution lies in separating the insertion of malicious behavior from the mechanism that triggers it in dense Transformers, addressing the optimization difficulty in prior QCB methods [1, 2]. Rather than poisoning the entire model and relearning benign outputs over those same poisoned weights within small intervals [1, 2], or relying on explicit MoE routers and input triggers [6], SinkQCB encodes the target behavior in a few selected heads and uses sink attention to suppress them at full precision and activate them after quantization.
Moreover, SinkQCB is not a simple combination of [1, 5, 6], because realizing this attack in dense LLMs without an explicit router or input trigger introduces three difficulties that do not arise in prior studies. First, heads with high initial sink attention contribute little to the residual stream, so directly training their output-projection slices on poisoned data is ineffective, and Stage 1 lowers their sink attention on clean data before poisoning. Second, training on poisoned data can encode the target behavior into the sink token itself and hinder high sink attention from suppressing the attack, which Stage 2 mitigates by penalizing ∥WO(h)vs(h)∥22. Third, whereas BadMoE [6] uses an input trigger to steer an explicit router and Fu et al. [5] freely update weights for a single benign model state, our attack requires one checkpoint to exhibit high sink attention at full precision and low sink attention after quantization while maintaining benign utility, which Stage 3 achieves by freezing PP and updating PS within quantization-preserving intervals.
Beyond the attack method, our experiments show that weight quantization alone can shift sink attention in the selected heads to activate the target behavior, while confining the target behavior to a small subset of attention heads preserves full-precision utility better than prior QCB methods that modify the whole model (Tables 1, 2, and 3). We believe that adapting and incorporating ideas from related areas into LLM security is meaningful and sheds light on a better understanding of LLM vulnerabilities, which may further motivate the community for further exploration.
Parameter separation reduces optimization difficulty because within the constrained weight space, Egashira et al. [1] must simultaneously optimize multiple objectives, whereas SinkQCB only needs to change sink attention in the selected heads. Specifically, because Egashira et al. [1] update weights trained on malicious data across the whole model within quantization-preserving intervals, optimization must simultaneously (1) remove the malicious behavior at full precision, (2) learn benign responses, and (3) recover general model utility lost during earlier poisoning. In SinkQCB, only PP is trained on malicious data and stays frozen in Stage 3. Increasing sink attention in the selected heads within the intervals suppresses the contribution of PP to the residual stream so that the remaining parameters produce benign responses, achieving both (1) and (2) at full precision through a single change in sink attention. Moreover, because PS is never updated on malicious data, SinkQCB does not face the problem in (3) of recovering utility lost during poisoning (while clean NLL and distillation keep PS close to its existing benign state), making the constrained optimization simpler.
Beyond final ASRs, Table 3 evaluates this optimization across the three stages, showing that Stage 3 raises sink attention in the selected heads from 0.0022 to 0.9648 in BF16 within the quantization-preserving intervals while keeping quantized sink attention at 0.0025. Together with the low BF16 ASR, high post-quantization ASR, and preserved utility in Tables 1 and 2, these results support the effectiveness of parameter separation under the constraints.