acceptodds
Under review as a conference paper at ICLR 2027

Quantization-Conditioned Backdoors on Large Language Models via Attention Sinks

Abstract

Post-training quantization is widely used to reduce the deployment cost of large language models (LLMs), but recent studies have shown that quantization may be exploited as a backdoor trigger: a released model may appear benign before quantization while exhibiting attacker-specified behaviors after quantization. Existing quantization-conditioned backdoor attacks first implant the target behavior into the quantized model and then restore benign behavior at full precision while keeping the weights within a subspace that yields the same quantized values as the backdoored counterpart. This requires optimizing a complex objective within a highly constrained weight space, leading to limited attack effectiveness. To address this issue, we introduce SinkQCB, which separates the insertion of malicious behavior from the mechanism that triggers it. More specifically, the malicious behaviors are encoded in a small set of attention heads, and attention sinks are utilized to keep these heads suppressed at full precision and activated after quantization. Extensive experiments on commonly considered LLMs, malicious tasks, and quantization methods demonstrate that SinkQCB achieves a higher attack success rate (ASR) after quantization while keeping ASR at or below 1% and maintaining competitive performance on benign tasks at full precision.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Large Language Models, Backdoor Attacks, Post-Training Quantization, Attention Sinks, Model Security

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_2i3SOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SinkQCB, a quantization-conditioned backdoor attack for large language models (LLMs). SinkQCB encodes malicious behavior into the output-projection slices of selected attention heads and uses attention sinks to control their contribution: high sink attention suppresses the attack in the full-precision model, whereas quantization restores a low-sink-attention state that activates the malicious behavior. The method consists of three stages: activating the selected heads, injecting the target behavior, and restoring benign full-precision behavior while preserving the malicious quantized state. Experiments on Qwen2.5-1.5B, Llama-3.2-1B, and Mistral-7B evaluate content injection, code injection, and sentiment steering under NF4, FP4, and INT8 quantization.

Critical strengths

(1) The paper addresses an important and realistic security problem. It considers a model-supply-chain scenario in which downstream users independently apply post-training quantization to a released model. The threat model is practically relevant because the released BF16 model can appear benign during pre-deployment inspection while exhibiting attacker-specified behavior after quantization, potentially affecting real-world model distribution and deployment pipelines. (2) The idea of separating malicious-behavior encoding from activation is novel, drawing inspiration from the distinction between expert behavior and routing control in mixture-of-experts models. SinkQCB encodes the target behavior into the output-projection slices of selected attention heads and uses attention sinks to regulate their contribution to the residual stream. This routing-inspired design is adapted to dense Transformers, suppressing the malicious behavior in full precision while activating it after quantization.

Critical weaknesses

(1) The contribution of this paper is limited. SinkQCB combines existing techniques, including quantization-conditioned backdoors, attention-sink-based head modulation, and the separation of behavior encoding from contribution control. While their integration into dense Transformers is meaningful, these individual techniques have been explored in prior studies, making the proposed method appear somewhat incremental. (2) The authors argue in the Introduction that existing QCB methods suffer from optimization difficulties under quantization-preserving constraints, and propose parameter separation to address this issue. However, the experiments mainly report the final attack success rates without directly evaluating the effectiveness of the optimization process. (3) The authors state in Sections 4.2 and 4.5 that the quantization-preserving constraints ensure that Stage 3 retains the same quantized weights as Stage 2. However, Table 3 reports different INT8 attack success rates for Stage 2 (90.07%) and Stage 3 (91.73%). If the quantized models are identical, their outputs should be consistent under deterministic inference. (4) More baseline comparisons would be appreciated. The experiments only compare SinkQCB with Egashira et al. (2024), although more recent QCB methods, such as Zhan et al. (2026), are discussed in Section 2.2. (5) The labels in Figure 2(b) should be revised. All attention heads are labeled as “Head 1,” and their corresponding output projections are denoted as W_O^((1)). Different attention heads and their corresponding projection slices should be labeled correctly.”

Questions for authors

(1) In Sections 4.2 and 4.5, the authors state that Stage 3 preserves the quantized weights obtained in Stage 2. However, Table 3 reports different INT8 ASRs for the two stages (90.07% vs. 91.73%). Could the authors explain the reason for this discrepancy and clarify whether the quantized parameters are exactly identical across the two stages? (2) The authors claim that parameter separation alleviates the optimization difficulties of existing QCB methods. However, Stage 3 still requires optimization under quantization-preserving constraints. Could the authors clarify how the proposed parameter separation strategy reduces the optimization difficulty compared with Egashira et al. (2024), and what specific optimization limitations are addressed? (3) In Table 1, the baseline achieves 0.00% INT8 ASR on the Mistral-7B code-injection task, which the authors attribute to BF16 rounding errors at quantization boundaries. Could the authors clarify whether correcting this numerical issue would affect the comparison with SinkQCB? Additionally, why are more recent QCB methods, such as Zhan et al. (2026), not included in the comparisons?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 2i3S (Part 1/3)

Thank you for your thorough review and for recognizing our practical threat model and routing-inspired separation in dense Transformers. We address your concerns below.

W1: The contribution of SinkQCB.

Thank you for the comment. We would like to clarify that despite that we got inspiration from existing literature, Fu et al. [5] are not concerned with the security/safety aspect of LLMs. More specifically, Fu et al. [5] consider head collapse in benign pretraining, interpreting attention sinks as implicit MoE gates and using a load-balancing loss to keep attention heads active. Our contribution lies in separating the insertion of malicious behavior from the mechanism that triggers it in dense Transformers, addressing the optimization difficulty in prior QCB methods [1, 2]. Rather than poisoning the entire model and relearning benign outputs over those same poisoned weights within small intervals [1, 2], or relying on explicit MoE routers and input triggers [6], SinkQCB encodes the target behavior in a few selected heads and uses sink attention to suppress them at full precision and activate them after quantization.

Moreover, SinkQCB is not a simple combination of [1, 5, 6], because realizing this attack in dense LLMs without an explicit router or input trigger introduces three difficulties that do not arise in prior studies. First, heads with high initial sink attention contribute little to the residual stream, so directly training their output-projection slices on poisoned data is ineffective, and Stage 1 lowers their sink attention on clean data before poisoning. Second, training on poisoned data can encode the target behavior into the sink token itself and hinder high sink attention from suppressing the attack, which Stage 2 mitigates by penalizing ∥WO(h)vs(h)∥22\Vert W_O^{(h)}v_s^{(h)}\Vert_2^2. Third, whereas BadMoE [6] uses an input trigger to steer an explicit router and Fu et al. [5] freely update weights for a single benign model state, our attack requires one checkpoint to exhibit high sink attention at full precision and low sink attention after quantization while maintaining benign utility, which Stage 3 achieves by freezing PP\mathcal{P}_P and updating PS\mathcal{P}_S within quantization-preserving intervals.

Beyond the attack method, our experiments show that weight quantization alone can shift sink attention in the selected heads to activate the target behavior, while confining the target behavior to a small subset of attention heads preserves full-precision utility better than prior QCB methods that modify the whole model (Tables 1, 2, and 3). We believe that adapting and incorporating ideas from related areas into LLM security is meaningful and sheds light on a better understanding of LLM vulnerabilities, which may further motivate the community for further exploration.


W2 & Q2: Optimization under quantization-preserving constraints.

Parameter separation reduces optimization difficulty because within the constrained weight space, Egashira et al. [1] must simultaneously optimize multiple objectives, whereas SinkQCB only needs to change sink attention in the selected heads. Specifically, because Egashira et al. [1] update weights trained on malicious data across the whole model within quantization-preserving intervals, optimization must simultaneously (1) remove the malicious behavior at full precision, (2) learn benign responses, and (3) recover general model utility lost during earlier poisoning. In SinkQCB, only PP\mathcal{P}_P is trained on malicious data and stays frozen in Stage 3. Increasing sink attention in the selected heads within the intervals suppresses the contribution of PP\mathcal{P}_P to the residual stream so that the remaining parameters produce benign responses, achieving both (1) and (2) at full precision through a single change in sink attention. Moreover, because PS\mathcal{P}_S is never updated on malicious data, SinkQCB does not face the problem in (3) of recovering utility lost during poisoning (while clean NLL and distillation keep PS\mathcal{P}_S close to its existing benign state), making the constrained optimization simpler.

Beyond final ASRs, Table 3 evaluates this optimization across the three stages, showing that Stage 3 raises sink attention in the selected heads from 0.0022 to 0.9648 in BF16 within the quantization-preserving intervals while keeping quantized sink attention at 0.0025. Together with the low BF16 ASR, high post-quantization ASR, and preserved utility in Tables 1 and 2, these results support the effectiveness of parameter separation under the constraints.

AuthorsOpenReview ↗

Response to Reviewer 2i3S (Part 2/3)

W3 & Q1: INT8 ASR discrepancy between Stages 2 and 3.

Thank you for pointing out this discrepancy. Our statement that Stages 2 and 3 yield strictly identical quantized weights was too strong for finite-precision arithmetic. In Stage 3, we constrain updates in PS\mathcal{P}_S using intervals derived from the Stage 2 quantized weights and introduce safety margins at the interval boundaries with a fixed default margin parameter. In our BF16 implementation, however, rounding the interval boundaries to the BF16 dtype can occasionally exceed this fixed margin, allowing a small number of boundary-clipped weights to map to an adjacent quantized value. As a result, the quantized weights are not bit-for-bit identical between Stages 2 and 3, leading to slight differences in outputs and ASRs under deterministic decoding.

This minor boundary effect does not undermine our attack mechanism because PP\mathcal{P}_P stays frozen in Stage 3 with unchanged quantized values, and the quantized sink attention of the selected heads remains nearly identical (0.0024 vs. 0.0025 in Table 3), preserving high INT8 ASR (90.07% vs. 91.73%). We have clarified the effect of boundary rounding in the discussion of Table 3.


W4 & Q3: Additional baselines and the Mistral-7B/Code/INT8 comparison.

Thank you for highlighting the Mistral-7B code-injection comparison. In Egashira et al. [1], a smaller poisoning learning rate better preserves benign utility, but leaves the implanted behavior sensitive to BF16 boundary rounding under INT8 quantization. A larger poisoning learning rate keeps INT8 ASR high against boundary rounding, at the cost of lower benign utility. We evaluated poisoning learning rates of 2×10−62\times10^{-6}, 4×10−64\times10^{-6}, 5×10−65\times10^{-6}, and 1×10−51\times10^{-5} on the 164 HumanEval prompts, using a fixed repair learning rate raised from our initial run to match stronger poisoning, and updated Tables 1 and 2 of the revised manuscript using the 4×10−64\times10^{-6} setting (the benign Mistral-7B model achieves 0.5018/0.6694 on TruthfulQA MC1/MC2).

MethodBF16↓NF4↑FP4↑INT8↑MC1↑MC2↑
Egashira et al. [1] (2×10−62\times10^{-6})0.0021.340.000.610.40640.5724
Egashira et al. [1] (4×10−64\times10^{-6})0.0094.5196.9589.020.39290.5562
Egashira et al. [1] (5×10−65\times10^{-6})0.0098.7898.1795.730.38800.5480
Egashira et al. [1] (1×10−51\times10^{-5})0.0099.39100.0097.560.33050.4876
SinkQCB0.0096.9596.3497.560.50430.6560

Raising the learning rate from 2×10−62\times10^{-6} to 1×10−51\times10^{-5} steadily increases ASR after quantization up to 97.56% to 100.00%, while lowering TruthfulQA MC1/MC2 from 0.4064/0.5724 to 0.3305/0.4876 as stronger updates across the entire model during poisoning degrade benign utility. SinkQCB achieves comparable ASR after quantization (96.34% to 97.56%) and 0.00% BF16 ASR while preserving much higher benign utility (0.5043/0.6560).

AuthorsOpenReview ↗

Response to Reviewer 2i3S (Part 3/3)

Regarding additional baselines, among the QCB studies in Section 2.2, Egashira et al. [3] focus on GGUF quantization and Dong et al. [4] study robustness to downstream fine-tuning, which address different settings from ours. As Zhan et al. [2] was concurrent with our work, it was not yet included in our initial experimental evaluation. Following your suggestion, we have now reproduced Zhan et al. [2] using its public implementation under the same model and task settings, added the results to Tables 1 and 2 of the revised manuscript, and report ASR (%) and TruthfulQA MC1/MC2 below.

ModelTaskMethodBF16↓NF4↑FP4↑INT8↑MC1↑MC2↑
Llama-3.2-1BContentZhan et al. [2]0.8780.0068.7363.270.29500.4743
SinkQCB0.1397.5397.7398.730.31700.4896
CodeZhan et al. [2]4.88100.0096.9595.730.27780.4369
SinkQCB0.00100.0097.56100.000.29250.4615
SentimentZhan et al. [2]1.5099.50100.00100.000.27050.4548
SinkQCB1.00100.00100.00100.000.29500.4674
Qwen2.5-1.5BContentZhan et al. [2]0.6099.6787.0097.530.32440.4899
SinkQCB0.1399.8098.4091.730.32440.4912
CodeZhan et al. [2]1.8370.7373.1734.760.31090.4816
SinkQCB0.0073.7894.5187.200.28890.4610
SentimentZhan et al. [2]1.00100.0099.00100.000.30600.4546
SinkQCB0.00100.00100.00100.000.30720.4761
Mistral-7BContentZhan et al. [2]4.0080.138.3384.870.33410.5181
SinkQCB0.1385.5386.8084.470.49330.6490
CodeZhan et al. [2]0.6195.1296.9598.170.44800.6203
SinkQCB0.0096.9596.3497.560.50430.6560
SentimentZhan et al. [2]3.5084.5085.5092.500.33900.5062
SinkQCB0.0096.5097.0096.500.49330.6534

Across all nine settings, SinkQCB achieves lower BF16 ASR than Zhan et al. [2], while achieving higher or comparable ASR after quantization on most settings with more consistent performance across all three quantization formats. SinkQCB also preserves higher benign utility on TruthfulQA in eight of the nine settings, with the largest gains on Mistral-7B.

On Mistral-7B content injection, which is included in the paper and public code of Zhan et al. [2], our reproduced checkpoint matches their Table 1 results on BF16 (4.00% vs. 4.2%), NF4 (80.13% vs. 81.3%), and INT8 (84.87% vs. 83.6%), while giving 8.33% ASR on FP4. Running their default configuration gave 0.00% FP4 ASR in our experiments. Tuning the learning rate raised FP4 ASR to 8.33%, and further increases either raised BF16 ASR or hurt utility. Zhan et al. [2] (Table 1) similarly report a much lower FP4 ASR on this setting (35.5%, vs. 81.3% on NF4 and 83.6% on INT8), along with ASRs below 8% (5.5% to 7.4%) on several 4-bit GGUF formats.


W5: Attention head and projection labels in Figure 2(b).

Thank you for pointing this out. We have corrected the labels in Figure 2(b) in the revised manuscript to distinguish the attention heads and their corresponding output-projection slices.

References

[1] Egashira, K., Vero, M., Staab, R., He, J., & Vechev, M. (2024). Exploiting LLM quantization. Advances in Neural Information Processing Systems, 37, 41709-41732.
[2] Zhan, X., Egashira, K., Staab, R., Vero, M., & Vechev, M. (2026). Widening the Gap: Exploiting LLM Quantization via Outlier Injection. In Trustworthy AI for Good (AI4GOOD) Workshop @ ICML 2026.
[3] Egashira, K., Staab, R., Vero, M., He, J., & Vechev, M. (2025). Mind the gap: A practical attack on GGUF quantization. In Proceedings of the 42nd International Conference on Machine Learning (Vol. 267, pp. 15038-15065). PMLR.
[4] Dong, P., Li, H., & Guo, S. (2025). Durable quantization conditioned misalignment attack on large language models. In International Conference on Learning Representations (Vol. 2025, pp. 21789-21801).
[5] Fu, Z., Zeng, W., Wang, R., & Li, M. (2026). Attention sink forges native MoE in attention layers: Sink-aware training to address head collapse. arXiv preprint arXiv:2602.01203.
[6] Wang, Q., Pang, Q., Lin, X., Wang, S., & Wu, D. (2025). BadMoE: Backdooring mixture-of-experts LLMs via optimizing routing triggers and infecting dormant experts. arXiv preprint arXiv:2504.18598.

Reviewer_K9eDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes SinkQCB, an attack method that makes an LLM behave maliciously after quantization while looking normal at full precision. It uses the gating property of multi-head attention and applies a three-step finetuning to the parameters up to and including the layer with the highest sink attention. EXperiments on three LLMs, three attack tasks, and three quantization formats show that SinkQCB achieves a higher ASR after quantization than the baseline method (Egashira et al., 2024), while the ASR at full precision is near zero.

Critical strengths

1 This paper gives a concrete construction of QCB: it selects the layer and heads with the highest sink attention, and uses a carefully designed three-step finetuning to effectively control their contribution through sink attention.

2 The authors conduct experiments that examine the effect of the attention sink and clearly show by comparison that the malicious information depends primarily on the selected heads.

Critical weaknesses

1 The experiments only take one baseline (Egashira et al., 2024), while there exist other QCB methods that can be compared with; a specific case in the comparison with the baseline (Mistral-7B/Code/INT8) is confounded by the baseline's threshold-crossing issue in quantization. Therefore, the empirical results are not sufficient to reflect the true contribution of this paper.

2 The novelty is limited in both mechanism and methodology: the attention sink gating directly comes from Fu et al. (2026), and the core attack method derives from Egashira et al. (2024). Whether the new parts (layer/head selection and sink control) have incremental contribution requires further ablation.

Questions for authors

1 Have you compared SinkQCB with other baselines under the same model and task settings? For Mistral-7B/Code/INT8, can you provide a corrected baseline that avoids the BF16 interval-rounding threshold-crossing, so that the comparison is fair?

2 Is the layer with the highest sink attention the optimal layer to inject malicious information? Under the budget of comparable training resources, can different layers and heads achieve the target behavior in the second step, and what is the gating effect of each?

5 comments
AuthorsOpenReview ↗

Response to Reviewer K9eD (Part 1/3)

Thank you for your constructive feedback and for recognizing our three-step construction and head-level analysis. We address your concerns below.

W1 & Q1: Additional baselines and the Mistral-7B/Code/INT8 comparison.

Thank you for highlighting the Mistral-7B code-injection comparison. In Egashira et al. [1], a smaller poisoning learning rate better preserves benign utility, but leaves the implanted behavior sensitive to BF16 boundary rounding under INT8 quantization. A larger poisoning learning rate keeps INT8 ASR high against boundary rounding, at the cost of lower benign utility. We evaluated poisoning learning rates of 2×10−62\times10^{-6}, 4×10−64\times10^{-6}, 5×10−65\times10^{-6}, and 1×10−51\times10^{-5} on the 164 HumanEval prompts, using a fixed repair learning rate raised from our initial run to match stronger poisoning, and updated Tables 1 and 2 of the revised manuscript using the 4×10−64\times10^{-6} setting (the benign Mistral-7B model achieves 0.5018/0.6694 on TruthfulQA MC1/MC2).

MethodBF16↓NF4↑FP4↑INT8↑MC1↑MC2↑
Egashira et al. [1] (2×10−62\times10^{-6})0.0021.340.000.610.40640.5724
Egashira et al. [1] (4×10−64\times10^{-6})0.0094.5196.9589.020.39290.5562
Egashira et al. [1] (5×10−65\times10^{-6})0.0098.7898.1795.730.38800.5480
Egashira et al. [1] (1×10−51\times10^{-5})0.0099.39100.0097.560.33050.4876
SinkQCB0.0096.9596.3497.560.50430.6560

Raising the learning rate from 2×10−62\times10^{-6} to 1×10−51\times10^{-5} steadily increases ASR after quantization up to 97.56% to 100.00%, while lowering TruthfulQA MC1/MC2 from 0.4064/0.5724 to 0.3305/0.4876 as stronger updates across the entire model during poisoning degrade benign utility. SinkQCB achieves comparable ASR after quantization (96.34% to 97.56%) and 0.00% BF16 ASR while preserving much higher benign utility (0.5043/0.6560).

Regarding additional baselines, among the QCB studies in Section 2.2, Egashira et al. [3] focus on GGUF quantization and Dong et al. [4] study robustness to downstream fine-tuning, which address different settings from ours. As Zhan et al. [2] was concurrent with our work, it was not yet included in our initial experimental evaluation. Following your suggestion, we have now reproduced Zhan et al. [2] using its public implementation under the same model and task settings, added the results to Tables 1 and 2 of the revised manuscript, and report ASR (%) and TruthfulQA MC1/MC2 below.

ModelTaskMethodBF16↓NF4↑FP4↑INT8↑MC1↑MC2↑
Llama-3.2-1BContentZhan et al. [2]0.8780.0068.7363.270.29500.4743
SinkQCB0.1397.5397.7398.730.31700.4896
CodeZhan et al. [2]4.88100.0096.9595.730.27780.4369
SinkQCB0.00100.0097.56100.000.29250.4615
SentimentZhan et al. [2]1.5099.50100.00100.000.27050.4548
SinkQCB1.00100.00100.00100.000.29500.4674
Qwen2.5-1.5BContentZhan et al. [2]0.6099.6787.0097.530.32440.4899
SinkQCB0.1399.8098.4091.730.32440.4912
CodeZhan et al. [2]1.8370.7373.1734.760.31090.4816
SinkQCB0.0073.7894.5187.200.28890.4610
SentimentZhan et al. [2]1.00100.0099.00100.000.30600.4546
SinkQCB0.00100.00100.00100.000.30720.4761
Mistral-7BContentZhan et al. [2]4.0080.138.3384.870.33410.5181
SinkQCB0.1385.5386.8084.470.49330.6490
CodeZhan et al. [2]0.6195.1296.9598.170.44800.6203
SinkQCB0.0096.9596.3497.560.50430.6560
SentimentZhan et al. [2]3.5084.5085.5092.500.33900.5062
SinkQCB0.0096.5097.0096.500.49330.6534

Across all nine settings, SinkQCB achieves lower BF16 ASR than Zhan et al. [2], while achieving higher or comparable ASR after quantization on most settings with more consistent performance across all three quantization formats. SinkQCB also preserves higher benign utility on TruthfulQA in eight of the nine settings, with the largest gains on Mistral-7B.

On Mistral-7B content injection, which is included in the paper and public code of Zhan et al. [2], our reproduced checkpoint matches their Table 1 results on BF16 (4.00% vs. 4.2%), NF4 (80.13% vs. 81.3%), and INT8 (84.87% vs. 83.6%), while giving 8.33% ASR on FP4. Running their default configuration gave 0.00% FP4 ASR in our experiments. Tuning the learning rate raised FP4 ASR to 8.33%, and further increases either raised BF16 ASR or hurt utility. Zhan et al. [2] (Table 1) similarly report a much lower FP4 ASR on this setting (35.5%, vs. 81.3% on NF4 and 83.6% on INT8), along with ASRs below 8% (5.5% to 7.4%) on several 4-bit GGUF formats.

AuthorsOpenReview ↗

Response to Reviewer K9eD (Part 2/3)

W2 & Q2: Method novelty and layer/head selection.

Thank you for the comment. We would like to clarify that despite that we got inspiration from existing literature, Fu et al. [5] are not concerned with the security/safety aspect of LLMs. More specifically, Fu et al. [5] consider head collapse in benign pretraining, interpreting attention sinks as implicit MoE gates and using a load-balancing loss to keep attention heads active. Our contribution lies in separating the insertion of malicious behavior from the mechanism that triggers it in dense Transformers, addressing the optimization difficulty in prior QCB methods [1, 2]. Rather than poisoning the entire model and relearning benign outputs over those same poisoned weights within small intervals [1, 2], SinkQCB encodes the target behavior in a few selected heads and uses sink attention to suppress them at full precision and activate them after quantization.

Moreover, SinkQCB is not a simple combination of Fu et al. [5] and Egashira et al. [1], because using attention sinks for a quantization-conditioned attack introduces three difficulties beyond benign training. First, heads with high initial sink attention contribute little to the residual stream, so directly training their output-projection slices on poisoned data is ineffective, and Stage 1 lowers their sink attention on clean data before poisoning. Second, training on poisoned data can encode the target behavior into the sink token itself and hinder high sink attention from suppressing the attack, which Stage 2 mitigates by penalizing ∥WO(h)vs(h)∥22\Vert W_O^{(h)}v_s^{(h)}\Vert_2^2. Third, whereas Fu et al. [5] freely update weights for a single benign model state, our attack requires one checkpoint without any input trigger to exhibit high sink attention at full precision and low sink attention after quantization while maintaining benign utility, which Stage 3 achieves by freezing PP\mathcal{P}_P and updating PS\mathcal{P}_S within quantization-preserving intervals.

Beyond the attack method, our experiments show that weight quantization alone can shift sink attention in the selected heads to activate the target behavior, while confining the target behavior to a small subset of attention heads preserves full-precision utility better than prior QCB methods that modify the whole model (Tables 1, 2, and 3). We believe that adapting and incorporating ideas from related areas into LLM security is meaningful and sheds light on a better understanding of LLM vulnerabilities, which may further motivate the community for further exploration.

AuthorsOpenReview ↗

Response to Reviewer K9eD (Part 3/3)

Regarding layer and head selection under the same training budget, varying the target layer strongly affects ASR after quantization, whereas varying the selected heads within a fixed middle layer has little effect on ASR and mainly affects benign utility.

To evaluate layer selection, we test layers at roughly 20%, 40%, 64% (our choice, the middle layer with the highest initial sink attention), 80%, and 90% depth on Qwen2.5-1.5B-Instruct (28 layers) for content injection, selecting the four heads with the highest initial sink attention in each layer under the same training and evaluation settings.

Target layer (depth)BF16↓NF4↑FP4↑INT8↑MC1↑MC2↑
L6 (21.4%)0.1378.4760.0093.200.31460.4739
L11 (39.3%)0.13100.00100.00100.000.30110.4585
L18 (64.3%, Ours)0.1399.8098.4091.730.32440.4912
L22 (78.6%)0.1396.0014.3366.930.31460.4689
L25 (89.3%)0.1360.070.2048.130.31210.4716

Across all tested layers, BF16 ASR remains at 0.13%, whereas ASR after quantization varies substantially with layer depth. Middle layers (L11 and L18) maintain high ASR across all three quantization formats, while shallow (L6) and very deep (L22 and L25) layers show clear drops after quantization. When ℓ∗\ell^{\ast} is a shallow layer, the target behavior encoded in PP\mathcal{P} _ P must pass through many subsequent fixed layers in PF\mathcal{P} _ F, which weakens its effect on the output. When ℓ∗\ell^* is very close to the output, few layers remain to form the target response, and PS\mathcal{P} _ S spans most of the preceding layers in the network, so quantization errors accumulate across more layers. Between the two middle layers, L11 reaches 100.00% ASR after quantization but yields the lowest TruthfulQA scores (0.3011/0.4585), whereas L18 maintains high ASR after quantization (91.73% to 99.80%) and achieves the highest benign utility (0.3244/0.4912). Thus, the middle layer with the highest initial sink attention is not the only layer with high ASR, but offers the best trade-off between attack effectiveness and benign utility.

Fixing the target layer to L18, we also compare our selection with random head selection (averaged over five randomly sampled head sets to account for sampling variance, reported as mean ± standard deviation) and a low-sink head set.

Head selectionBF16↓NF4↑FP4↑INT8↑MC1↑MC2↑
Random0.13 ± 0.0098.69 ± 2.4298.20 ± 2.3389.72 ± 6.200.3084 ± 0.01740.4561 ± 0.0170
Low-sink0.1399.6099.3388.870.30350.4712
Ours0.1399.8098.4091.730.32440.4912

All head sets learn the target behavior in Stage 2 and keep ASR low at full precision and high after quantization, while our selection preserves higher benign utility (0.3244/0.4912 vs. 0.3084/0.4561 for random and 0.3035/0.4712 for low-sink) because suppressing heads on which the original model relies less reduces utility loss at full precision. We have added both comparisons to Appendix D.1 (Layer and Head Selection) in the revised manuscript.

To test how sink attention suppresses malicious behavior and whether it depends on the selected heads, Tables 3 and 4 in Section 5.4 provide direct ablations on Qwen2.5-1.5B content injection. With the output-projection slices trained in Stage 2 frozen, Stage 3 raises sink attention in the selected heads from 0.0022 to 0.9648 and reduces BF16 ASR from 91.13% to 0.13%, whereas INT8 quantization returns their sink attention to 0.0025 and restores ASR to 91.73% (Table 3). At inference time, zeroing the selected heads of the quantized model drops ASR to at most 0.13%, while zeroing an equal number of random non-selected heads leaves it between 85.20% and 99.78% (Table 4).


References

[1] Egashira, K., Vero, M., Staab, R., He, J., & Vechev, M. (2024). Exploiting LLM quantization. Advances in Neural Information Processing Systems, 37, 41709-41732.
[2] Zhan, X., Egashira, K., Staab, R., Vero, M., & Vechev, M. (2026). Widening the Gap: Exploiting LLM Quantization via Outlier Injection. In Trustworthy AI for Good (AI4GOOD) Workshop @ ICML 2026.
[3] Egashira, K., Staab, R., Vero, M., He, J., & Vechev, M. (2025). Mind the gap: A practical attack on GGUF quantization. In Proceedings of the 42nd International Conference on Machine Learning (Vol. 267, pp. 15038-15065). PMLR.
[4] Dong, P., Li, H., & Guo, S. (2025). Durable quantization conditioned misalignment attack on large language models. In International Conference on Learning Representations (Vol. 2025, pp. 21789-21801).
[5] Fu, Z., Zeng, W., Wang, R., & Li, M. (2026). Attention sink forges native MoE in attention layers: Sink-aware training to address head collapse. arXiv preprint arXiv:2602.01203.

Reviewer_hiT2OpenReview ↗
Rating
4(+1)
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes SinkQCB, a quantization-conditioned backdoor attack against LLMs. The key idea is to separate the parameters that encode malicious behavior from those that control its contribution to model outputs. The method first reduces sink attention in selected heads (stage 1), then trains their output-projection slices to encode malicious behavior (stage 2). Finally, it restores benign behavior at full precision by increasing sink attention while restricting parameter updates to preserve the malicious model’s quantized values (stage 3). Experiments across three model families, three attack tasks, and three quantization formats show that the full-precision model behaves normally while its quantized model outputs malicious behavior.

Critical strengths

This is a good paper. The paper presents a clear use of attention sinks to control quantization-conditioned malicious behavior, supported by strong attack results and informative head-level interventions.

  1. Using attention sinks to suppress or expose the contribution of selected heads for quantization-conditioned attacks is very insightful.

  2. The training objectives make sense and support the proposed mechanism. The experimental results also support SinkQCB's effectiveness.

  3. The paper is organized well and easy to follow.

Critical weaknesses

Major concerns:

  1. I notice that the method selects heads with high initial sink attention as the starting point (before stage 1), but subsequent training (stage 1, stage 2, and stage 3) actually changes their attention distributions. So, I am not sure if this selection is necessary. Can random or low-sink heads achieve similar results under the same procedure?

  2. The entire optimization includes different hyperparameters like λsel,λoth,λout,λrep,λdist\lambda_{sel}, \lambda_{oth}, \lambda_{out}, \lambda_{rep}, \lambda_{dist}. I am not sure if the final attack results are very sensitive to these hyperparameters. It would be better if the paper could provide some sensitivity analysis on these hyperparameters.

  3. In Equation (10), the term �0��0� encourages the selected heads to attend more strongly to the sink. However, increased sink attention does not necessarily reduce their contribution or suppress malicious behavior unless the sink’s projected value contribution remains small. Line 168 states that sink value vectors often have smaller norms than those of ordinary tokens, but how is this property maintained during Stage 3? Although the second term in Equation (9) penalizes �1��1� during Stage 2, Stage 3 updates upstream parameters, which can change �2��2� even when �3��3� is frozen. Could the projected sink contribution increase during restoration? Would it be necessary to retain a regularization term similar to the one in Equation (9) in the Stage 3 objective?

Minor concerns:

  1. I suggest the paper could give a more detailed explanation of why the quantized version of the final model after Stage 3 could restore the malicious behavior. I stopped at lines 210 to 214 for a long time and tried to figure out the reason. The key idea became clear after I read Section 4.2: Stage 3 changes the full-precision weights to restore benign behavior while constraining them to preserve the malicious model’s Stage 2 quantized values.

I will increase my score if the author could address my concerns.

Questions for authors

See weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer hiT2 (Part 1/2)

Thank you for your positive assessment of the mechanism, the experimental results, and the presentation. We address your concerns below.

W1: Random or low-sink head selection.

We choose heads by initial sink attention to limit the utility loss at full precision. In Stage 2, the output-projection slices of the selected heads are trained to produce the malicious behavior, and they stay frozen in Stage 3. Stage 3 then changes the sink attention of these heads so that they are suppressed at full precision. If the original model relies little on these heads, suppressing them may have a smaller effect on benign utility.

We compare our selection with random head selection (averaged over five randomly sampled head sets to account for sampling variance, reported as mean ± standard deviation) and a low-sink head set on Qwen2.5-1.5B-Instruct for content injection under the same training and evaluation settings.

Head selectionBF16↓NF4↑FP4↑INT8↑MC1↑MC2↑
Random0.13 ± 0.0098.69 ± 2.4298.20 ± 2.3389.72 ± 6.200.3084 ± 0.01740.4561 ± 0.0170
Low-sink0.1399.6099.3388.870.30350.4712
Ours0.1399.8098.4091.730.32440.4912

Random and low-sink heads also achieve low ASR at full precision and high ASR after quantization. Our selection preserves higher benign utility, with MC1/MC2 scores of 0.3244/0.4912, compared with averages of 0.3084/0.4561 for random selection and 0.3035/0.4712 for low-sink selection. These results support our motivation for selecting heads to limit utility loss during full-precision restoration. We have added this comparison to Appendix D.1 (Layer and Head Selection) in the revised manuscript.


W2: Sensitivity of hyperparameters.

Thank you for raising this concern. Overall, the final attack results are not overly sensitive to these hyperparameters. Specifically, four of the five loss coefficients are fixed across all three models and all three attack tasks with λsel=λoth=λout=0.5\lambda_{\mathrm{sel}}=\lambda_{\mathrm{oth}}=\lambda_{\mathrm{out}}=0.5 and λdist=3.0\lambda_{\mathrm{dist}}=3.0 (Appendix C), while only λrep\lambda_{\mathrm{rep}} is set between 1.0 and 8.0. To isolate the effect of each coefficient, we run a controlled sweep on Qwen2.5-1.5B-Instruct for code injection around a single reference run (λsel=λoth=λout=0.5\lambda_{\mathrm{sel}}=\lambda_{\mathrm{oth}}=\lambda_{\mathrm{out}}=0.5, λdist=3.0\lambda_{\mathrm{dist}}=3.0, λrep=8.0\lambda_{\mathrm{rep}}=8.0), halving or doubling one coefficient at a time while keeping the others fixed.

CoefficientValueBF16↓NF4↑FP4↑INT8↑MC1↑MC2↑
Reference--0.0083.5495.7395.120.28030.4794
λsel\lambda_{\mathrm{sel}}0.250.0092.6886.5994.510.30600.4760
λsel\lambda_{\mathrm{sel}}1.000.0093.90100.0098.780.30720.4809
λoth\lambda_{\mathrm{oth}}0.250.0084.1596.3496.950.31460.4982
λoth\lambda_{\mathrm{oth}}1.000.0092.0797.5699.390.27780.4609
λout\lambda_{\mathrm{out}}0.250.0087.2096.9596.340.28640.4361
λout\lambda_{\mathrm{out}}1.000.0098.7898.78100.000.28520.4494
λrep\lambda_{\mathrm{rep}}4.000.0082.9395.1296.950.29010.4770
λrep\lambda_{\mathrm{rep}}16.000.0081.1094.5196.950.29010.4652
λdist\lambda_{\mathrm{dist}}1.500.0083.5494.5197.560.29380.4862
λdist\lambda_{\mathrm{dist}}6.000.0077.4493.2998.170.30720.4843
AuthorsOpenReview ↗

Response to Reviewer hiT2 (Part 2/2)

Across these variations, all checkpoints maintain zero ASR at BF16 while achieving 77.44%–100.00% ASR after quantization and stable TruthfulQA utility, confirming that SinkQCB remains effective without sensitive hyperparameter tuning. Increasing λsel\lambda_{\mathrm{sel}} to 1.0 more strongly shifts the selected heads' attention away from the sink token in Stage 1, giving their output-projection slices stronger non-sink signal to encode the malicious behavior in Stage 2 and thereby improving ASR across all three quantization formats. Increasing λoth\lambda_{\mathrm{oth}} to 1.0 more strongly encourages the layer output to be dominated by the selected heads in Stage 1, amplifying the contribution of the poisoned heads after quantization and raising ASR across all three formats, at the cost of a slight utility drop due to reduced contributions from the remaining heads. Increasing λrep\lambda_{\mathrm{rep}} to 16 places stronger emphasis on maximizing full-precision sink attention within the Stage 2 quantization-preserving intervals, which can shift the constrained parameters closer to quantization boundaries and slightly reduce ASR under NF4 and FP4 quantization. Increasing λdist\lambda_{\mathrm{dist}} from 3.0 to 6.0 improves both TruthfulQA scores over the reference while reducing NF4 and FP4 ASR, reflecting a trade-off between utility recovery and attack retention. Increasing λout\lambda_{\mathrm{out}} more strongly suppresses the selected heads’ output contribution at the sink token, encouraging the malicious behavior to be encoded through non-sink tokens, which is consistent with the higher ASR across all three quantization formats when λout\lambda_{\mathrm{out}} is increased to 1.0. We have added these analyses to Appendix D.2 (Hyperparameter Sensitivity) in the revised manuscript.


W3: Contribution of attention sinks.

Because Stage 3 updates the upstream parameters in PS\mathcal{P}_S, vs(h)v_s^{(h)} is not fixed and the projected sink contribution ∥WO(h)vs(h)∥22\Vert W_O^{(h)}v_s^{(h)}\Vert_2^2 shows minor changes during restoration in our experiments. Nevertheless, it is not necessary to retain a regularization term similar to Equation (9) in the Stage 3 objective. First, Stage 2 already penalizes ∥WO(h)vs(h)∥22\Vert W_O^{(h)}v_s^{(h)}\Vert_2^2 while training WO(h)W_O^{(h)}, and WO(h)W_O^{(h)} remains frozen in Stage 3, which discourages the malicious behavior from being encoded through the sink token. Second, Stage 3 optimizes clean NLL and KL distillation toward the original benign model on clean data, so as sink attention approaches 1, any substantial increase in the projected sink output that disturbs benign predictions is naturally penalized by these restoration objectives. In our experiments, the target behavior remains consistently suppressed at full precision without adding an explicit sink-norm penalty in Stage 3.


Minor concern: Detailed explanation of Stage 3.

Thank you for pointing out the unclear explanation in lines 210–214. In the third step (corresponding to Stage 3), each routing-related parameter is constrained to an interval whose full-precision values all quantize to its second-step (Stage 2) value. Quantization therefore recovers the second-step router, which routes inputs to the selected expert. Since this expert remains frozen after the second step, it still produces the learned target behavior. We have made this explanation explicit in Section 4.1.

AuthorsOpenReview ↗

Gentle Reminder before the Discussion Deadline

Dear Reviewer hiT2,

Thank you again for your valuable time and reviews. We are writing to kindly check whether our rebuttal has addressed your concerns. If so, we would appreciate it if you would consider updating your rating. If any questions remain, we are more than happy to discuss further before the discussion phase ends.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal Updates and Manuscript Revisions

Dear Area Chairs and Reviewers,

We sincerely thank the Area Chair and all three reviewers for their time and constructive feedback. We are encouraged that the reviewers recognized the practical relevance of our supply-chain threat model (Reviewer 2i3S), the insight of decoupling malicious-behavior encoding from sink-attention gating in dense Transformers (Reviewers hiT2, 2i3S), and our three-stage training design and head-level analyses (Reviewers hiT2, K9eD). During the rebuttal, we conducted additional experiments, clarified our mechanism, and revised the manuscript to address all remaining concerns. Below is a brief summary of our main updates:

Novelty and methodological contribution (Reviewers K9eD, 2i3S). We clarified how SinkQCB differs from prior quantization-conditioned backdoors and benign attention-sink studies. Rather than simply combining existing techniques, SinkQCB confines malicious behavior to a few attention heads and uses precision-dependent sink attention to keep them suppressed at full precision and activated after quantization without input triggers or explicit routers. We detailed the technical challenges unique to this setting and how our training objectives resolve them.

Additional baselines and fair comparisons (Reviewers K9eD, 2i3S). We reproduced the concurrent work of Zhan et al. (2026) across all nine model-task settings and incorporated the results into Tables 1 and 2. SinkQCB achieves lower BF16 ASR in all nine settings, higher or comparable post-quantization ASR with better stability across formats, and higher TruthfulQA utility in eight of the nine settings. We also re-evaluated Egashira et al. (2024) on Mistral-7B code injection across multiple poisoning learning rates, demonstrating the trade-off between post-quantization ASR and benign utility.

Layer/head selection and mechanism validation (Reviewers hiT2, K9eD). We added comparisons across five target layers as well as random and low-sink head selection in Appendix D.1, showing that our selection provides the best trade-off between attack effectiveness and full-precision utility. We also highlighted how sink-attention measurements and head-level interventions (Tables 3 and 4) confirm the causal role of the selected heads.

Hyperparameter sensitivity and constrained optimization (Reviewers hiT2, 2i3S). We added sensitivity experiments across all five loss coefficients in Appendix D.2, confirming that SinkQCB remains effective without sensitive tuning. We also explained why parameter separation simplifies optimization under quantization-preserving constraints and why an explicit sink-norm penalty is unnecessary in Stage 3.

Numerical consistency and presentation (Reviewers hiT2, 2i3S). We clarified the minor INT8 ASR difference between Stages 2 and 3 caused by BF16 rounding near quantization boundaries (Section 5.4), expanded the explanation of Stage 3 weight constraints (Section 4.1), and corrected the attention-head and projection labels in Figure 2(b).

We thank Reviewer hiT2 for following up to confirm that their concerns have been addressed and for maintaining a positive assessment of our work. We believe that our responses, new experiments, and manuscript revisions have thoroughly addressed the questions raised by all reviewers, and we respectfully ask the Area Chairs and Reviewers to take these updates into account in their final assessment.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.