acceptodds
Under review as a conference paper at ICLR 2027

ReQOR: Operator-Discrepancy Subspace Repair for Quantization-Conditioned Backdoors in Large Language Models

Abstract

Quantization-Conditioned Backdoors (QCBs) cause large language models (LLMs) to exhibit malicious behavior after quantization while remaining benign at high precision. Existing defenses often rely on security-guided configuration selection or iterative optimization, but do not identify which quantization-induced changes should be repaired. We isolate module-local quantization effects from upstream drift by evaluating the high-precision and quantized versions of each module on identical states from the quantized inference trajectory, and formulate QCB defense as repairing the resulting same-state operator discrepancies. We propose ReQOR, a training-free defense that identifies the dominant directions of these discrepancies on benign calibration data and restores the full-precision computation along them in closed form, yielding a factorized residual adapter. ReQOR uses neither attack information nor gradient-based optimization and supports both full-span and low-rank repair. Experiments across four LLMs, three QCB attacks, three malicious behaviors, and three quantization formats show substantial QCB mitigation while largely preserving benign utility. Further analysis shows that dominant operator-discrepancy directions provide more effective and rank-efficient repair than rank-matched random, low-energy, or raw weight-error directions, while low-rank ReQOR reduces deployment memory overhead.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Quantization-Conditioned Backdoor, Large Language Models, LLM Security, Model Quantization, Backdoor Defense, AI Safety, Trustworthy AI

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_zY67OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReQOR, a training-free defense against quantization-conditioned backdoors in LLMs. The method identifies module-level quantization discrepancies by evaluating full-precision and quantized operators on the same states from the quantized inference trajectory, and constructs a repair subspace from these discrepancies. The resulting correction is compiled into a factorized residual adapter that can be deployed with the quantized model without retaining the full-precision checkpoint. The paper evaluates ReQOR on four instruction-tuned LLMs, three QCB attacks, three malicious behaviors, and three quantization formats, showing strong attack mitigation while largely preserving benign utility. Additional analyses show that dominant discrepancy directions provide more effective low-rank repair than random or raw weight-error directions, and that rank-32 repair substantially reduces memory overhead compared with full-span repair

Critical strengths

1. The defense operates under realistic deployment constraints and achieves strong mitigation in the evaluated settings. ReQOR requires only benign calibration data and does not rely on attack labels, malicious samples, or attack-specific feedback. In Table 2, rank-32 ReQOR reduces Qwen2.5-3B ELQ jailbreak ASR from 82.00% to 0.00% and ACL content-injection ASR from 76.44% to 0.00% while maintaining similar benign utility. These results show that the defense can substantially reduce QCB behavior without the large utility degradation observed for some baselines.

2. The paper provides a clear analysis of why the proposed repair space is effective. The factorized residual formulation makes the correction explicit and avoids storing the full-precision model at deployment. The matched-rank comparisons in Table 3 show that dominant discrepancy directions are more effective than random, low-energy, or raw weight-error directions in the evaluated cases, supporting the role of the proposed repair subspace. The rank-memory analysis further shows the trade-off between full-span and low-rank repair.

Critical weaknesses

1. The distinction from activation-aware error reconstruction methods is not sufficiently clear. ReQOR builds its correction from an activation-dependent operator discrepancy subspace, which shares similarities with prior activation-aware quantization error reconstruction methods such as QERA. The paper discusses differences in terms of trajectory selection and the QCB setting, but it does not clearly show which part of the proposed formulation is responsible for an advantage over these methods. The paper mentions QERA and EoRA but does not include them in the empirical comparison. Improvements over random directions, weight-space SVD, and L2 QER-style reconstruction do not by themselves establish an advantage over the closest activation-aware baselines. A direct comparison with QERA or similar methods would help clarify the contribution of ReQOR.

2. The paper does not clearly show the benefit of using quantized-trajectory calibration. The same-state operator discrepancy removes upstream state drift by construction, but the experiments do not directly test whether collecting states from the quantized inference trajectory is important for the observed gains. The current ablations mainly compare different repair directions and reconstruction choices, so it remains unclear how much of the improvement comes from the trajectory-based formulation itself rather than from using activation-dependent corrections in general. In addition, a large discrepancy on benign calibration data does not necessarily imply that the corresponding directions are relevant to QCB suppression. Although ReQOR works on the evaluated attacks, the paper does not examine whether these directions remain effective across different calibration sets, domains, or adaptive attack scenarios. The calibration analysis provides some evidence, but it is limited to full-span repair on a single model and attack setting.

Questions for authors

1. How does ReQOR differ from activation-aware reconstruction methods such as QERA? When using the same calibration states and repair rank, what is the main difference between ReQOR and QERA-style corrections? In particular, how does the method differ in the bias-free linear case, and which part of the design (e.g., trajectory selection, covariance handling, or the QCB setting) is responsible for the improvement?

2. How important is the use of quantized inference trajectories? With the same correction rule and calibration budget, how does ReQOR perform when the repair subspace is built from full-precision trajectories or simpler prompt-based calibration states?

3. How stable is low-rank ReQOR across different calibration settings? Does rank-32 repair maintain similar performance with different benign calibration samples or domains, especially for cases where some attack success remains? How much of this robustness is supported by the current experiments, and how much remains to be tested under adaptive attacks?

4 comments
AuthorsOpenReview ↗

Response to Reviewer zY67 (1/3)

We sincerely thank the reviewer for their careful assessment of our work and constructive suggestions for clarifying our contributions and strengthening our experimental evaluation.

Relation to Activation-Aware Reconstruction (QERA/EoRA) ([W1][Q1])

Please refer to our common response, “Relation to Activation-Aware Reconstruction (QERA/EoRA),” where we provide the mathematical relationship between ReQOR and QERA/EoRA, along with controlled comparisons under matched calibration conditions.

Importance of Quantized-Trajectory Calibration ([W2][Q2])

We agree that our original ablations kept the state-collection procedure fixed. We therefore compared six state sources across nine settings with the same correction rule, calibration budget, and rank (r=1r=1): autoregressive generation, teacher forcing, and prompt-only states, each collected with the quantized or the full-precision model.

State collectionAvg. (Quantized)Avg. (Full-precision)Worst (Quantized)Worst (Full-precision)
Autoregressive generation0.002.370.0012.67
Teacher forcing0.071.710.676.67
Prompt-only0.070.300.672.00

ASR (%) across nine settings; details are in our response to Reviewer sXWh [W1][Q4].

The important factor is collecting states with the quantized model rather than the full-precision model. With generation, full-precision states leave residual ASR in five of the nine settings (up to 12.67%), whereas quantized states reach 0% in all nine. Among quantized-model states, the collection method made little difference (0.00–0.07% average ASR), so autoregressive generation is our default way of collecting such states, not a requirement.

Regarding whether the gain comes from the trajectory or from activation-dependent correction in general, our common response on QERA/EoRA addresses this directly. These methods are activation-dependent corrections, yet with their original full-precision calibration they leave residual ASR, and they also improve when calibrated on quantized-model states. Activation dependence alone is thus not sufficient, and the benefit of quantized-model states is not specific to ReQOR.

We have added this ablation to Appendix L.1 and clarified in Section 5.6 that the key requirement is the use of quantized-model states.

AuthorsOpenReview ↗

Response to Reviewer zY67 (2/3)

Relevance of Benign Discrepancy Directions ([W2])

We agree that a large discrepancy on benign data does not by itself imply relevance to QCB suppression. We therefore tested this through a decomposition of the full-precision–quantized weight difference, using the intermediate malicious checkpoint of each attack. We found that the directions ReQOR identifies from benign data capture the components that remove the backdoor.

QCB attack mechanism. The evaluated PGD-based QCB attacks involve two stages. First, malicious behavior is injected into a full-precision (FP) model. Second, PGD suppresses this behavior in the FP model while preserving its activation after quantization. Let WMW_M denote the malicious FP model after Stage 1, WFW_F the FP model after Stage 2, and WQW_Q the common quantized model. The discrepancy seen by ReQOR therefore decomposes as

WF−WQ=(WF−WM)+(WM−WQ),W_F-W_Q=(W_F-W_M)+(W_M-W_Q),

where WF−WMW_F-W_M is the Stage-2 update that makes the full-precision model benign. WMW_M is unavailable to the defender and is used only for this analysis. ReQOR's rank-rr projector PrP_r is computed exactly as in the paper, from benign calibration data and without WMW_M.

We investigated this relationship through two complementary analyses, over nine configurations (ACL/Content Injection/Qwen2.5-3B, ELQ/Jailbreak/Gemma-2-2B, Q-Misalign/Over-refusal/Llama-3.2-3B × INT8, NF4, FP4).

First, we measured how much of the operator-level energy of the Stage-2 update lies in ReQOR's rank-rr subspace, on the same benign quantized-model states used for calibration. ReQOR's subspace captures 35.15% at rank 1 and 48.63% at rank 32, compared with only 0.03% and 1.02% for random subspaces of the same rank.

Second, we tested whether this captured component is sufficient to suppress the malicious behavior. We added to the Stage-1 model only the part of the Stage-2 update that lies in the subspace and discarded the rest:

Wtest=WM+Pr(WF−WM).W_{\mathrm{test}}=W_M+P_r(W_F-W_M).

We then measured ASR and compared the results with matched-rank random projections.

Model variantAverage ASR (%) ↓
Stage-1 malicious FP53.26
Random-projected update (Rank 1)51.93
Random-projected update (Rank 32)52.96
ReQOR-projected update (Rank 1)7.11
ReQOR-projected update (Rank 32)0.07
Stage-2 FP0.00

Average over nine QCB configurations. Random projections use seed 0.

With ReQOR's rank-32 subspace, ASR is 0% in eight of the nine configurations and 0.67% in the remaining one. In contrast, random subspaces leave the malicious behavior of the Stage-1 model almost unchanged.

These results indicate that ReQOR's benign-calibrated directions are not merely large quantization errors: they align with the weight changes that suppress the backdoor, and the component they capture is sufficient to suppress the malicious behavior in all evaluated attacks.

AuthorsOpenReview ↗

Response to Reviewer zY67 (3/3)

Calibration Analysis and Adaptive Attacks ([W2][Q3])

To examine whether ReQOR remains effective across different calibration sets and domains, we extended the calibration analysis to a wider range of conditions: both rank-32 and full-span repair, four calibration domains (Dolly, SODA, GSM8K, MBPP), three additional disjoint Dolly subsets, and five settings including two where residual ASR remains. All runs use the same budget (N=50N=50, T=16T=16).

SettingRepairDolly (3 subsets)SODAGSM8KMBPP
Three representative settings‡Rank-32 / Full0.00 ± 0.000.000.000.00
ELQ / Injection / Qwen2.5-1.5BRank-321.11 ± 0.441.782.005.33
Full0.30 ± 0.130.221.330.22
Q-Misalign / Jailbreak / Gemma-2-2BRank-325.56 ± 0.224.6710.2210.89
Full1.70 ± 0.341.341.561.33

ASR (%), averaged over INT8, NF4, and FP4. Dolly reports the mean ± sample standard deviation across three disjoint Dolly calibration subsets. ‡ ACL/Injection/Qwen2.5-3B, ELQ/Jailbreak/Gemma-2-2B, and Q-Misalign/Over-refusal/Llama-3.2-3B.

In the three representative settings, both rank-32 and full-span repair reach 0% ASR under every calibration set. In the two settings where residual ASR remains, full-span repair is stable (0.22–1.33% and 1.33–1.70%), whereas rank-32 repair varies more (1.11–5.33% and 4.67–10.89%). Rank-32 repair therefore maintains its performance where it already suppresses the attack; in the cases where some attack success remains, it varies across calibration sets, but even its worst case (10.89%, for Q-Misalign/Jailbreak/Gemma-2-2B) is far below the undefended ASR in that setting (84.22%).

We further evaluated the outlier-injection QCB suggested by Reviewer pFKM [Q2], which differs from the PGD-based attacks in our original evaluation. Across eight attack-activated settings, ReQOR reduces average ASR from 68.17% to 6.92% (rank-32) and 0.33% (full-span), providing additional evidence of robustness across different QCB constructions.

However, none of these attacks is optimized against ReQOR, so our results do not establish robustness to adaptive attacks. The PGD-based attacks constrain only the weights, keeping them within the intervals that preserve the quantized values. To evade ReQOR, an adaptive attacker would additionally need to be activation-aware: the weight changes that make the full-precision model benign would have to stay out of the discrepancy subspace observed on benign activations. Whether this is achievable while preserving the quantized behavior is an open question. In the current attacks, these changes are clearly exposed (see our response on [W2], relevance of benign discrepancy directions), and calibration on unrelated domains such as math or code still suppresses the jailbreak attack.

We have acknowledged this limitation in Section 6 (Conclusion) of the revised manuscript and identified adaptive attacks specifically designed to evade ReQOR as a direction for future work.

AuthorsOpenReview ↗

Follow-up on Rebuttal and Revisions for Reviewer zY67

Dear Reviewer zY67,

We would like to briefly follow up on our rebuttal and revised manuscript. We have addressed your concerns through several additional analyses, as detailed in our responses:

  • QERA/EoRA comparison ([W1][Q1]): Mathematical analysis and controlled comparisons under matched conditions (Common Response to Reviewers sXWh and zY67).
  • Quantized-trajectory calibration ([W2][Q2]): Controlled comparison of six calibration-state sources (Response 1/3).
  • Relevance of benign discrepancy directions ([W2]): Analysis showing that the identified directions capture components responsible for suppressing malicious behavior (Response 2/3).
  • Calibration robustness and adaptive attacks ([W2][Q3]): Additional evaluations across calibration domains and subsets, including challenging settings, and discussion of adaptive-attack limitations (Response 3/3).

In addition, following Reviewer pFKM's suggestion, we evaluated a non-PGD outlier-injection attack to assess generalization beyond the original PGD-based attacks. Under this attack, existing baselines largely fail, whereas ReQOR substantially outperforms them, reducing average ASR from 68.17% to 6.92% (rank 32) and 0.33% (full-span) while preserving benign utility. These results are presented in Table 3 (Section 5.2) and Appendix F, with further discussion in our response to Reviewer pFKM ([Q2]).

We have also incorporated the relevant findings into the revised manuscript.

We would greatly appreciate hearing whether our responses have addressed your main concerns or if any questions remain. With the discussion period nearing its end, we would be happy to provide further clarification as promptly as possible.

Thank you again for your thoughtful feedback.

Reviewer_sXWhOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies quantization-conditioned backdoors in LLMs and proposes ReQOR, a training-free defense using benign calibration prompts. It measures differences between full-precision and quantized linear operators on identical states from quantized generation trajectories, extracts dominant output directions through SVD, and restores full-precision components using closed-form residual adapters. Experiments across four models, three attacks, three malicious behaviors, and three quantization formats show substantial mitigation with generally preserved average utility.

Critical strengths
  1. ReQOR offers a practical defense construction using only the supplied full-precision checkpoint, its quantized counterpart, and benign prompts. It requires neither attack-specific feedback nor gradient optimization, and the exact factorized correction allows deployment without retaining the full-precision model.

  2. The evaluation combines multiple attack and quantization settings with controlled direction ablations. Full-span repair substantially suppresses attacks while improving average MMLU across all four tested models, and matched-rank interventions provide evidence that the selected directions matter beyond simply adding repair capacity.

Critical weaknesses
  1. The evidence for the calibration design is incomplete. Calibration uses only Dolly prompts, and the direction ablations keep the state-collection procedure fixed. The results therefore leave both sensitivity to calibration domain and the additional benefit of quantized generation trajectories unclear, limiting confidence in the generality and necessity of this central design.

  2. The trade-off among mitigation, benign utility, and deployment cost is incompletely characterized. Favorable overall averages conceal substantial utility losses for rank-32 repair in some configurations, while full-span repair increases the average model footprint by about 74%. Calibration runtime, peak construction memory, and inference throughput are not quantified, leaving the practical choice between the two variants insufficiently supported.

Questions for authors
  1. How sensitive is ReQOR to the domain of the benign calibration data? Would calibration on code, mathematics, or everyday questions provide comparable protection under the same calibration budget and repair rank?
  2. How should a defender choose the repair rank without security feedback while preserving benign utility? The configuration-level results suggest that rank-32 repair can substantially reduce utility in some settings despite favorable overall averages.
  3. What distinguishes ReQOR’s repair construction from activation-aware quantization-error reconstruction methods such as QERA and EoRA? Could a matched-rank comparison clarify the additional benefit of ReQOR?
  4. How much does ReQOR benefit from collecting states along quantized generation trajectories? Could you compare against prompt-only states and states from full-precision generation under a matched calibration budget and repair rank?
4 comments
AuthorsOpenReview ↗

Response to Reviewer sXWh (1/2)

We sincerely thank the reviewer for their positive assessment of our work and for their thoughtful and constructive feedback.

Sensitivity to Calibration Domain and Samples ([W1][Q1])

Our additional experiments show that ReQOR maintains strong attack mitigation across diverse benign calibration domains and independently sampled Dolly subsets, with full-span repair showing greater stability than Rank-32.

We evaluated four benign datasets: Dolly (general instruction following), SODA [1] (everyday conversations), GSM8K [2] (mathematics), and MBPP [3] (code generation). We also tested three newly sampled, disjoint Dolly calibration subsets. All experiments used the same calibration budget (N=50,T=16)(N=50, T=16) and repair ranks (32 and full-span) across INT8, NF4, and FP4.

SettingRepairDolly (3 subsets)SODAGSM8KMBPP
ACL / Injection / Qwen2.5-3BRank-320.00 ± 0.000.000.000.00
Full0.00 ± 0.000.000.000.00
ELQ / Jailbreak / Gemma-2-2BRank-320.00 ± 0.000.000.000.00
Full0.00 ± 0.000.000.000.00
Q-Misalign / Over-refusal / Llama-3.2-3BRank-320.00 ± 0.000.000.000.00
Full0.00 ± 0.000.000.000.00
ELQ / Injection / Qwen2.5-1.5BRank-321.11 ± 0.441.782.005.33
Full0.30 ± 0.130.221.330.22
Q-Misalign / Jailbreak / Gemma-2-2BRank-325.56 ± 0.224.6710.2210.89
Full1.70 ± 0.341.341.561.33

ASR (%), averaged over INT8, NF4, and FP4. Dolly reports mean ± sample standard deviation across three newly sampled, disjoint calibration subsets.

Both Rank-32 and full-span ReQOR achieved 0% ASR across all calibration domains and Dolly subsets in three settings. In the remaining two settings, full-span repair consistently maintained low ASR, while Rank-32 showed greater variation across calibration domains.

These results indicate that ReQOR does not depend on a particular Dolly subset or calibration domain. Full-span repair provides more consistent mitigation across the evaluated domains, while Rank-32 offers a more memory-efficient alternative.

Benefit of Quantized-Trajectory Calibration ([W1][Q4])

Under the same calibration budget, quantized-model states give lower ASR than full-precision states for every state-collection procedure, while the procedure itself makes little difference on the quantized model.

We compared six state sources: autoregressive generation, teacher forcing, and prompt-only states, each collected with either the quantized or the full-precision model. We fixed the repair rank at r=1r=1 to make differences between state sources more apparent, while keeping the calibration budget and repair procedure unchanged.

State collectionAvg. (Quantized)Avg. (Full-precision)Worst (Quantized)Worst (Full-precision)
Autoregressive generation0.002.370.0012.67
Teacher forcing0.071.710.676.67
Prompt-only0.070.300.672.00

ASR (%) across nine settings (three attack settings × three quantization formats), with repair rank r=1r=1. ReQOR's default calibration source is quantized autoregressive generation (XQgenX_Q^{gen}).

With generation, full-precision states leave residual ASR in five of the nine settings (up to 12.67%), whereas quantized states reach 0% in all nine. These results support calibrating on quantized-model states. Among quantized-model states, the collection method made little difference (0.00–0.07% average ASR). Teacher forcing, however, requires reference responses, whereas generation and prompt-only collection need only the prompts.

References

[1] Kim, Hyunwoo, et al. "Soda: Million-scale dialogue distillation with social commonsense contextualization." Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. 2023.

[2] Cobbe, Karl, et al. "Training verifiers to solve math word problems." arXiv preprint arXiv:2110.14168 (2021).

[3] Austin, Jacob, et al. "Program synthesis with large language models." arXiv preprint arXiv:2108.07732 (2021).

AuthorsOpenReview ↗

Response to Reviewer sXWh (2/2)

Mitigation–Utility–Deployment Trade-off and Rank Selection ([W2][Q2])

We agree that averages alone do not show how utility varies across configurations. We therefore report configuration-level utility and examine the configurations where rank-32 repair loses the most utility. In addition to the model footprint reported in the paper, we measure construction time, peak construction memory, and inference throughput.

Configuration-level utility and rank sensitivity. We define severe degradation as a drop of more than 5 percentage points in either MMLU or TruthfulQA-MC2 relative to the full-precision checkpoint. Across 107 configurations, it occurs in 46 cases for the undefended quantized models, 16 for rank-32 ReQOR, and 4 for full-span ReQOR, compared with 14–16 for Gaussian noise, 14 for QuantGuard, 16 for L2QER, 39 for FlipGuard, and 71 for QVec. Of the 16 cases for rank-32 repair, 15 were already severely degraded before defense. The table below shows the three configurations where rank-32 repair loses the most utility (ACL over-refusal on Llama-3.2-3B/NF4, Llama-3.2-3B/FP4, and Qwen2.5-3B/FP4).

RankMMLU ↑TruthfulQA-MC2 ↑ASR ↓
148.4136.811.56
846.3932.981.33
3248.5332.972.67
25653.1039.001.33
Full55.9246.550.67

Average (%) across the three configurations.

ASR is between 0.67% and 2.67% at every rank, close to the full-precision level (0.44%) and far below the undefended models (73.78%). MMLU, which drops from 58.83% in full precision to 38.15% without defense, is recovered to 46–49% for ranks up to 32 and to 53.10% and 55.92% at rank 256 and full span. TruthfulQA-MC2 recovers more slowly: it remains well below the full-precision level (50.75%) at low ranks and approaches it only at full span. In these configurations, a higher rank therefore improves utility rather than mitigation.

Construction and deployment cost. To quantify the practical cost of each variant, we measured construction and inference performance on Qwen2.5-3B under ELQ content injection, averaged across INT8, NF4, and FP4.

VariantConstruction TimePeak VRAMFootprintInference VRAMThroughputLatency
Quantized––2.88 GiB3.24 GiB28.51 tok/s44.58 ms/tok
ReQOR-32161.0 s16.01 GiB2.99 GiB3.35 GiB18.93 tok/s61.60 ms/tok
ReQOR-Full174.6 s16.01 GiB5.48 GiB5.85 GiB19.16 tok/s60.92 ms/tok

The full-precision reference checkpoint occupies 11.49 GiB. Construction is performed once offline, and the full-precision checkpoint is not retained during inference.

Both variants required approximately three minutes for construction, with peak GPU memory usage of 16.01 GiB. At deployment, Rank-32 increased the model footprint by only 3.8% and inference VRAM by 3.4% relative to the quantized model. Across models, Rank-32 increased the average footprint by 3.2%, compared with approximately 74% for full-span repair. Both variants reduced inference throughput by approximately 33%, with little difference between ranks. Thus, Rank-32 primarily offers memory savings rather than inference speed improvements in our current implementation.

Practical rank selection. We fixed r=32r=32 in advance for memory efficiency and matched-rank comparison with L2QER, without using attack feedback or tuning on utility. Since ASR is close to the full-precision level at every rank, the rank need not be tuned using security feedback and can be set in advance from the memory budget. Higher ranks recover more utility at a higher memory cost, while the inference overhead does not depend on the rank.

Distinction from Activation-Aware Reconstruction (QERA/EoRA) ([Q3])

Please refer to our common response, “Relation to Activation-Aware Reconstruction (QERA/EoRA),” where we provide the mathematical relationship between ReQOR and QERA/EoRA, along with controlled comparisons under matched calibration conditions.

AuthorsOpenReview ↗

Follow-up on Rebuttal and Revisions for Reviewer sXWh

Dear Reviewer sXWh,

We would like to briefly follow up on our rebuttal and revised manuscript. We have addressed your concerns through several additional analyses, as detailed in our responses:

  • Calibration domain and sampling sensitivity ([W1][Q1]): Additional experiments across different benign calibration domains and independently sampled subsets (Response 1/2).
  • Quantized-trajectory calibration ([W1][Q4]): Controlled comparison of six calibration-state sources (Response 1/2).
  • Mitigation–utility–deployment trade-off ([W2][Q2]): Configuration-level utility, rank sensitivity, construction costs, inference overhead, and practical rank selection (Response 2/2).
  • QERA/EoRA comparison ([Q3]): Mathematical analysis and controlled comparisons under matched conditions (Common Response to Reviewers sXWh and zY67).

In addition, following Reviewer pFKM's suggestion, we evaluated a non-PGD outlier-injection attack to assess generalization beyond the original PGD-based attacks. Under this attack, existing baselines largely fail, whereas ReQOR substantially outperforms them, reducing average ASR from 68.17% to 6.92% (rank 32) and 0.33% (full-span) while preserving benign utility. These results are presented in Table 3 (Section 5.2) and Appendix F, with further discussion in our response to Reviewer pFKM ([Q2]).

We have also incorporated the relevant findings into the revised manuscript.

We would greatly appreciate hearing whether our responses have addressed your main concerns or if any questions remain. With the discussion period nearing its end, we would be happy to provide further clarification as promptly as possible.

Thank you again for your constructive feedback.

Thanks for the detailed responses and additional experiments. The calibration ablations, matched comparisons with QERA/EoRA, and cost measurements substantially address my main concerns. The comparison clarifies the close relationship to existing reconstruction methods. I will maintain my current score.

Reviewer_pFKMOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper presents a training-free defense against quantization-conditioned backdoors (QCB), named Repairing Quantized Operators via Subspace Restoration (ReQOR). The method first calculates the difference between the hidden states calculated by the full-precision (benign) version and the quantized (malicious) version, and adds an adapter that makes the representation of the quantized version closer to the full-precision version. The effectiveness is shown through a substantial number of experiments with four LLMs, three attack settings, and three methods.

Critical strengths
  • Motivation, method, and connection to related work are all cleanly explained, making the paper easy to follow.
  • The authors provide a substantial amount of experimental results, showing the consistent success of the proposed method.
Critical weaknesses
  • According to Table 2, the baseline Gaussian noise defense seems to work decently well. Given this, it is unclear whether it’s worth having the proposed adapter at the cost of a marginal increase in memory footprint (Section 5.5), which directly goes against the objective of quantization.
Questions for authors
  • I’d be interested in hearing the authors’ opinion about whether the community should focus on “detecting” these attacks to avoid using compromised models, or rather “defending” against QCB attacks so that users can still use the compromised models by removing the malicious behavior from them.
  • The three attacks considered share PGD (training within a range that preserves quantized values) as their underlying idea. Could it be extended to attacks that are based on other, unique intuition, such as an outlier-insertion-based attack [1]?

[1] https://arxiv.org/abs/2605.15152

5 comments
AuthorsOpenReview ↗

Response to Reviewer pFKM (1/2)

We sincerely thank the reviewer for taking the time to carefully evaluate our work and for providing thoughtful and constructive feedback that has helped us further strengthen our paper.

Comparison with Gaussian noise defense ([W1])

We agree that Gaussian noise with σ=10−3\sigma=10^{-3} is competitive on the three attacks in Table 2. However, this effectiveness depends on the choice of σ\sigma and the type of QCB attacks. As shown in Appendix D, a larger noise scale (σ=10−2\sigma=10^{-2}) effectively suppresses QCB attacks but severely degrades benign utility, while a smaller scale (σ=10−4\sigma=10^{-4}) better preserves utility but leaves substantial residual ASR in some settings. Selecting an appropriate noise scale is therefore difficult without attack-specific feedback.

More importantly, the effectiveness of Gaussian perturbation depends on the underlying QCB construction. The three attacks in Table 2 are all PGD-based: they keep the full-precision weights within intervals that preserve the malicious quantized weights, so the injected behavior can be disrupted by appropriately chosen Gaussian perturbations. The outlier-injection attack suggested by the reviewer does not share this property (see [Q2]). Across the eight settings where the attack successfully activated, Gaussian noise with σ=10−3\sigma=10^{-3} leaves an average ASR of 58.67% (attacked: 68.17%), whereas ReQOR reduces it to 6.92% at rank 32 and 0.33% with full-span repair, using the same fixed defense configurations.

Rather than applying random weight perturbations, ReQOR restores full-precision operator components along discrepancy directions identified from benign calibration data. This incurs additional memory, but rank-32 repair increases the average model footprint by only 3.2% across models (Table 4(a)). ReQOR thus provides effective mitigation across distinct QCB constructions with modest memory overhead.

Detection vs. Repair as a Defense Strategy ([Q1])

In our view, both directions are worth pursuing, but for an individual deployer, repair is the more practical choice. Detection is most useful at the ecosystem level, where a flagged checkpoint can be removed from a model hub, and discarding a model known to be compromised is a reasonable response.

For a deployer, however, detection has the following limitations. First, it requires knowing what to look for: in the QCB setting, the malicious behavior is dormant in full precision, and the deployer does not know which behavior may appear after quantization. Second, it can fail silently: a missed detection leaves the attack active without any warning. Third, even a correct detection does not yield a usable model, and a trusted replacement may not exist.

Repair addresses each of these. It can be applied to any model with benign data alone, it does not rely on a detection outcome that may be wrong, and it leaves the deployer with a usable model. To be practical, however, repair must solve two problems of its own: keeping the added deployment cost low, and not degrading a clean model when applied unnecessarily. We believe this is where research on repair should focus. ReQOR is our attempt in this direction: at rank 32, it increases the model footprint by only 3.2% on average (Table 4(a)), and it does not reduce the utility of clean models (Appendix Table 19).

AuthorsOpenReview ↗

Response to Reviewer pFKM (2/2)

Generalization Beyond PGD-Based QCB Attacks ([Q2])

To evaluate whether ReQOR is effective beyond PGD-based QCB attacks, we conducted additional experiments on an outlier-injection QCB, which uses a different attack mechanism. We evaluated three model–behavior combinations: Qwen2.5-3B/content injection, Llama-3.2-3B/over-refusal, and Qwen2.5-7B/jailbreak. We compared ReQOR with Gaussian noise (σ=10−3\sigma=10^{-3}) and QVec (α=0.3\alpha=0.3), and measured benign utility using MMLU and TruthfulQA-MC2.

Model / BehaviorQuant.AttackedGaussianQVecReQOR-32ReQOR-Full
Qwen2.5-3B / Content injectionINT896.0078.6794.670.000.00
NF474.6751.3355.330.670.00
FP4†0.000.001.330.000.00
Llama-3.2-3B / Over-refusalINT860.0056.6768.000.000.67
NF450.0042.6747.331.330.00
FP452.0026.6746.002.000.00
Qwen2.5-7B / JailbreakINT871.3372.6774.000.670.00
NF466.0069.3374.0022.000.67
FP475.3371.3365.3328.671.33

Gaussian uses σ=10−3\sigma=10^{-3} and QVec uses α=0.3\alpha=0.3. † The attack did not activate on Qwen2.5-3B/FP4, so this setting is excluded from the averages below.

Across the eight configurations where the attack was activated, the average ASR was 68.17%. Gaussian noise and QVec reduced it to 58.67% and 65.58%, respectively, whereas ReQOR-32 reduced it to 6.92% and ReQOR-Full to 0.33%. Although ReQOR-32 had higher residual ASR on Qwen2.5-7B under NF4 and FP4, it still outperformed both baselines.

MethodASR ↓MMLU ↑TruthfulQA-MC2 ↑
Attacked68.1763.4052.41
Gaussian58.6762.0252.57
QVec65.5862.2551.76
ReQOR-326.9263.5852.25
ReQOR-Full0.3363.9552.48

ReQOR reduced ASR without lowering benign utility: relative to the attacked models, average MMLU changed by +0.18 (ReQOR-32) and +0.55 (ReQOR-Full) points, compared with −1.38 and −1.15 for Gaussian noise and QVec.

These results show that ReQOR's effectiveness is not limited to PGD-based QCB attacks. It also mitigates outlier-injection QCBs, which rely on a different attack mechanism. The Qwen2.5-7B results further demonstrate its effectiveness beyond the 3B-scale models considered in our main experiments.

We have added these experimental results to Appendix F of the revised manuscript and briefly discussed them in Section 5.2.

Thank you for the rebuttal. While I am quite convinced by the discussion so far and have become more positive in my assessment, I still find the paper itself to be a score of 2 (especially given the sparse grading scheme of 1-4).

Here are the things that I think are missing to cross the bar:

  • I find your rebuttal discussion (W1) particularly convincing with this new experiment, but the description of the outlier-insertion experiment in the PDF is not fully rigorous. I do not think “outlier-injection QCB attack” is a common term that could be understood without the review/rebuttal context.
  • For acceptance, I believe the strength of your approach should be much more clearly visible/articulated in Table 2 and the associated discussion (Section 5.2). The current draft is still unclear about why the proposed method is the best overall.
  • An unusual (sub) caption at the bottom of Table 12: "The attack did not activate on Qwen2.5-3B/FP4; this configuration is excluded from the averages below"; it also appears in other tables.

If these are fully addressed, I would be positive about increasing my score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Common Response to Reviewers sXWh and zY67: Relation to QERA and EoRA (1/2)

We sincerely thank all reviewers for their careful evaluation of our work and their thoughtful and constructive feedback. Below, we address a question raised by both Reviewers sXWh and zY67: how ReQOR differs from activation-aware reconstruction methods such as QERA and EoRA.

Relation to Activation-Aware Reconstruction (QERA/EoRA)

QERA and EoRA both compute a low-rank correction to the quantized weights by minimizing the layer output error on calibration activations. QERA-exact uses the square root of the input autocorrelation matrix RR, while EoRA transforms the weight error into the eigenspace of RR before applying SVD. When RR is invertible, and the calibration states and repair rank are identical, the two methods yield the same correction, and so does ReQOR for bias-free linear modules. ReQOR, however, calibrates on states from the quantized model rather than the full-precision model and, unlike QERA-exact and EoRA, does not require RR to be invertible. This matters in our setting, where the limited benign calibration data makes RR rank-deficient (the average rank⁡(X)/din\operatorname{rank}(X)/d_{\mathrm{in}} is 29.2%).

Mathematical relationship. Consider a bias-free linear module and assume that all methods use the same calibration states XX. Let E=(WF−WQ)⊤E=(W_F-W_Q)^\top be the weight error, R=X⊤X/nR=X^\top X/n, and PVP_V the projection onto the top-rr eigenvectors of E⊤REE^\top RE. When RR is invertible, ReQOR, QERA-exact (Theorem 1), and EoRA all yield C=EPVC=EP_V. In our setting, where RR is rank-deficient, each method yields the following correction:

  • ReQOR: still EPVEP_V, obtained directly from the SVD of the discrepancy matrix XEXE without inverting RR.
  • QERA-exact (Remark 1): with the diagonal perturbation Sϵ=R1/2+ϵIS_\epsilon=R^{1/2}+\epsilon I, the correction is EPVϵEP_{V_\epsilon}, where VϵV_\epsilon contains the top eigenvectors of E⊤Sϵ2EE^\top S_\epsilon^2 E. It converges to EPVEP_V as ϵ→0+\epsilon\to0^+ under an eigengap condition.
  • EoRA: with small eigenvalues truncated, the correction is PXEPVP_XEP_V, where PX=X+XP_X=X^+X projects onto the observed input subspace. It does not extend to inputs outside the calibration span.
  • QERA-approx: EPVDEP_{V_D}, where VDV_D contains the top eigenvectors of E⊤diag⁡(R)EE^\top\operatorname{diag}(R)E. It ignores correlations between input dimensions, so VDV_D generally differs from VV.
AuthorsOpenReview ↗

Common Response to Reviewers sXWh and zY67: Relation to QERA and EoRA (2/2)

Controlled comparison of calibration states and the handling of RR. We compared the four methods on ELQ/Jailbreak/Gemma-2-2B/INT8 using the same repair rank (r=1r=1), calibration budget (N=50N=50 prompts, T=16T=16 tokens), attacked checkpoint, and evaluation protocol. We chose rank 1 to make differences in subspace selection and reconstruction more apparent under a limited correction capacity. For QERA-exact, we set ϵ\epsilon to 1% of the average diagonal value of RR, adopting the damping-magnitude heuristic used in GPTQ (Frantar et al., 2022). We considered three calibration sources: teacher-forced states from the full-precision model (XFTFX_F^{TF}), teacher-forced states from the quantized model (XQTFX_Q^{TF}), and autoregressively generated states from the quantized model (XQgenX_Q^{gen}). The first corresponds to the standard practice of reconstruction methods, which calibrate on activations of the original model, while the last is ReQOR's default.

MethodXFTFX_F^{TF}XQTFX_Q^{TF}XQgenX_Q^{gen}
QERA-exact (Remark 1)6.671.330.00
QERA-approx44.6743.3342.67
EoRA20.0013.338.00
ReQOR6.670.670.00

ASR (%) over 150 attack prompts; lower is better. EoRA was evaluated using sequential eigenspace compensation with numerical-rank truncation.

Trajectory selection. QERA-exact, EoRA, and ReQOR all benefit from switching from full-precision teacher-forced states to quantized-model states, with a smaller additional gain from autoregressive generation: ASR decreases from 6.67% to 1.33% to 0.00% for QERA-exact, from 20.00% to 13.33% to 8.00% for EoRA, and from 6.67% to 0.67% to 0.00% for ReQOR. QERA-approx changes little (44.67% to 42.67%). These results indicate that calibration on states from the quantized model is an important factor in QCB mitigation, and that its benefit extends to existing activation-aware reconstruction methods.

Covariance handling. The treatment of RR matters. QERA-approx retains over 42% ASR on all three sources, suggesting that its diagonal approximation of RR fails to capture directions important for QCB mitigation. EoRA benefits from XQgenX_Q^{gen} but still leaves 8.00% ASR, which is consistent with its correction being restricted to the observed input subspace; since EoRA also compensates sequentially, we do not attribute the entire difference to truncation alone. QERA-exact matches ReQOR on the same calibration sources once the diagonal perturbation of Remark 1 is applied, as the analysis above predicts. ReQOR obtains this result without the perturbation or its parameter ϵ\epsilon, and QERA-exact reaches 0% ASR only with ReQOR's calibration source.

QCB-specific objective. QERA and EoRA aim to reconstruct quantization errors and recover the general utility of compressed models, whereas ReQOR is designed to suppress malicious behaviors activated by quantization while preserving benign utility. This objective motivates calibrating on states from the quantized model and a design that works with limited benign calibration data.

We have added a formal proposition establishing the mathematical relationship between ReQOR, QERA, and EoRA, along with a controlled comparison under matched calibration conditions, in Appendix B of the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.