- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes ReQOR, a training-free defense against quantization-conditioned backdoors in LLMs. The method identifies module-level quantization discrepancies by evaluating full-precision and quantized operators on the same states from the quantized inference trajectory, and constructs a repair subspace from these discrepancies. The resulting correction is compiled into a factorized residual adapter that can be deployed with the quantized model without retaining the full-precision checkpoint. The paper evaluates ReQOR on four instruction-tuned LLMs, three QCB attacks, three malicious behaviors, and three quantization formats, showing strong attack mitigation while largely preserving benign utility. Additional analyses show that dominant discrepancy directions provide more effective low-rank repair than random or raw weight-error directions, and that rank-32 repair substantially reduces memory overhead compared with full-span repair
Critical strengths
1. The defense operates under realistic deployment constraints and achieves strong mitigation in the evaluated settings. ReQOR requires only benign calibration data and does not rely on attack labels, malicious samples, or attack-specific feedback. In Table 2, rank-32 ReQOR reduces Qwen2.5-3B ELQ jailbreak ASR from 82.00% to 0.00% and ACL content-injection ASR from 76.44% to 0.00% while maintaining similar benign utility. These results show that the defense can substantially reduce QCB behavior without the large utility degradation observed for some baselines.
2. The paper provides a clear analysis of why the proposed repair space is effective. The factorized residual formulation makes the correction explicit and avoids storing the full-precision model at deployment. The matched-rank comparisons in Table 3 show that dominant discrepancy directions are more effective than random, low-energy, or raw weight-error directions in the evaluated cases, supporting the role of the proposed repair subspace. The rank-memory analysis further shows the trade-off between full-span and low-rank repair.
Critical weaknesses
1. The distinction from activation-aware error reconstruction methods is not sufficiently clear. ReQOR builds its correction from an activation-dependent operator discrepancy subspace, which shares similarities with prior activation-aware quantization error reconstruction methods such as QERA. The paper discusses differences in terms of trajectory selection and the QCB setting, but it does not clearly show which part of the proposed formulation is responsible for an advantage over these methods. The paper mentions QERA and EoRA but does not include them in the empirical comparison. Improvements over random directions, weight-space SVD, and L2 QER-style reconstruction do not by themselves establish an advantage over the closest activation-aware baselines. A direct comparison with QERA or similar methods would help clarify the contribution of ReQOR.
2. The paper does not clearly show the benefit of using quantized-trajectory calibration. The same-state operator discrepancy removes upstream state drift by construction, but the experiments do not directly test whether collecting states from the quantized inference trajectory is important for the observed gains. The current ablations mainly compare different repair directions and reconstruction choices, so it remains unclear how much of the improvement comes from the trajectory-based formulation itself rather than from using activation-dependent corrections in general. In addition, a large discrepancy on benign calibration data does not necessarily imply that the corresponding directions are relevant to QCB suppression. Although ReQOR works on the evaluated attacks, the paper does not examine whether these directions remain effective across different calibration sets, domains, or adaptive attack scenarios. The calibration analysis provides some evidence, but it is limited to full-span repair on a single model and attack setting.
Questions for authors
1. How does ReQOR differ from activation-aware reconstruction methods such as QERA? When using the same calibration states and repair rank, what is the main difference between ReQOR and QERA-style corrections? In particular, how does the method differ in the bias-free linear case, and which part of the design (e.g., trajectory selection, covariance handling, or the QCB setting) is responsible for the improvement?
2. How important is the use of quantized inference trajectories? With the same correction rule and calibration budget, how does ReQOR perform when the repair subspace is built from full-precision trajectories or simpler prompt-based calibration states?
3. How stable is low-rank ReQOR across different calibration settings? Does rank-32 repair maintain similar performance with different benign calibration samples or domains, especially for cases where some attack success remains? How much of this robustness is supported by the current experiments, and how much remains to be tested under adaptive attacks?
4 comments
Response to Reviewer zY67 (2/3)
Relevance of Benign Discrepancy Directions ([W2])
We agree that a large discrepancy on benign data does not by itself imply relevance to QCB suppression. We therefore tested this through a decomposition of the full-precision–quantized weight difference, using the intermediate malicious checkpoint of each attack. We found that the directions ReQOR identifies from benign data capture the components that remove the backdoor.
QCB attack mechanism. The evaluated PGD-based QCB attacks involve two stages. First, malicious behavior is injected into a full-precision (FP) model. Second, PGD suppresses this behavior in the FP model while preserving its activation after quantization. Let denote the malicious FP model after Stage 1, the FP model after Stage 2, and the common quantized model. The discrepancy seen by ReQOR therefore decomposes as
where is the Stage-2 update that makes the full-precision model benign. is unavailable to the defender and is used only for this analysis. ReQOR's rank- projector is computed exactly as in the paper, from benign calibration data and without .
We investigated this relationship through two complementary analyses, over nine configurations (ACL/Content Injection/Qwen2.5-3B, ELQ/Jailbreak/Gemma-2-2B, Q-Misalign/Over-refusal/Llama-3.2-3B × INT8, NF4, FP4).
First, we measured how much of the operator-level energy of the Stage-2 update lies in ReQOR's rank- subspace, on the same benign quantized-model states used for calibration. ReQOR's subspace captures 35.15% at rank 1 and 48.63% at rank 32, compared with only 0.03% and 1.02% for random subspaces of the same rank.
Second, we tested whether this captured component is sufficient to suppress the malicious behavior. We added to the Stage-1 model only the part of the Stage-2 update that lies in the subspace and discarded the rest:
We then measured ASR and compared the results with matched-rank random projections.
| Model variant | Average ASR (%) ↓ |
|---|---|
| Stage-1 malicious FP | 53.26 |
| Random-projected update (Rank 1) | 51.93 |
| Random-projected update (Rank 32) | 52.96 |
| ReQOR-projected update (Rank 1) | 7.11 |
| ReQOR-projected update (Rank 32) | 0.07 |
| Stage-2 FP | 0.00 |
Average over nine QCB configurations. Random projections use seed 0.
With ReQOR's rank-32 subspace, ASR is 0% in eight of the nine configurations and 0.67% in the remaining one. In contrast, random subspaces leave the malicious behavior of the Stage-1 model almost unchanged.
These results indicate that ReQOR's benign-calibrated directions are not merely large quantization errors: they align with the weight changes that suppress the backdoor, and the component they capture is sufficient to suppress the malicious behavior in all evaluated attacks.
Response to Reviewer zY67 (3/3)
Calibration Analysis and Adaptive Attacks ([W2][Q3])
To examine whether ReQOR remains effective across different calibration sets and domains, we extended the calibration analysis to a wider range of conditions: both rank-32 and full-span repair, four calibration domains (Dolly, SODA, GSM8K, MBPP), three additional disjoint Dolly subsets, and five settings including two where residual ASR remains. All runs use the same budget (, ).
| Setting | Repair | Dolly (3 subsets) | SODA | GSM8K | MBPP |
|---|---|---|---|---|---|
| Three representative settings‡ | Rank-32 / Full | 0.00 ± 0.00 | 0.00 | 0.00 | 0.00 |
| ELQ / Injection / Qwen2.5-1.5B | Rank-32 | 1.11 ± 0.44 | 1.78 | 2.00 | 5.33 |
| Full | 0.30 ± 0.13 | 0.22 | 1.33 | 0.22 | |
| Q-Misalign / Jailbreak / Gemma-2-2B | Rank-32 | 5.56 ± 0.22 | 4.67 | 10.22 | 10.89 |
| Full | 1.70 ± 0.34 | 1.34 | 1.56 | 1.33 |
ASR (%), averaged over INT8, NF4, and FP4. Dolly reports the mean ± sample standard deviation across three disjoint Dolly calibration subsets. ‡ ACL/Injection/Qwen2.5-3B, ELQ/Jailbreak/Gemma-2-2B, and Q-Misalign/Over-refusal/Llama-3.2-3B.
In the three representative settings, both rank-32 and full-span repair reach 0% ASR under every calibration set. In the two settings where residual ASR remains, full-span repair is stable (0.22–1.33% and 1.33–1.70%), whereas rank-32 repair varies more (1.11–5.33% and 4.67–10.89%). Rank-32 repair therefore maintains its performance where it already suppresses the attack; in the cases where some attack success remains, it varies across calibration sets, but even its worst case (10.89%, for Q-Misalign/Jailbreak/Gemma-2-2B) is far below the undefended ASR in that setting (84.22%).
We further evaluated the outlier-injection QCB suggested by Reviewer pFKM [Q2], which differs from the PGD-based attacks in our original evaluation. Across eight attack-activated settings, ReQOR reduces average ASR from 68.17% to 6.92% (rank-32) and 0.33% (full-span), providing additional evidence of robustness across different QCB constructions.
However, none of these attacks is optimized against ReQOR, so our results do not establish robustness to adaptive attacks. The PGD-based attacks constrain only the weights, keeping them within the intervals that preserve the quantized values. To evade ReQOR, an adaptive attacker would additionally need to be activation-aware: the weight changes that make the full-precision model benign would have to stay out of the discrepancy subspace observed on benign activations. Whether this is achievable while preserving the quantized behavior is an open question. In the current attacks, these changes are clearly exposed (see our response on [W2], relevance of benign discrepancy directions), and calibration on unrelated domains such as math or code still suppresses the jailbreak attack.
We have acknowledged this limitation in Section 6 (Conclusion) of the revised manuscript and identified adaptive attacks specifically designed to evade ReQOR as a direction for future work.
Follow-up on Rebuttal and Revisions for Reviewer zY67
Dear Reviewer zY67,
We would like to briefly follow up on our rebuttal and revised manuscript. We have addressed your concerns through several additional analyses, as detailed in our responses:
- QERA/EoRA comparison ([W1][Q1]): Mathematical analysis and controlled comparisons under matched conditions (Common Response to Reviewers sXWh and zY67).
- Quantized-trajectory calibration ([W2][Q2]): Controlled comparison of six calibration-state sources (Response 1/3).
- Relevance of benign discrepancy directions ([W2]): Analysis showing that the identified directions capture components responsible for suppressing malicious behavior (Response 2/3).
- Calibration robustness and adaptive attacks ([W2][Q3]): Additional evaluations across calibration domains and subsets, including challenging settings, and discussion of adaptive-attack limitations (Response 3/3).
In addition, following Reviewer pFKM's suggestion, we evaluated a non-PGD outlier-injection attack to assess generalization beyond the original PGD-based attacks. Under this attack, existing baselines largely fail, whereas ReQOR substantially outperforms them, reducing average ASR from 68.17% to 6.92% (rank 32) and 0.33% (full-span) while preserving benign utility. These results are presented in Table 3 (Section 5.2) and Appendix F, with further discussion in our response to Reviewer pFKM ([Q2]).
We have also incorporated the relevant findings into the revised manuscript.
We would greatly appreciate hearing whether our responses have addressed your main concerns or if any questions remain. With the discussion period nearing its end, we would be happy to provide further clarification as promptly as possible.
Thank you again for your thoughtful feedback.
Response to Reviewer zY67 (1/3)
We sincerely thank the reviewer for their careful assessment of our work and constructive suggestions for clarifying our contributions and strengthening our experimental evaluation.
Relation to Activation-Aware Reconstruction (QERA/EoRA) ([W1][Q1])
Please refer to our common response, “Relation to Activation-Aware Reconstruction (QERA/EoRA),” where we provide the mathematical relationship between ReQOR and QERA/EoRA, along with controlled comparisons under matched calibration conditions.
Importance of Quantized-Trajectory Calibration ([W2][Q2])
We agree that our original ablations kept the state-collection procedure fixed. We therefore compared six state sources across nine settings with the same correction rule, calibration budget, and rank (r=1): autoregressive generation, teacher forcing, and prompt-only states, each collected with the quantized or the full-precision model.
ASR (%) across nine settings; details are in our response to Reviewer sXWh [W1][Q4].
The important factor is collecting states with the quantized model rather than the full-precision model. With generation, full-precision states leave residual ASR in five of the nine settings (up to 12.67%), whereas quantized states reach 0% in all nine. Among quantized-model states, the collection method made little difference (0.00–0.07% average ASR), so autoregressive generation is our default way of collecting such states, not a requirement.
Regarding whether the gain comes from the trajectory or from activation-dependent correction in general, our common response on QERA/EoRA addresses this directly. These methods are activation-dependent corrections, yet with their original full-precision calibration they leave residual ASR, and they also improve when calibrated on quantized-model states. Activation dependence alone is thus not sufficient, and the benefit of quantized-model states is not specific to ReQOR.
We have added this ablation to Appendix L.1 and clarified in Section 5.6 that the key requirement is the use of quantized-model states.