- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates quantization-aware backdoor attacks against LVLMs, highlighting that dormant backdoors in full-precision models may be activated during quantization. The authors propose GhostQuant, a framework that addresses the challenges posed by the fragility of visual features under low-bit quantization through a boundary-aware parameter selection strategy, partial pre-quantization, and minimal perturbation injection. Experiments across multiple LVLMs demonstrate the effectiveness of the proposed method. Although the technical details are generally sound and the experimental evaluation is relatively comprehensive, the paper has notable limitations in terms of conceptual novelty, and comparisons with robustness baselines.
Critical strengths
-
The paper is well structured and logically organized. The presentation is coherent, with smooth transitions between sections, making the core ideas and technical framework easy to follow.
-
The empirical evaluation is relatively comprehensive, covering multiple LVLMs. The results demonstrate high attack success rates (ASRs) while largely preserving benign model performance.
-
The authors provide a relatively candid discussion of the method's limitations and potential advantages, reflecting a rigorous approach to scientific evaluation.
Critical weaknesses
-
The paper does not provide a substantive solution to the vulnerability of quantization-induced backdoors to parameter updates. Given the growing attention to quantization-related security risks, defenders may be able to identify such vulnerabilities and weaken or eliminate the backdoor through standard fine-tuning or noise injection. This limitation raises concerns about the practical effectiveness of the proposed attack in realistic adversarial settings.
-
The robustness evaluation lacks direct comparisons with existing relevant baselines. Consequently, it remains unclear to what extent GhostQuant outperforms conventional or related baseline methods, making it difficult to assess its relative advantages.
-
In terms of overall contribution, the paper primarily extends existing research on quantization-related backdoor attacks to the LVLM setting. It does not introduce a substantial conceptual breakthrough or a fundamentally novel methodology, and it shares limitations with related prior work.
-
The paper has some structural shortcomings. It lacks a dedicated "2. Related Work" section, with discussions of prior work instead distributed across Sections 1.1 and 1.2 of the Introduction.
Questions for authors
See weaknesses.
2 comments
I appreciate the authors' response and the additional experimental evaluations. These efforts have improved the structure of the related work section and further validated GhostQuant’s robustness advantages over the relevant baselines. However, the proposed method remains notably vulnerable to standard parameter-update-based defenses, such as benign fine-tuning. Given these inherent limitations regarding parameter-update defenses, as well as the incremental nature of the conceptual innovation, I have decided to maintain my original score.
We thank the reviewer for the constructive feedback and positive assessment of our presentation and evaluation. Below is our clarification:
W3: A Novel Method and Its Advantages
Existing methods rely on two-stage fine-tuning, which accumulates substantial weight modifications. They first implant a backdoor, then restore benign full-precision behavior under quantization constraints. These two stages substantially change model weights and can incur a fine-tuning tax. Model performance can drop noticeably, making it easier for users to notice problems during normal use. Dynamic quantizers are sensitive to weight changes, making this approach difficult to extend. ELQ, Q-Misalign, and ACL support only INT8, FP4, and NF4. Widely used quantizers such as GGUF and AWQ remain unsupported. This limited quantizer coverage restricts their practical threat.
GhostQuant introduces a different method based on small, controlled perturbations to the original weights. These perturbations make only minimal changes to the model. Beyond adapting the attack to LVLMs, our method achieves better utility preservation and broader quantizer coverage. On Qwen3VL-2B for caption hijacking, its average relative utility drop is only 1.55%, compared with 12.31%–14.01% for the baselines. A single GhostQuant checkpoint supports static quantizers and dynamic quantizers such as GGUF and AWQ. This substantially broadens its practical threat.
W2: Defense Robustness
We provide direct robustness comparisons with ELQ, Q-Misalign, and ACL under Gaussian noise injection, benign fine-tuning and Wanda pruning. All methods were retrained on Qwen3VL-2B for caption hijacking and evaluated under matched defense settings. We report NF4 results below. Complete results across NF4, FP4 and INT8 are added in Section 5.6.
Gaussian noise injection
Benign fine-tuning
Wanda pruning
GhostQuant retains higher ASR under moderate noise. At a noise variance of 10−5, it achieves 79.5% ASR, compared with 41.5%–52.5% for the baselines. All attacks become ineffective at 10−4. Q-Misalign shows greater persistence under benign fine-tuning. Its contrastive task vector (CTV) strategy compares parameter changes from benign and harmful fine-tuning. This helps select parameters that support harmful behavior and are less affected by benign fine-tuning.
GhostQuant is substantially more robust to Wanda pruning. At 20% pruning, it retains 95.5% ASR, compared with 0.5%–1.5% for the baselines. Our selection threshold r∣θi∣ grows with weight magnitude. Larger weights are therefore more likely to be selected when boundary distances are comparable. Wanda purning also favors larger weights when activation norms are comparable. This shared preference helps GhostQuant remain more robust to Wanda pruning.
W1: Robustness Improvements
GhostQuant improves pruning robustness over prior methods (W2). Although GhostQuant follows a different methodological approach from ELQ-style two-stage fine-tuning, we demonstrate that Q-Misalign’s CTV strategy can be incorporated into GhostQuant to improve fine-tuning persistence.
We evaluate GhostQuant with CTV on Qwen3VL-2B for caption hijacking. CTV uses MIMIC-IT to select 40% of the parameters. We set r = 0.15 for NF4 and FP4, and r = 0.01 for INT8. Both variants are fine-tuned on ScienceQA for 500 steps. The learning rate is 10⁻⁶, and the batch size is 8. The table reports NF4 ASR (%). Complete results for all three quantizers are now added in Appendix C.10.
Incorporating CTV substantially improves backdoor persistence under fine-tuning. After 60 steps, GhostQuant + CTV achieves 59.0% ASR, compared with 12.0% for the original GhostQuant. After 100 steps, it retains 33.5% ASR, compared with 0.5%.
W4: Related Work Organization
Sections 1.1–1.3 were intended to form a separate Related Work section. The
\section{Related Work}command was accidentally deleted during formatting. We apologize for this oversight. We have added a Section 2, Related Work, to the revised manuscript.