GhostQuant: The Ghost that Appears Only After Quantization

Model quantization is widely used to deploy deep learning models on resource-constrained devices. However, it may introduce previously overlooked security risks. Attackers can implant backdoors that are inactive in full-precision model but get activated after quantization. In Large Vision-Language Models (LVLMs), building such attacks is challenging. Visual backdoors are easily erased by low-bit quantization. Moreover, most existing methods support only static quantizers, limiting their real-world threat. They also noticeably degrade the model's full-precision utility, making the attack easier to detect. To solve these, we propose GhostQuant, a stealthy quantization-aware backdoor for LVLMs. GhostQuant optimizes boundary-sensitive parameters under quantized constraints. It then applies minimal perturbations to activate the backdoor after quantization while preserving full-precision behavior. Experiments show that GhostQuant outperforms existing methods across diverse LVLMs. For example, on Qwen3VL-2B with Caption Hijacking, GhostQuant achieves an average ASR of 92.5% across 12 quantization methods while limiting the full-precision utility deviation to only 1.55%. In contrast, existing baselines support only three static quantizers and cause utility deviations of 12.31%–14.01%.