DQ-PAD: Discriminative Quantized Prototype Alignment for Adversarial Patch Defense

Adversarial patches with diverse appearances challenge the generalization of defenses for object detection. Existing methods mainly localize adversarial patches by detecting statistical anomalies, learning discriminative patch features, or measuring reconstruction errors from generative models, but may struggle to distinguish diverse patches from benign regions. To address this issue, we propose Discriminative Quantized Prototype Alignment for Adversarial Patch Defense (DQ-PAD), which identifies adversarial patches by measuring how well image regions match benign visual patterns. DQ-PAD uses prototypes from a pretrained VQ-VAE2 codebook to represent benign visual patterns and computes an anomaly score based on the distance between each regional feature and its nearest prototype. However, since VQ-VAE2 is primarily trained for image reconstruction, some benign regions may match these prototypes poorly, while some patch regions may remain close to them. To address this limitation, we introduce patch-aware projection learning, which learns a projection matrix from labeled benign and patch regions while keeping the encoder and codebook frozen. The projection jointly maps features and prototypes into a low-dimensional space and selects more discriminative feature directions to reduce the matching distance of benign regions while enlarging the distance gap between patch and benign regions. In the pedestrian detection setting, we compare DQ-PAD with seven defense baselines using ten held-out adversarial patches and three object detectors, and demonstrate that DQ-PAD improves the average defense performance by up to 8.96% over existing SOTA methods. Physical-world experiments further demonstrate its applicability in real-world scenarios.