Unveiling Adversarial Patches: Prototype-Guided Detection and Localization

Adversarial patch attacks imperil modern vision systems because they are localized, physically realizable, and highly transferable. Existing defenses typically target either global-level detection (clean vs. patch) or patch localization, but most operate as black boxes and provide limited interpretability. We propose PatchTrace, a prototype-based adversarial patch defense that unifies global attack detection and patch localization within a single interpretable framework. PatchTrace learns to separate clean and anomaly prototype banks, detecting attacks by explicitly competing between normal and adversarial evidence. This dual-bank design enables highly accurate global-level detection of unseen patches and, via a novel Prototype Residual Attention Map (PRAM), it yields faithful, dense localization of adversarial regions. Unlike prior prototype-based defenses that remain restricted to global explanations, PatchTrace extends interpretability to the pixel level, supporting a coherent global-to-local defense pipeline. Extensive experiments demonstrate that PatchTrace achieves near-perfect detection on unseen patches while delivering stronger, interpretable localization than existing defenses, even in noisy settings. We assert that the proposed PatchTrace is the first prototype-driven, unified end-to-end framework that simultaneously achieves these four properties (detection, localization, interpretability, and robustness), and it can pave the way for a unified development of adversarial patch defense approaches in the future.