- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces PatchTrace, a supervised framework for joint adversarial-patch detection and pixel-level localization. The method uses a ConvNeXt-Tiny backbone and two prototype banks representing clean and adversarial evidence. Global responses from the two banks are fused using Cross-Prototype Attention Fusion (CPAF) to determine whether an image contains an adversarial patch. For localization, the proposed Prototype Residual Attention Map (PRAM) compares anomaly and clean prototype responses spatially, and the resulting signal is decoded into a patch mask. The detector also gates localization so that a mask is produced only when an attack is detected.
The model is trained with image-level clean/attacked labels, ground-truth patch masks, prototype alignment and separation objectives, and dense localization losses. Evaluation includes cross-patch transfer from ImageNet-Patch to unseen patches on COCO, cross-dataset evaluation on SmuDPatch, GAP, noisy corruptions, and several physical-world datasets. PatchTrace reports strong detection and localization on ImageNet-Patch and SmuDPatch, although performance decreases substantially for broader attack distributions and some physical-world settings.
Critical strengths
- Unified and interpretable global-to-local formulation.
A major strength is that detection and localization are not implemented as independent modules with unrelated evidence. The same clean and anomaly prototype responses drive both the global attack decision and the PRAM localization signal. This makes the explanation more directly tied to the model decision than a purely post-hoc saliency mechanism and provides a coherent motivation for the architecture.
- Strong cross-patch localization and detection under the primary benchmarks.
The reported results on ImageNet-Patch and SmuDPatch are strong. PatchTrace reaches approximately 99.8% detection accuracy and 84.0% localization mIoU on the primary silent ImageNet-Patch evaluation, while also obtaining 99.1% detection accuracy and 70.6% mIoU on SmuDPatch without adaptation. The improvements over NAPGuard, DIFFender, and SAM-based baselines are substantial, particularly for localization.
- Good experimental analysis and computational efficiency.
The paper contains useful loss ablations, prototype analyses, noisy evaluations, OOD patch experiments, and computational comparisons. The ablations demonstrate that different parts of the objective affect localization and detection differently. The reported inference cost is also substantially lower than diffusion-based defenses such as DIFFender while maintaining much stronger localization performance in the reported benchmarks.
Critical weaknesses
- The robustness evaluation does not test an adaptive attacker or controlled cross-victim-model generalization.
The central claim is that the learned anomaly prototypes capture transferable adversarial-patch evidence rather than memorizing individual patch patterns. The paper convincingly evaluates transfer across patch patterns and, to some extent, attack-generation methods. However, I could not find an experiment where an attacker explicitly optimizes a patch to evade PatchTrace, for example by simultaneously reducing anomaly-prototype activation or increasing clean-prototype activation. I also could not find a controlled experiment where patches are generated against different victim architectures and then evaluated with the same PatchTrace model. Cross-patch transfer is not equivalent to cross-victim-model transfer. This is important because the learned prototype bank may partially capture artifacts induced by the attack generators or victim models represented in the benchmark rather than a model-independent notion of adversarial "patchness."
- Generalization deteriorates substantially in the broader and physical-world settings.
The near-perfect results on the primary benchmark do not carry over consistently to more diverse attacks. On GAP, which contains 25 patch types from eight attack methods, detection accuracy falls to 59.8%. On APRICOT, physical-world recall is only 23.7%. Furthermore, the physical-world subsets contain only attacked images, so these experiments cannot measure false positives on clean physical imagery. These results are especially important because the motivation of the paper emphasizes physical and safety-critical adversarial patches. The gap between the primary benchmark results and these broader settings weakens the claim of robust, general adversarial-patch detection.
- Localization relies heavily on explicit dense patch supervision, which weakens the interpretation that localization naturally emerges from prototype reasoning.
PatchTrace is fully supervised with ground-truth patch masks, and the ablation results show that removing the localization loss reduces mIoU from 90.4% to approximately 0.9%, while detection remains almost unaffected. Thus, the prototype-based detector alone does not appear to provide useful dense localization. The strong final localization performance is therefore heavily dependent on conventional pixel-level mask supervision and a learned decoder. This does not invalidate the method, but it makes the claim that prototype evidence itself provides faithful dense localization less convincing. It also raises practical questions about how the method would scale to attack families for which accurate patch masks are unavailable.
Questions for authors
-
Clarification / material experiment: What victim model or models were used to optimize the adversarial patches in the ImageNet-Patch and SmuDPatch benchmarks? Are all patched samples guaranteed to successfully attack the corresponding victim model? Please clarify the attack-generation and filtering protocol, since this affects what the anomaly bank is actually learning.
-
Material experiment: Can the authors evaluate cross-victim-model generalization? For example, train PatchTrace using patches optimized against one victim architecture and evaluate patches optimized against substantially different architectures such as CNN and ViT families. This would help determine whether the learned prototypes capture generic adversarial-patch properties or model-specific attack artifacts.
-
Material experiment: How does PatchTrace perform against an adaptive attacker that knows the defense? In particular, an attacker could optimize its original attack objective jointly with a term that suppresses anomaly-prototype activation, increases clean-prototype activation, or directly minimizes the PatchTrace detection score. Since PatchTrace is proposed as a defense, this evaluation could materially change my assessment of its robustness.
-
Clarification: The GAP detection accuracy is 59.8% and APRICOT recall is 23.7%, which are substantially below the main benchmark results. What characteristics of these attacks cause the failure? Is the degradation associated with naturalistic appearance, attack-generation method, victim architecture, patch scale, physical transformations, or another factor? A stratified analysis would help clarify the scope of the method's generalization.
10 comments
Overall, the analyses identify distinct challenges: physical patch contrast and scale on APRICOT, and low-saturation, naturalistic appearance on GAP.
Other/New Concerns and Discussion
We are actively working to resolve other concerns [if pending] and look forward to a positive assessment of our responses; we will be happy to address any new comments.
$\color{green}{\text{computational efficiency of the proposed approach, as requested by R-jTVY.}} $
Thank you for your detailed rebuttal and the additional experiments. I sincerely appreciate the effort invested in addressing my concerns and providing further clarification.
The new adaptive attack experiments and stratified analyses are valuable additions that help clarify the strengths and limitations of the proposed approach. I will examine these results more carefully before reaching my final assessment.
Nevertheless, several important concerns remain:
-
Novelty and methodological contribution: The approach primarily combines established components, including prototype-based representations, attention mechanisms, and supervised localization. I remain uncertain whether their integration constitutes a sufficiently substantial methodological advance over existing approaches in the literature.
-
Generalization and robustness: The substantial performance degradation on GAP and APRICOT remains concerning, particularly given the paper's emphasis on transferable adversarial-patch detection. The additional analyses help explain these failures but do not fully address the underlying generalization limitations.
-
Adaptive attacks: While I appreciate the newly introduced white-box experiments, the considerable reduction in detection recall under stronger adaptive attacks raises further questions about robustness against defense-aware adversaries.
-
Dependence on dense supervision: The dramatic localization performance drop when removing the localization loss continues to raise questions about how much localization capability originates from prototype-based reasoning rather than conventional pixel-level supervision.
Overall, I appreciate the authors' responsiveness and the additional evidence. However, I believe these remaining issues are important for evaluating the paper's overall novelty, robustness, and significance relative to existing literature. I will carefully reconsider the rebuttal and the extent to which these concerns have been addressed before finalizing my assessment.
Responses and Thanks for the comments which further demonstrate the strength of the proposed work
1
Attack-generation protocol: ImageNet-Patch (Pattern Recognition'2023) uses adversarial patches generated against AlexNet, ResNet-18, and SqueezeNet, while SmuDPatch (ECCV'2026) uses ResNet-50, ViT, and MobileNetV2. Both are established, published benchmarks, and PatchTrace uses their existing adversarial patch samples without generating new attacks. Importantly, PatchTrace learns from the visual characteristics of adversarial patch regions rather than requiring every patch to fool its original victim model successfully.
2
Our evaluation already includes patches generated using diverse victim architectures: AlexNet, ResNet-18, and SqueezeNet (ImageNet-Patch), and ResNet-50, MobileNetV2, and ViT-B/16 (SmuDPatch). PatchTrace achieves strong cross-benchmark generalization without retraining, indicating that its learned representations transfer across different attack-generation settings. However, these benchmarks also differ in their patch-generation procedures, so this evaluation does not fully isolate the effect of victim architecture.
3. White-Box Adaptive Attack Evaluation
We thank the reviewer for this valuable suggestion. We conducted additional white-box adaptive attack experiments to evaluate PatchTrace against defense-aware adversaries.
Experimental setup: The attacker has full gradient access to PatchTrace and the victim ResNet-50. We optimize patch pixels using 200-step PGD, combining the targeted attack objective with three defense-aware objectives: (i) minimizing the detection score, (ii) suppressing anomaly and increasing clean prototype responses, and (iii) suppressing PRAM activation. We evaluate nine unseen COCO patches.
Table 3: White-box adaptive attack evaluation (%). Joint ASR represents attacks that both fool the victim and evade PatchTrace. The weight w controls the contribution of defense-evasion objectives.
PatchTrace maintains 95.2% detection recall against PRAM-targeted attacks and 89.2% detection recall when jointly optimized with w=0.1. Increasing the evasion weight to w=1 and w=10 produces joint attack success rates of 21.7% and 36.2%, respectively, illustrating the trade-off between attack strength and detection. Under EOT (noise and spatial shifts), PatchTrace retains 83.3% detection recall, with joint attack success decreasing to 2.9%. These results provide a more comprehensive assessment of PatchTrace under a strong white-box threat model.
4. Stratified Analysis
We conducted additional stratified analyses of APRICOT and GAP to identify the factors affecting generalization.
APRICOT: We analyzed all 873 physical adversarial images, examining patch contrast, scale, shape, victim architecture, target class, and imaging conditions.
Table 4: APRICOT detection recall across attack characteristics (%).
Patch contrast and scale show the strongest associations with detection performance (AUROC 0.69/0.63). Higher-contrast and larger patches are detected more reliably, while image sharpness and perspective warping have limited influence. Patch shape, victim architecture, and target class also contribute to performance variation
Controlled experiments further show that Gaussian blur (σ=4) reduces digital-patch detection from 100% to 0%, highlighting sensitivity to changes in patch appearance. On APRICOT, adjusting the operating threshold improves recall from 23.7% to 44.6% at 5% FPR, and to 59.5% with 2× multi-crop inferencewithout retraining
GAP: We additionally analyzed 1,942 annotated GAP images across patch appearance, scale, contrast, and sharpness. Unlike APRICOT, colour saturation is the strongest associated factor (AUROC 0.79): detection recall rises from 1.6% for low-saturation patches to 49.1% for high-saturation patches. Patch scale has a secondary, non-monotonic association, while sharpness shows limited influence. This suggests that muted, naturalistic patch appearances contribute to the generalization gap in the GAP