- Rating
- 4
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces ADoRe, a training-free token pruning method specifically designed to accelerate Vision Transformer (ViT) encoders for dense Anomaly Detection (AD). The authors identify a critical flaw in applying standard classification-based pruning to AD: standard methods drop tokens with low global influence, which inadvertently discards small, subtle defects while preserving salient normal regions (causing false positives). The paper shifts the paradigm by proposing that AD should prune tokens whose normality is already settled. ADoRe computes a trimmed mean of tokens to serve as a normality reference, accumulates the deviation rank of each token across a specific band of intermediate layers, and prunes the most normal patches. These pruned locations are directly assigned an anomaly score of zero. Extensive experiments across industrial (MVTec-AD, VisA), medical, and video AD datasets using CLIP, DINOv2, and AI-VAD backbones show ADoRe achieves up to 2.47× encoder speedup without accuracy degradation, often improving it. Furthermore, reinvesting the saved latency into larger backbones or more crops yields net accuracy gains over unpruned baselines.
Critical strengths
- Insightful Formulation and Theoretical Grounding: The paper brilliantly identifies the task gap between image classification and dense anomaly detection regarding token pruning. The decomposition of pruning impact into and , supported by Proposition 1 and the analysis of "attention dilution" (Appendix A.2), elevates the paper from a simple efficiency hack to a rigorous study of representation dynamics in ViTs.
- Broad Applicability and Practical Utility: ADoRe is training-free and transfers seamlessly across different modalities (image/video), backbones (CLIP/DINOv2), and learning paradigms (zero-shot/few-shot). The experiment demonstrating "reinvesting the savings" (Fig. 4) is a highly practical and compelling way to prove the real-world value of the proposed acceleration.
Critical weaknesses
- Vulnerability to Large Defects due to Fixed Budget: As honestly acknowledged in Appendix D, the method relies on a fixed token keep-ratio to maintain hardware efficiency (rectangular tensors). Consequently, when an anomaly covers a large portion of the image (e.g., >20%), the fixed budget forces the premature pruning of actual anomalous tokens, leading to a severe drop in pixel-level AUROC (e.g., -8.79 on MVTec-AD for 40-100% area).
- Dependence on Background Homogeneity: The method relies on the "trimmed mean" to establish a normality reference. While this works well for standard industrial/medical AD where the background is dominant and relatively uniform, it may struggle in highly complex, cluttered, or multi-object scenes where the normal tokens themselves form a highly multi-modal distribution.
Questions for authors
- How does the trimmed mean reference perform when the normal background is highly heterogeneous or textured? Does forcing a single mean vector across diverse normal tokens risk misclassifying valid but rare normal textures as anomalies?
- Regarding the limitation with large defects (Appendix D): Have you experimented with a dynamic, threshold-based exit criterion during inference? Even though this would sacrifice batched speedup (requiring padding), it could serve as a "safe mode" for datasets where large defects are expected.
Rebuttal for Reviewer 5i3q
We thank the reviewer for the encouraging assessment, and for recognizing the insight of our formulation and theoretical analysis, as well as the broad applicability and practical value of ADoRe. We address each concern below.
Following the suggestion, we implemented this safe mode as a threshold-based dynamic exit, with its average FLOPs matched to the fixed budget (Table R1). It recovers accuracy on defects covering 20–40% of the image (pixel AUROC from 77.8 to 81.5), while the overall accuracy remains similar. As the reviewer anticipated, it requires padding for batched inference. We added it to Section 5.4 (Table 6, "Adaptive budget") as an option for datasets where large defects are expected.
Table R1. Fixed budget vs. safe mode, by defect area (change in pp in parentheses). AnomalyCLIP, MVTec-AD, 70%. GFLOPs are averaged over the images in each group.
ADoRe remains effective in multi-object scenes. We evaluated it on MVTec LOCO, where several kinds of normal objects appear in each image (Table R2). ADoRe stays above the unpruned model at both ratios and performs best among the pruning methods on most metrics. We added these results to Appendix C.13 (Table S16).
Table R2. AnomalyCLIP on MVTec LOCO (breakfast_box, pushpins, screw_bag, and splicing_connectors, the four multi-object categories). Each cell reports (P-AUROC, P-AUPRO) on structural anomalies.
It performs well when the normal background is heterogeneous, as the MVTec LOCO results above show (Table R2). And no, it does not risk misclassifying valid but rare normal textures as anomalies: the deviation from μˉ only decides which tokens to release, so such a token is simply kept and scored by the detector as usual (Appendix C.13).