acceptodds
Under review as a conference paper at ICLR 2027

ADoRe: Normality-Guided Token Pruning for Efficient Anomaly Detection

Abstract

Recent anomaly detectors reach strong accuracy, but their deployment is bound by throughput. A faster decision shortens inspection cycle time and raises the number of objects a surveillance system handles in real time. Defects are small, however, so inputs must stay at high resolution, and the Vision Transformer (ViT) encoder accounts for most of the inference time. Token pruning is a natural remedy, yet existing methods inherit from image classification the intuition that a token which cannot change a single global prediction is safe to discard. This intuition fails in anomaly detection, which labels every location rather than the whole image: faint defects are discarded while salient normal regions are kept. What can be released is a token whose normality is settled. Under ground-truth labels, releasing such tokens carries none of the accuracy cost that pruning usually incurs and can reverse it, for two reasons: anomalous patches would keep their scores while released ones can only lose theirs, and the survivors, encoded with fewer tokens, become easier to separate. Based on this, we propose ADoRe, a training-free module that scores each token against a normality reference trimmed to resist the anomalies it must separate, and accumulates its rank over a band of intermediate layers. The most normal patches are released and enter the anomaly map as zero without interpolation. Across CLIP, DINOv2, and video detectors, ADoRe accelerates the encoder by up to 2.47 while matching or exceeding the unpruned baseline in most settings, and at a fixed time budget, spending the savings on a larger encoder or more crops surpasses the unpruned pipeline.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Anomaly Detection, Token Pruning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_5i3qOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces ADoRe, a training-free token pruning method specifically designed to accelerate Vision Transformer (ViT) encoders for dense Anomaly Detection (AD). The authors identify a critical flaw in applying standard classification-based pruning to AD: standard methods drop tokens with low global influence, which inadvertently discards small, subtle defects while preserving salient normal regions (causing false positives). The paper shifts the paradigm by proposing that AD should prune tokens whose normality is already settled. ADoRe computes a trimmed mean of tokens to serve as a normality reference, accumulates the deviation rank of each token across a specific band of intermediate layers, and prunes the most normal patches. These pruned locations are directly assigned an anomaly score of zero. Extensive experiments across industrial (MVTec-AD, VisA), medical, and video AD datasets using CLIP, DINOv2, and AI-VAD backbones show ADoRe achieves up to 2.47× encoder speedup without accuracy degradation, often improving it. Furthermore, reinvesting the saved latency into larger backbones or more crops yields net accuracy gains over unpruned baselines.

Critical strengths
  1. Insightful Formulation and Theoretical Grounding: The paper brilliantly identifies the task gap between image classification and dense anomaly detection regarding token pruning. The decomposition of pruning impact into Δscoring\Delta_{scoring} and Δencoding\Delta_{encoding}, supported by Proposition 1 and the analysis of "attention dilution" (Appendix A.2), elevates the paper from a simple efficiency hack to a rigorous study of representation dynamics in ViTs.
  2. Broad Applicability and Practical Utility: ADoRe is training-free and transfers seamlessly across different modalities (image/video), backbones (CLIP/DINOv2), and learning paradigms (zero-shot/few-shot). The experiment demonstrating "reinvesting the savings" (Fig. 4) is a highly practical and compelling way to prove the real-world value of the proposed acceleration.
Critical weaknesses
  1. Vulnerability to Large Defects due to Fixed Budget: As honestly acknowledged in Appendix D, the method relies on a fixed token keep-ratio to maintain hardware efficiency (rectangular tensors). Consequently, when an anomaly covers a large portion of the image (e.g., >20%), the fixed budget forces the premature pruning of actual anomalous tokens, leading to a severe drop in pixel-level AUROC (e.g., -8.79 on MVTec-AD for 40-100% area).
  2. Dependence on Background Homogeneity: The method relies on the "trimmed mean" to establish a normality reference. While this works well for standard industrial/medical AD where the background is dominant and relatively uniform, it may struggle in highly complex, cluttered, or multi-object scenes where the normal tokens themselves form a highly multi-modal distribution.
Questions for authors
  1. How does the trimmed mean reference μˉ\bar{\mu} perform when the normal background is highly heterogeneous or textured? Does forcing a single mean vector across diverse normal tokens risk misclassifying valid but rare normal textures as anomalies?
  2. Regarding the limitation with large defects (Appendix D): Have you experimented with a dynamic, threshold-based exit criterion during inference? Even though this would sacrifice batched speedup (requiring padding), it could serve as a "safe mode" for datasets where large defects are expected.
1 comment
AuthorsOpenReview ↗

Rebuttal for Reviewer 5i3q

We thank the reviewer for the encouraging assessment, and for recognizing the insight of our formulation and theoretical analysis, as well as the broad applicability and practical value of ADoRe. We address each concern below.

[W1] "Vulnerability to Large Defects due to Fixed Budget: As honestly acknowledged in Appendix D, the method relies on a fixed token keep-ratio to maintain hardware efficiency (rectangular tensors). Consequently, when an anomaly covers a large portion of the image (e.g., >20%), the fixed budget forces the premature pruning of actual anomalous tokens, leading to a severe drop in pixel-level AUROC (e.g., -8.79 on MVTec-AD for 40-100% area)."

[Q2] "Regarding the limitation with large defects (Appendix D): Have you experimented with a dynamic, threshold-based exit criterion during inference? Even though this would sacrifice batched speedup (requiring padding), it could serve as a "safe mode" for datasets where large defects are expected."

Following the suggestion, we implemented this safe mode as a threshold-based dynamic exit, with its average FLOPs matched to the fixed budget (Table R1). It recovers accuracy on defects covering 20–40% of the image (pixel AUROC from 77.8 to 81.5), while the overall accuracy remains similar. As the reviewer anticipated, it requires padding for batched inference. We added it to Section 5.4 (Table 6, "Adaptive budget") as an option for datasets where large defects are expected.

Table R1. Fixed budget vs. safe mode, by defect area (change in pp in parentheses). AnomalyCLIP, MVTec-AD, 70%. GFLOPs are averaged over the images in each group.

Defect area# of imagesP-AUROC (fixed)P-AUROC (safe mode)P-AUPRO (fixed)P-AUPRO (safe mode)GFLOPs (fixed)GFLOPs (safe mode)
≤20%1,20092.292.4 (+0.2)83.283.0 (−0.2)638638
20–40%1877.881.5 (+3.7)59.765.2 (+5.5)638665
>40%4081.081.0 (0.0)53.550.2 (−3.3)638691
All1,25888.688.7 (+0.1)81.681.3 (−0.3)638637

[W2] "Dependence on Background Homogeneity: The method relies on the "trimmed mean" to establish a normality reference. While this works well for standard industrial/medical AD where the background is dominant and relatively uniform, it may struggle in highly complex, cluttered, or multi-object scenes where the normal tokens themselves form a highly multi-modal distribution."

ADoRe remains effective in multi-object scenes. We evaluated it on MVTec LOCO, where several kinds of normal objects appear in each image (Table R2). ADoRe stays above the unpruned model at both ratios and performs best among the pruning methods on most metrics. We added these results to Appendix C.13 (Table S16).

Table R2. AnomalyCLIP on MVTec LOCO (breakfast_box, pushpins, screw_bag, and splicing_connectors, the four multi-object categories). Each cell reports (P-AUROC, P-AUPRO) on structural anomalies.

Method50%70%
Unpruned(78.9, 62.6)(78.9, 62.6)
EViT(77.6, 63.9)(74.5, 59.8)
ToMe(79.8, 65.4)(77.3, 57.1)
ATS(80.9, 65.0)(79.9, 63.3)
GTP-ViT(80.0, 61.6)(70.9, 46.5)
ADoRe(80.5, 67.3)(80.9, 67.7)

[Q1] "How does the trimmed mean reference μ̄ perform when the normal background is highly heterogeneous or textured? Does forcing a single mean vector across diverse normal tokens risk misclassifying valid but rare normal textures as anomalies?"

It performs well when the normal background is heterogeneous, as the MVTec LOCO results above show (Table R2). And no, it does not risk misclassifying valid but rare normal textures as anomalies: the deviation from μˉ\bar{\mu} only decides which tokens to release, so such a token is simply kept and scored by the detector as usual (Appendix C.13).

Reviewer_H8bTOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ADoRe, a training-free token pruning module for accelerating ViT-based anomaly detectors. It scores each patch by its cosine deviation from a trimmed mean of the tokens, accumulates within-layer ranks over a band of three intermediate layers, and assigns zero anomaly score to the pruned locations. Experiments with CLIP-based zero-shot detectors (AnomalyCLIP, FAPrompt), a 1-shot DINOv2 detector (SubspaceAD), and a video detector (AI-VAD) report speedup with comparable or improved accuracy.

Critical strengths

The observation that pruning criteria should differ between classification and dense anomaly detection is interesting and well motivated. The method is simple, training-free, and deployment-friendly.

Critical weaknesses
  1. Limited range of detection settings. On images, the paper cover essentially two paradigms: CLIP-based zero-shot detection (AnomalyCLIP, FAPrompt) and PCA-based 1-shot detection (SubspaceAD). The method is therefore validated only in the zero-shot and 1-shot regimes, and its effectiveness in standard few-shot settings (e.g., 2-, 4- shot) and the full-shot (all images) setting remains unclear. Experiments in these settings are needed to support the claimed generality.

  2. Missing mainstream UAD paradigms. The most practical and widely used unsupervised AD paradigms are not evaluated: reconstruction-based methods (e.g., Dinomaly) and memory-bank-based methods (e.g., ViT-based PatchCore, AnomalyDINO). Both rely heavily on dense patch features from a frozen ViT. The authors should validate ADoRe under these popular paradigms.

  3. Source of the accuracy gain is unclear. The reported improvement may come mainly from zeroing the scores of the selected tokens, which acts as a mask fused with the detector output, rather than from pruning itself. An ablation is needed: apply the ADoRe criterion to the unpruned model and zero the same token set.

  4. The experiments use only large inputs (518/672), which yield many small patches that are easier to prune. Effectiveness at lower resolutions (e.g., 224/336) should be demonstrated.

Overall, I currently lean towards absolute borderline (2.5), and I will consider raising my score to 3 if these concerns are adequately addressed.

Questions for authors

Please address weaknesses

2 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer H8bT [1/2]

We thank the reviewer for the constructive feedback, and for finding our observation on the task gap between classification and anomaly detection interesting and well motivated, and our method simple, training-free, and deployment-friendly. We address each concern below.

[W1] "Limited range of detection settings. On images, the paper cover essentially two paradigms: CLIP-based zero-shot detection (AnomalyCLIP, FAPrompt) and PCA-based 1-shot detection (SubspaceAD). The method is therefore validated only in the zero-shot and 1-shot regimes, and its effectiveness in standard few-shot settings (e.g., 2-, 4-shot) and the full-shot (all images) setting remains unclear. Experiments in these settings are needed to support the claimed generality."

Following the reviewer's suggestion, we ran SubspaceAD in the 2-shot, 4-shot, and full-shot settings (Tables R1 and R2). ADoRe outperforms existing reductions on most settings, showing that its effectiveness carries over to the standard few-shot and full-shot regimes. We added these results to Appendix C.10 (Tables S12 and S13).

Table R1. SubspaceAD (DINOv2-g/14) in the 2-shot, 4-shot, and full-shot settings on MVTec-AD. Each cell reports (I-AUROC, I-AP), (P-AUROC, P-AUPRO).

Method2-shot4-shotFull-shot
Unpruned(97.8, 98.9), (97.8, 94.6)(98.2, 99.1), (97.9, 94.7)(99.2, 99.6), (98.2, 95.0)
Prune 50%
EViT(97.8, 98.9), (96.6, 93.5)(98.1, 99.0), (96.7, 93.6)(99.1, 99.5), (96.1, 92.6)
ToMe(97.7, 98.8), (97.8, 94.6)(98.1, 99.0), (97.9, 94.7)(98.9, 99.4), (98.1, 94.9)
ATS(96.6, 98.1), (97.2, 93.7)(97.3, 98.2), (97.3, 93.8)(98.8, 99.2), (97.7, 94.4)
GTP-ViT(94.1, 97.0), (96.0, 91.3)(95.3, 97.2), (96.1, 91.5)(96.8, 98.4), (96.7, 92.2)
ADoRe (Ours)(98.1, 99.2), (97.6, 94.7)(98.2, 99.2), (97.7, 94.7)(99.2, 99.6), (97.8, 95.1)
Prune 70%
EViT(97.8, 99.0), (95.8, 91.9)(98.1, 99.0), (95.9, 91.9)(99.0, 99.4), (94.9, 91.4)
ToMe(97.2, 98.3), (97.7, 94.4)(97.4, 98.4), (97.8, 94.5)(98.1, 98.7), (98.0, 94.8)
ATS(92.4, 95.8), (95.5, 90.8)(94.5, 96.2), (95.6, 91.3)(96.7, 97.9), (96.1, 91.6)
GTP-ViT(71.4, 85.5), (69.8, 31.4)(72.3, 86.0), (69.5, 30.7)(75.6, 87.3), (69.7, 29.9)
ADoRe(98.1, 99.2), (96.3, 94.0)(98.2, 99.1), (96.4, 94.2)(99.2, 99.6), (96.9, 94.3)

Table R2. SubspaceAD (DINOv2-g/14) in the 2-shot, 4-shot, and full-shot settings on VisA.

Method2-shot4-shotFull-shot
Unpruned(94.3, 93.9), (98.4, 95.4)(94.4, 93.7), (98.4, 95.9)(98.2, 98.3), (99.1, 96.5)
Prune 60%
EViT(95.2, 95.0), (97.9, 94.9)(95.3, 94.7), (97.9, 94.5)(98.3, 98.4), (98.0, 93.3)
ToMe(94.4, 94.4), (98.3, 95.1)(94.6, 94.1), (98.3, 95.3)(97.8, 98.0), (99.0, 96.0)
ATS(90.3, 89.6), (98.0, 94.4)(90.6, 89.6), (98.1, 94.8)(95.6, 95.2), (99.0, 95.9)
GTP-ViT(92.3, 91.7), (98.1, 94.8)(92.5, 91.6), (98.2, 95.0)(97.2, 97.0), (99.0, 96.0)
ADoRe(95.6, 95.6), (98.3, 96.3)(95.8, 95.7), (98.4, 96.4)(98.3, 98.3), (99.1, 97.0)
Prune 80%
EViT(95.6, 95.7), (96.5, 91.8)(95.8, 95.3), (96.4, 90.9)(98.2, 98.3), (95.4, 86.4)
ToMe(94.2, 94.3), (98.2, 94.8)(94.1, 93.8), (98.2, 95.0)(96.9, 97.3), (98.9, 95.5)
ATS(83.8, 82.5), (96.9, 90.4)(84.8, 83.0), (97.0, 91.1)(91.0, 88.8), (98.4, 92.7)
GTP-ViT(91.3, 90.6), (98.0, 94.3)(91.5, 90.4), (98.1, 94.7)(96.5, 96.2), (98.9, 95.8)
ADoRe(95.8, 95.8), (98.5, 96.3)(96.0, 95.9), (98.5, 96.3)(98.3, 98.4), (98.9, 96.6)
AuthorsOpenReview ↗

Rebuttal for Reviewer H8bT [2/2]

[W2] "Missing mainstream UAD paradigms. The most practical and widely used unsupervised AD paradigms are not evaluated: reconstruction-based methods (e.g., Dinomaly) and memory-bank-based methods (e.g., ViT-based PatchCore, AnomalyDINO). Both rely heavily on dense patch features from a frozen ViT. The authors should validate ADoRe under these popular paradigms."

We applied ADoRe to the reconstruction-based Dinomaly [3] and to two memory-bank-based methods, ViT-based PatchCore [2] and AnomalyDINO [1] (Table R3). On both memory-bank detectors, ADoRe accelerates the encoder by up to 1.9× while improving over the unpruned model in most settings. On Dinomaly, ADoRe keeps the unpruned accuracy at a smaller speedup. Dinomaly reconstructs the features of encoder blocks 2–9, so its encoder ends at block 9 and only a few blocks follow the band. We added these results to Appendix C.11 (Table S14).

Table R3. Memory-bank and reconstruction-based detectors. Each cell reports (I-AUROC, I-AP), (P-AUROC, P-AUPRO).

SettingSpeedupUnprunedADoRe
AnomalyDINO (1-shot)
MVTec-AD, 50%1.66×(97.0, 98.2), (96.0, 92.0)(97.2, 98.4), (96.7, 93.0)
MVTec-AD, 70%1.85×(97.0, 98.2), (96.0, 92.0)(97.2, 98.5), (96.3, 92.3)
VisA, 60%1.71×(89.1, 90.8), (97.3, 91.7)(90.1, 92.1), (98.7, 94.3)
VisA, 80%1.89×(89.1, 90.8), (97.3, 91.7)(90.5, 92.0), (97.9, 93.1)
PatchCore (ViT) (full-shot)
MVTec-AD, 50%1.48×(99.2, 99.7), (97.3, 93.5)(99.3, 99.7), (97.5, 94.5)
MVTec-AD, 70%1.63×(99.2, 99.7), (97.3, 93.5)(99.3, 99.7), (96.8, 93.0)
VisA, 60%1.51×(96.0, 96.5), (98.4, 90.8)(96.8, 97.2), (99.1, 95.3)
VisA, 80%1.79×(96.0, 96.5), (98.4, 90.8)(96.6, 96.9), (98.6, 94.2)
Dinomaly (full-shot)
MVTec-AD, 35%1.29×(99.7, 99.8), (98.3, 94.6)(99.7, 99.9), (98.3, 94.5)
VisA, 55%1.39×(98.8, 99.0), (98.8, 95.1)(98.8, 99.0), (98.6, 95.4)

[W3] "Source of the accuracy gain is unclear. The reported improvement may come mainly from zeroing the scores of the selected tokens, which acts as a mask fused with the detector output, rather than from pruning itself. An ablation is needed: apply the ADoRe criterion to the unpruned model and zero the same token set."

The gain does not come from zeroing alone: pruning itself adds a positive gain at every pruning ratio. The requested ablation is the decomposition in Table S3, where Δscoring\Delta_\text{scoring} zeroes, in the unpruned model, the scores of exactly the tokens that ADoRe releases, and Δencoding\Delta_\text{encoding} is the further gain of the pruned model over this masked one. We now discuss this in Section 3.2 and Appendix C.2.

[W4] "The experiments use only large inputs (518/672), which yield many small patches that are easier to prune. Effectiveness at lower resolutions (e.g., 224/336) should be demonstrated."

Following this suggestion, we evaluated AnomalyCLIP at 336 and 224 (Table R4). ADoRe improves accuracy over the unpruned model in most settings. As the reviewer anticipated, the speedup shrinks, from up to 2.47× at 518 to 1.89× at 336 and 1.62× at 224, since fewer tokens leave less to save. We use large inputs because recent ViT-based detectors run at high resolution to capture small defects (e.g., 518 for AnomalyCLIP [4] and AA-CLIP [5]), and we follow their official settings. We added these results to Appendix C.12 (Table S15).

Table R4. AnomalyCLIP at lower resolutions. Each cell reports (P-AUROC, P-AUPRO). Speedup is the measured ViT encoder latency relative to the unpruned model at the same resolution. At 518, the speedup is 1.85–2.47× (Table 1).

ResolutionSettingUnprunedADoReSpeedup
336MVTec-AD, 50%(88.2, 80.5)(89.7, 83.1)1.45×
MVTec-AD, 70%(88.2, 80.5)(88.2, 81.7)1.65×
VisA, 60%(93.3, 80.1)(95.4, 84.7)1.64×
VisA, 80%(93.3, 80.1)(94.4, 85.5)1.89×
224MVTec-AD, 50%(87.5, 77.5)(88.4, 78.8)1.29×
MVTec-AD, 70%(87.5, 77.5)(86.0, 77.2)1.50×
VisA, 60%(91.9, 74.7)(94.0, 79.8)1.43×
VisA, 80%(91.9, 74.7)(91.7, 79.3)1.62×

[1] Damm, et al. AnomalyDINO: Boosting patch-based few-shot anomaly detection with DINOv2. WACV, 2025.

[2] Roth, et al. Towards total recall in industrial anomaly detection. CVPR, 2022.

[3] Guo, et al. Dinomaly: The less is more philosophy in multi-class unsupervised anomaly detection. CVPR, 2025.

[4] Zhou, et al. AnomalyCLIP: Object-agnostic prompt learning for zero-shot anomaly detection. ICLR, 2024.

[5] Ma, et al. AA-CLIP: Enhancing zero-shot anomaly detection via anomaly-aware CLIP. CVPR, 2025.

Reviewer_bwirOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents ADoRe, a training‑free token reduction module for accelerating Vision Transformer‑based anomaly detection (ViT‑AD). The core motivation is that in anomaly detection (AD), tokens should be pruned according to their estimated normality rather than their importance for global classification predictions. ADoRe scores tokens based on their deviation from the relative censored mean, accumulates intra‑layer ranks within a cross‑layer band, and assigns zero anomaly scores to pruned positions. Experiments are conducted across four anomaly detectors and seven datasets, reporting up to 2.47× encoder speedup. The authors further explore reinvesting saved computational budgets into additional cropping or larger backbones.

Critical strengths

1, The task motivation is well‑grounded, supported by qualitative examples and quantitative measurements of anomalous‑token retention rates. 2, The module requires no extra training. Its scoring mechanism, rank accumulation, and mapping rules are straightforward. 3, Evaluations cover CLIP‑ and DINOv2‑based image anomaly detectors as well as video appearance encoders, spanning industrial, medical, and video benchmarks. Random‑selection control experiments further verify that token selection matters under the evaluated pruning schedules and mapping schemes. The authors also discuss failure cases for large‑defect scenarios and the limited scope of single‑layer encoding analysis.

Critical weaknesses

1, The primary concern lies in the upper‑bound reliability of normality‑driven token selection. Although Table S2 shows that ADoRe mis‑prunes only 4.3% of anomalous tokens under 60% pruning on VisA (versus 13.5% for EViT), the claim that “normality is reliably determined” requires qualifications given the failure modes reported by the authors themselves. Selection relies on relative ranking with a fixed pruning budget, and the initial pruning step only considers a single layer. At 80% pruning on VisA, the scoring contribution drops to −0.2, while an encoding contribution of +1.0 offsets this to yield an overall positive net gain. For the largest‑defect subset of MVTec‑AD, pixel‑level AUROC decreases by 8.79 points with a 60.2% mis‑pruning rate for anomalous tokens. Appendix D derives a theoretical lower bound of 12.1% for budget‑induced information leakage, while the practical leakage reaches 31.1%. This bound demonstrates that partial performance loss is unavoidable under this pruning budget, yet it does not account for the remaining performance gap. 2, The joint effects of token selection and backfilling are not fully disentangled. While competing baselines adopt aggregation, merging, or nearest‑survivor recovery, ADoRe directly assigns zero values to pruned positions. Table S5 only tests alternative backfilling strategies under ADoRe’s own selection criterion; it does not perform cross‑validation between different selectors and different backfilling rules. Consequently, it remains unclear how much of ADoRe’s advantages in main tables stem from the selection criterion itself versus score suppression at pruned positions. The manuscript lacks direct evidence for this selector‑by‑backfilling interaction. 3, Reproducibility issues persist. Section 4.2 defines a retention statistic and places the band where this statistic rises while sufficient subsequent layers remain available, but provides no deterministic thresholds or algorithmic procedure for band selection. Appendix B.1 lists the final selected bands yet offers no guidance for migrating to alternative backbones. Section 5.3 mentions contextual cropping for maximal‑size targets with fixed fusion settings, yet cropping dimensions, target sampling counts, and fusion hyper-parameters are not fully specified.

Questions for authors

1, Positioning relative to prior work: please clarify the technical distinctions from KeepAD and ideally provide comparative experimental results. The closest baseline in the current evaluation is GPT‑ViT (2024); why are more recent papers not included for comparison? 2, Under identical pruning schedules, could the authors conduct cross‑evaluation of “zero‑value / nearest‑survivor backfilling” for ADoRe alongside representative pruning baselines (referring to Table 1 and Table S5)? 3, What are the exact rules for mapping calibration statistics to the selected pruning band? What are the contextual‑cropping dimensions, target‑selection counts, and score‑fusion hyper-parameters used in Figure 4a?

5 comments
AuthorsOpenReview ↗

Rebuttal for Reviewer bwir [1/3]

We thank the reviewer for the detailed and constructive feedback, and for recognizing the well-grounded motivation of our work, the simplicity of our training-free design, and the breadth of our evaluation across detectors and domains. We address each concern below.

[W1] "The primary concern lies in the upper-bound reliability of normality-driven token selection. Although Table S2 shows that ADoRe mis-prunes only 4.3% of anomalous tokens under 60% pruning on VisA (versus 13.5% for EViT), the claim that "normality is reliably determined" requires qualifications given the failure modes reported by the authors themselves. … Appendix D derives a theoretical lower bound of 12.1% for budget-induced information leakage, while the practical leakage reaches 31.1%. This bound demonstrates that partial performance loss is unavoidable under this pruning budget, yet it does not account for the remaining performance gap."

Normality-driven selection is reliable when the anomaly is small, as is usual, and we now state this condition in Section 4.2. The remaining gap comes from the few images with very large defects: the 31.1% is pooled over anomalous tokens, so these images dominate it. On the 95.4% of MVTec-AD images whose defect covers at most 20% of the image, leakage averages 10.5% per image and pixel AUROC stays close to the unpruned model, so the gap is small.

The leakage beyond the 12.1% bound is selection error, and it is expected on large defects (Table R1): the reference is estimated from the image's own tokens, so an anomaly that is no longer a minority of the image cannot stand out as a deviation. Even there, ADoRe's leakage stays below that of EViT and ATS. We added this breakdown to Appendix D (Table S18).

Table R1. Ratio (%) of anomalous tokens pruned, by defect size. Forced-by-budget error: the ratio any selector must prune, because the image has more anomalous tokens than the budget keeps. Selection error: the ratio pruned beyond that. Total error: their sum. Large: the defect covers more than 20% of the image. Small: at most 20%. AnomalyCLIP, MVTec-AD, 70%.

Defect size# of imagesForced-by-budget errorSelection errorTotal error (ADoRe)Total error (EViT)Total error (ATS)
Large(>20%)5834.823.958.664.062.1
Small(≤20%)1,2000.016.416.428.729.6
All1,25812.119.031.141.040.9
AuthorsOpenReview ↗

Rebuttal for Reviewer bwir [2/3]

The reviewer's four other points are answered below.

"Selection relies on relative ranking with a fixed pruning budget, …"

The fixed budget is not a significant source of error. To check this, we replaced it with an adaptive budget, which thresholds the same deviation score so that each image keeps as many tokens as it needs. At the same average FLOPs, the overall accuracy is almost unchanged (pixel AUROC 88.6 vs. 88.7, Table R2). We also report this experiment in the revision (Section 5.4, Table 6).

Table R2. Fixed vs. adaptive budget, by defect size (change in pp in parentheses). Large: the defect covers more than 20% of the image. Small: at most 20%. AnomalyCLIP, MVTec-AD, 70%. GFLOPs are averaged over the images in each group.

Defect size# of imagesP-AUROC (fixed)P-AUROC (adaptive)P-AUPRO (fixed)P-AUPRO (adaptive)GFLOPs (fixed)GFLOPs (adaptive)
Large(>20%)5878.381.9 (+3.6)56.359.5 (+3.2)638683
Small(≤20%)1,20092.292.4 (+0.2)83.283.0 (−0.2)638638
All1,25888.688.7 (+0.1)81.681.3 (−0.3)638637

"… and the initial pruning step only considers a single layer."

The initial step prunes the fewest anomalous tokens of the three band layers (1.36% on MVTec-AD and 0.14% on VisA, Table R3). The budget is spread over the three layers, so the initial step removes only the tokens with the smallest deviation, which are the clearest normal ones. The later steps then use the deviation accumulated over the band rather than a single layer. We added this result to Appendix C.8 (Table S10).

Table R3. Ratio (%) of anomalous tokens pruned at each band layer (P1: first, P2: second, P3: third band layer), among those that reach it. AnomalyCLIP.

SettingMethodP1P2P3
MVTec-AD, 50%EViT6.137.9811.25
ADoRe1.364.618.63
VisA, 60%EViT1.051.3911.34
ADoRe0.140.783.39

"At 80% pruning on VisA, the scoring contribution drops to −0.2, while an encoding contribution of +1.0 offsets this to yield an overall positive net gain."

While the scoring contribution of ADoRe drops to −0.2 at 80% pruning, that of the CLS criterion drops to −3.7 (Table S3, Appendix C.2). The −0.2 appears only at this most aggressive ratio: at 50–70% it stays positive (+0.6 to +0.9).

"For the largest-defect subset of MVTec-AD, pixel-level AUROC decreases by 8.79 points with a 60.2% mis-pruning rate for anomalous tokens."

This is the limitation we report in the Discussion (Appendix D). As shown there, such defects are rare: this subset holds 40 of the 1,258 anomalous images in MVTec-AD (3.2%) and none in VisA (Table S17). We clarified its cause in the revision: the reference is estimated from the image's own tokens, so a defect larger than the trimmed fraction (τ = 0.2) pulls the reference toward itself.

[W2] "The joint effects of token selection and backfilling are not fully disentangled. While competing baselines adopt aggregation, merging, or nearest-survivor recovery, ADoRe directly assigns zero values to pruned positions. Table S5 only tests alternative backfilling strategies under ADoRe's own selection criterion; it does not perform cross-validation between different selectors and different backfilling rules. Consequently, it remains unclear how much of ADoRe's advantages in main tables stem from the selection criterion itself versus score suppression at pruned positions. The manuscript lacks direct evidence for this selector-by-backfilling interaction."

[Q2] "Under identical pruning schedules, could the authors conduct cross evaluation of "zero-value / nearest-survivor backfilling" for ADoRe alongside representative pruning baselines (referring to Table 1 and Table S5)?"

We disentangled the two by holding the backfilling fixed and varying only the selector. In the main tables, only ADoRe assigns zero values to pruned positions, while each baseline keeps its own rule (aggregation, merging, or nearest-survivor recovery). As requested in Q2, Table R4 applies the same zero-value backfilling to every baseline, which completes the other half of Table S5. ADoRe remains the best in almost all cases, so the remaining advantage comes from the selection criterion itself. We added these results to Section 5.4 (Table 5).

Table R4. Pixel (AUROC, AUPRO) with zero-value backfilling for every method, 70%.

MethodAnomalyCLIP, MVTec-ADAnomalyCLIP, VisAFAPrompt, MVTec-ADFAPrompt, VisA
EViT(86.1, 80.1)(93.9, 85.0)(86.4, 80.2)(93.6, 85.5)
ToMe(72.7, 48.9)(84.4, 58.9)(72.4, 31.4)(76.7, 32.6)
ATS(83.4, 73.4)(92.2, 81.9)(82.7, 71.5)(90.2, 79.6)
GTP-ViT(75.4, 57.8)(87.0, 65.7)(49.1, 1.1)(48.8, 1.7)
ADoRe(88.6, 81.6)(95.5, 86.4)(86.3, 82.5)(95.5, 88.5)
AuthorsOpenReview ↗

Rebuttal for Reviewer bwir [3/3]

[W3] "Reproducibility issues persist. Section 4.2 defines a retention statistic and places the band where this statistic rises while sufficient subsequent layers remain available, but provides no deterministic thresholds or algorithmic procedure for band selection. Appendix B.1 lists the final selected bands yet offers no guidance for migrating to alternative backbones. Section 5.3 mentions contextual cropping for maximal-size targets with fixed fusion settings, yet cropping dimensions, target sampling counts, and fusion hyper-parameters are not fully specified."

[Q3] "What are the exact rules for mapping calibration statistics to the selected pruning band? What are the contextual cropping dimensions, target selection counts, and score-fusion hyper-parameters used in Figure 4a?"

Following the reviewer's comment, we added a deterministic rule for band selection and the full contextual-cropping settings to the revision, as follows.

Band selection. We compute ϕ(l)\phi(l) (Eq. 5) with a 20% cut on 100 normal images from another dataset, and take as the band the three layers starting from the first layer where ϕ(l)\phi(l) reaches 95%. This gives blocks 9–11 for CLIP ViT-L/14 and 15–17 for DINOv2-g/14, and the same procedure applies unchanged to a new backbone. We added this rule to Section 4.2, the caption of Fig. 3, and Appendix B.1.

Contextual cropping in Fig. 4a. A context crop is added for the largest 22.5% of objects in each frame. It enlarges the bounding box by 1.5× and is resized to 336 × 336. Its normalized score is added to the original score with α = 0.5 for the largest 12.5% of objects and α = 0.1 for the rest. We added these settings to Appendix B.1.

[Q1] "Positioning relative to prior work: please clarify the technical distinctions from KeepAD and ideally provide comparative experimental results. The closest baseline in the current evaluation is GPT-ViT (2024); why are more recent papers not included for comparison?"

Following the reviewer's suggestion, we added three more recent methods, SAINT [2], MaMe [3], and RAPID [4], and ADoRe matches or outperforms all three (Table R5; Appendix C.9, Table S11). Our original baselines remain the standard comparisons in recent training-free token reduction work [1, 2]. As for KeepAD, it trains its token selectors on auxiliary defect data, while ADoRe needs no training (Section 2.2). A direct comparison is not yet possible, as its code has not been released, and KeepAD counts as contemporaneous work under the ICLR 2027 reviewer guidelines.

Table R5. Comparison with recent token reduction methods. AnomalyCLIP, Image (AUROC, AP) and Pixel (AUROC, AUPRO).

SettingMethodImagePixel
MVTec-AD, 50%SAINT(91.3, 96.2)(88.0, 80.1)
MaMe(89.3, 95.4)(88.3, 81.4)
RAPID(90.7, 95.8)(88.5, 81.6)
ADoRe(92.7, 96.8)(89.7, 82.4)
MVTec-AD, 70%SAINT(86.9, 93.5)(81.0, 62.1)
MaMe(87.8, 94.7)(88.1, 81.3)
RAPID(88.0, 94.5)(86.6, 75.6)
ADoRe(92.1, 96.7)(88.6, 81.6)
VisA, 60%SAINT(82.9, 85.9)(94.5, 85.5)
MaMe(80.9, 84.9)(94.4, 85.9)
RAPID(82.7, 86.0)(94.5, 85.4)
ADoRe(83.7, 86.9)(95.8, 85.9)
VisA, 80%SAINT(81.6, 84.9)(93.7, 83.8)
MaMe(79.1, 83.5)(94.2, 85.7)
RAPID(72.0, 76.3)(91.7, 71.7)
ADoRe(84.0, 86.9)(94.7, 86.3)

[1] Wang, et al. Zero-TPrune: Zero-shot token pruning through leveraging of the attention graph in pre-trained transformers. CVPR, 2024.

[2] Jeddi, et al. Similarity-aware token pruning: Your VLM but faster. arXiv, 2025.

[3] Huo and Li. MaMe: Matrix-based token merging. CVPR Findings, 2026.

[4] Choi and Jang. RAPID: Layer-wise redundancy-aware pruning and importance-driven token merging for efficient ViT. KDD, 2026.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Rebuttal

We thank the AC and all reviewers for their constructive feedback. Reviewers found the motivation well-grounded (all), the method simple and training-free (all), the evaluation broad across detectors and domains (bwir, 5i3q), and the theoretical analysis insightful (5i3q). We summarize the main concerns and our responses below.

1. Large defects and fixed budget (bwir, 5i3q)

  • Concern: A fixed budget can prune anomalous tokens when a defect covers a large area.
  • Response: Such defects are rare (58 of 1,258 MVTec-AD images), and even there ADoRe prunes fewer anomalous tokens than EViT and ATS (Table S18). The fixed budget is not a major source of error: replacing it with an adaptive budget at matched average FLOPs leaves overall accuracy almost unchanged (Table 6).

2. Selection vs. backfilling (bwir, H8bT)

  • Concern: The effects of token selection and zero-value backfilling could be separated further.
  • Response: With the same backfilling for all baselines, ADoRe remains the best in almost all cases (Table 5), and pruning adds a gain beyond zeroing at every ratio (Table S3).

3. Additional detectors and settings (H8bT)

  • Concern: Few-shot, full-shot, memory-bank and reconstruction-based detectors would broaden the evaluation.
  • Response: ADoRe outperforms existing reductions in most few- and full-shot settings (Tables S12–S13). On memory-bank and reconstruction-based detectors, it accelerates the encoder by up to 1.9× while matching or improving accuracy in most settings (Table S14).

4. Additional baselines (bwir)

  • Concern: Comparisons with more recent token reduction methods would help.
  • Response: ADoRe matches or outperforms SAINT, MaMe and RAPID (Table S11).

5. Lower resolution and multi-object scenes (H8bT, 5i3q)

  • Concern: Effectiveness at lower resolutions and on heterogeneous scenes could be examined further.
  • Response: At 336 and 224, ADoRe improves accuracy in most settings with speedups of up to 1.89× and 1.62× (Table S15). On MVTec LOCO, it stays above the unpruned model and leads on most metrics (Table S16).

6. Implementation details (bwir)

  • Concern: Band selection and cropping settings could be specified more precisely.
  • Response: We added a deterministic band-selection rule that transfers to new backbones (Sec. 4.2) and the full cropping settings (App. B.1).

We thank the AC and reviewers again for helping us improve the paper.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.