- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper aims to reduce the computational overhead of Transformer-based zero-shot anomaly detection (ZSAD) models. Based on the observation that anomalies usually cover only a small portion of an image and normal patches stabilize in earlier layers, the authors propose Fast-ZSAD. The method places an Anomaly-Aware Patch Router at layer 12 to exit confident normal patches early, while sending a fixed ratio () of suspicious patches through the deeper layers. To handle class imbalance during router training, the authors introduce Contrast-Aware Regularization. They also propose Depth-Spatial Feature Correction (DSFC) to mitigate feature drift in deeper layers by combining multi-depth representations using local spatial support and cross-depth feature transitions.
Critical strengths
This paper is well-written and very easy to follow. Evaluations across AACLIP, AD-DINOv3, and RareCLIP on industrial and medical datasets report average speedups of 1.58x–1.84x while largely preserving detection scores.
Critical weaknesses
- The method fixes the deep-inference budget to a constant ratio of . For completely normal images, still pushing 30% of the tokens through the remaining 12 Transformer layers wastes considerable compute. On the other hand, for anomalous images, while the Complete Miss Rate (CMR) at is 0% on MVTec AD, the Ground-Truth Anomaly Patch Recall (GT Recall@30%) is only 89.95% on AACLIP and 86.72% on AD-DINOv3. This means 10% to 13% of actual defect patches are exited early at layer 12 and never get processed by the deeper layers. Using a fixed Top- selection rather than a score-based threshold seems driven mainly by GPU batching constraints rather than actual image content.
- DSFC combines four hand-crafted terms: routing density, shallow feature coherence, transition magnitude, and neighborhood transition consistency. The final fusion weights rely on several normalization scalers, affine cosine mappings, and an exponential scaling hyperparameter. As shown in Fig. 9, performance degrades noticeably once , suggesting that the formulation is quite sensitive to hyperparameter tuning.
- Fig. 3 argues that sparse deep inference causes severe "prediction drift," which distorts features and increases false positives in deeper layers. However, Eq. 15 and Eq. 16 assign exponentially higher weights to deeper features that have larger transition distances (). If deeper layers are indeed prone to unfaithful drift, giving higher weights to layers with larger feature shifts seems to work against the paper's own motivation.
- In Table 5 (AD-DINOv3 on MVTec AD), the Naive Adaptive baseline gets 75.08 mScore, whereas adding the Map Router drops the score to 52.44 (-22.64 points) and raises CMR to 43.80% (Table 7). Distilling from the full model's own predictions causes the router to miss defects in almost half of the anomalous images. Simply attributing this drop to "uniform map supervision is insufficient" without analyzing the root cause (such as gradient imbalance, softmax saturation, or adapter interference) leaves an important failure mode unexplained.
- The experiments only compare against token reduction methods borrowed from general vision tasks (ATS, DTOP, Cropr), missing two straightforward baselines: (i) simply lowering the input resolution from to or , which gives a similar – speedup without changing the architecture; and (ii) directly taking predictions from layer 16 or layer 20 for all tokens without dynamic routing. Without these comparisons, it is hard to tell how much benefit comes specifically from patch-wise sparse routing.
Questions for authors
- In a single-image streaming setup (Batch Size = 1), how does Fast-ZSAD perform if patch selection is based on an absolute anomaly score threshold rather than a fixed Top-30% ratio? What is the actual runtime and skip rate on exclusively normal validation sets?
- How does Fast-ZSAD compare in mScore and latency against the full baseline executed at reduced input resolutions ( or ), or against early-terminating the entire model uniformly at layer 16?
- Provide head-to-head empirical results against KeepAD (Hou et al., 2026) under identical hardware and evaluation protocols.
- In Table 8, a non-uniform late-biased prior was hand-crafted for AD-DINOv3. What happens if this exact prior is applied to AACLIP and RareCLIP, or conversely, if the uniform prior is enforced on AD-DINOv3 across all datasets?
5 comments
W4: Diagnosing the Map-only Router Degradation on AD-DINOv3.
Thank you for highlighting this issue. The Map-only Router degradation on AD-DINOv3 is related to sigmoid saturation caused by the large similarity scaling factor in the original AD-DINOv3 anomaly scoring formulation. Specifically, the patch-level anomaly probability is computed as
{\exp(100z_i^\top t_a)+\exp(100z_i^\top t_n)} = \sigma(m_i), \quad m_i=100z_i^\top(t_a-t_n),$$ where $z_i$ denotes the projected patch feature, and $t_a$ and $t_n$ are the abnormal and normal text embeddings, respectively. The Map-only Router minimizes the MSE between its predicted anomaly probability $p_i$ and the full-depth teacher prediction $q_i$: $$\mathcal{L}_{\mathrm{map}} = \frac{1}{N}\sum_{i=1}^{N}(p_i-q_i)^2$$ The corresponding gradient with respect to the student logit $m_i$ is $$\frac{\partial\mathcal{L}_{\mathrm{map}}}{\partial m_i} =\frac{2}{N}(p_i-q_i)p_i(1-p_i).$$ The large similarity scaling factor can push predictions toward saturated regions, where $p_i(1-p_i)$ approaches zero and suppresses corrective gradients. Moreover, under normal-dominated supervision, average MSE may favor low anomaly responses without preserving the teacher's relative patch ranking. Consequently, Map-only may converge to a degenerate low-response solution with unreliable patch discrimination despite a low distillation loss. CAR mitigates this issue by extracting relative anomaly evidence from standardized teacher predictions and introducing feature-space compactness and separation objectives that do not directly depend on saturated sigmoid gradients. Accordingly, CAR improves mScore from 52.44 to 76.64 and reduces CMR from 43.80% to 0% on AD-DINOv3 (Tables 5 and 7). >W5, Q2: Comparison with Resolution Scaling and Uniform Early Exiting. > Thank you for your valuable suggestion. We conduct additional comparisons on AACLIP across five benchmarks, including reduced input resolutions (384 × 384 and 448 × 448) and uniform early exiting at layers 16 and 20. Performance–Efficiency Comparison on AACLIP: | Method | MVTec AD mScore ↑ | Speedup ↑ | VisA mScore ↑ | Speedup ↑ | BTAD mScore ↑ | Speedup ↑ | MPDD mScore ↑ | Speedup ↑ | MedAD mScore ↑ | Speedup ↑ | |---|---:|---:|---:|---:|---:|---:|---:|---:|---:|---:| | AACLIP (518×518) | 76.31 | – | 64.24 | – | 81.47 | – | 63.38 | – | 67.52 | – | | AACLIP (384×384) | 73.99 | 1.62× | 62.54 | 1.55× | 76.30 | 1.63× | 60.58 | 1.58× | 66.21 | **1.69×** | | AACLIP (448×448) | 76.08 | 1.26× | 63.58 | 1.30× | **80.71** | 1.33× | 61.95 | 1.24× | 67.10 | 1.36× | | Early Exit (L16) | 71.98 | 1.28× | 57.65 | 1.33× | 72.41 | 1.38× | 53.99 | 1.27× | 62.43 | 1.41× | | Early Exit (L20) | 73.25 | 1.13× | 57.97 | 1.15× | 73.73 | 1.17× | 56.61 | 1.13× | 61.84 | 1.18× | | **Fast-ZSAD (Ours)** | **76.51** | **1.63×** | **64.55** | **1.70×** | 80.40 | **1.80×** | **62.76** | **1.71×** | **68.85** | 1.62× | The results show that Fast-ZSAD achieves a favorable performance–efficiency trade-off compared with these simple acceleration alternatives. For instance, on MVTec AD, Fast-ZSAD achieves 76.51 mScore at 1.63× speedup, compared with 73.99 at 1.62× for resolution reduction to 384 × 384. Across five benchmarks, Fast-ZSAD achieves 1.62×–1.80× speedups while largely preserving the original detection performance. In contrast, uniform early exiting incurs larger performance degradation with lower speedups in the evaluated settings. >Q1: Absolute-Threshold Routing under Batch-1 Streaming Inference. > Thank you for this suggestion. We compare fixed Top-30% and absolute-threshold routing on AACLIP under batch-1 inference, with results shown below. | Method | Full-set mScore ↑ | Full-set Latency (ms/image) ↓ | Normal-only Latency (ms/image) ↓ | Normal Patch Skip Rate ↑ | Full Deep-stage Skip Rate ↑ | |---|---:|---:|---:|---:|---:| | Teacher | 76.31 | 98.31 | 98.90 | 0% | 0% | | Top-30% | 76.51 | 71.13 | 70.51 | 70.05% | 0% | | Threshold −8 | 76.38 | 86.20 | 82.16 | 62.72% | 23.84% | | Threshold −6 | 75.30 | 77.90 | 70.38 | 81.36% | 43.07% | | Threshold 0.3 | 74.80 | 58.63 | 54.22 | 99.99% | 91.49% | *Full deep-stage skip rate denotes the percentage of normal images for which no patch is forwarded to the remaining Transformer blocks. All latency measurements are conducted with batch size = 1.* Threshold-based routing can skip the entire deep stage for more normal images, but at the cost of detection accuracy. Notably, Top-30% achieves higher mScore than Threshold −6 (76.51 vs. 75.30) at nearly identical normal-image latency, supporting its favorable accuracy–efficiency trade-off.Q3: Clarification on the Comparison with KeepAD.
Thank you for this suggestion. A direct comparison with KeepAD is currently infeasible because its official repository provides only the README and license, without implementation code, checkpoints, or evaluation scripts. Moreover, KeepAD reports batch-1 FP32 throughput on an RTX 5090, whereas our experiments use batch size 32 on an RTX A6000, with different anomaly detectors and evaluation metrics, making direct comparisons of published efficiency numbers unfair. Methodologically, Fast-ZSAD accelerates existing ZSAD models through patch-wise adaptive inference, whereas KeepAD relies on a separately trained dense detector and progressive defect-preserving token pruning.
Q4: Generalization of Layer Priors across Models and Datasets.
Thank you for this valuable suggestion. We have conducted additional experiments by applying the late-biased prior to AACLIP and RareCLIP, and the uniform prior to AD-DINOv3 across five benchmarks. The results are shown below.
Cross-model and cross-dataset generalization of layer priors (mScore):
| Model | Prior | MVTec AD | VisA | BTAD | MPDD | MedAD | Avg. |
|---|---|---|---|---|---|---|---|
| AACLIP | Uniform | 76.51 | 64.55 | 80.40 | 62.76 | 68.85 | 70.61 |
| Late-biased | 75.57 | 63.57 | 78.19 | 62.41 | 66.58 | 69.26 | |
| RareCLIP | Uniform | 87.98 | 77.11 | 85.18 | 64.69 | 67.38 | 76.47 |
| Late-biased | 86.66 | 77.07 | 84.42 | 64.59 | 66.70 | 75.89 | |
| AD-DINOv3 | Uniform | 77.05 | 68.92 | 72.72 | 67.15 | 64.86 | 70.14 |
| Late-biased | 78.53 | 70.56 | 74.68 | 68.50 | 64.41 | 71.34 |
The results show that the uniform prior performs better on AACLIP and RareCLIP, while the late-biased prior achieves higher average performance on AD-DINOv3. This indicates that different ZSAD models benefit from different layer-prior configurations, supporting our baseline-specific design.
We sincerely thank the reviewer for the valuable and constructive comments, which help us improve our work. We address the concerns as follows.
Thank you for raising this important concern. Fast-ZSAD adopts a fixed computational budget while adaptively selecting anomaly-relevant patches for deeper inference. The default (ρ=0.3) is chosen based on the accuracy–efficiency trade-off in Fig. 7(a), providing predictable computation despite some redundancy on fully normal images.
Importantly, early-exited patches retain anomaly predictions learned through full-depth map distillation and CAR (Eq. 6), rather than raw shallow-layer scores. Thus, GT Recall@30% measures anomalous-patch coverage for deep refinement, not the final anomaly detection recall. Tables 6–7 further show that the learned router improves GT Recall over direct shallow-score routing from 85.98% to 89.95% on AACLIP and from 78.48% to 86.72% on AD-DINOv3.Our batch-1 experiments in Q1 further show that fixed Top-30% routing better preserves detection accuracy than threshold-based routing at comparable normal-image latency, supporting its use as an accuracy–efficiency choice rather than merely a GPU batching constraint.
Thank you for the constructive comments. We conduct leave-one-out ablations to evaluate the four DSFC signals on AACLIP and AD-DINOv3, and further examine the sensitivity to λsd on AD-DINOv3.
Individual Signal Ablation.
The ablation results suggest that these signals jointly contribute to the effectiveness of DSFC.
Sensitivity to λsd.
On AD-DINOv3, mScore varies by less than 0.30 points across λsd∈[0,5], remaining relatively stable over [0,3.5]. The degradation at larger values is less pronounced than on AACLIP (Fig. 9), indicating backbone-dependent sensitivity rather than a consistently narrow stable range.
Thank you for this insightful comment. Prediction drift and cross-depth feature evolution represent two distinct phenomena. Prediction drift refers to the discrepancy between sparse-token and full-token inference at the same depth (Fig. 3), whereas cross-depth evolution measures representation changes across successive layers. As shown in Fig. 1(b), anomalous patches exhibit larger cross-layer score variations, motivating us to exploit this feature evolution.
Importantly, DSFC does not determine fusion weights solely based on transition magnitude. As defined in Eq. (15), the depth signal is
Dil=d^ilκil,
where d^il measures the normalized transition magnitude and κil captures its consistency with neighboring patch transitions, reducing the influence of large but locally inconsistent changes. The fusion weights are then computed as
wil=∑q∈LDexp(λsdSiDiq)exp(λsdSiDil)
Together with local spatial support Si, these signals jointly determine the fusion weights, rather than simply favoring larger cross-depth changes. While they cannot eliminate all prediction drift, they enable spatially informed multi-depth feature fusion.The effectiveness of this design is supported by our ablation in Fig. 6, where removing depth modeling reduces mScore from 76.51 to 75.20 on AACLIP and from 78.53 to 77.05 on AD-DINOv3, demonstrating the benefit of incorporating cross-depth information into feature correction.