acceptodds
Under review as a conference paper at ICLR 2027

EvoSAE: Evolving Sparse Autoencoder for Few-Shot Industrial Anomaly Detection

Abstract

Few-shot anomaly detection (FSAD) aims to identify abnormal samples from only a limited number of normal examples, gaining increasing attention due to its effectiveness and flexibility for new industrial scenarios. However, it is usually hard for existing FSADs to discover sufficient knowledge from scarce reference samples. Moreover, most approaches focus on feature-level representation while overlooking the polysemanticity of neurons in foundation models. In the paper, we introduce EvoSAE, a self-improving FSAD framework built upon sparse autoencoders (SAE). Moving beyond feature-level memory construction, EvoSAE establishes an atom-level memory by disentangling features as sparse overcomplete directions. These semantic atoms provide more fine-grained representations of normal patterns and enhance the compositional capability for modeling unseen features. Importantly, we softly weight test patches according to their reconstruction errors, assigning larger weights to patches that are more likely to be normal. The weighted patches are then used to fine-tune the SAE during inference, enabling its semantic atoms to progressively adapt to the test distribution. Extensive experiments on three industrial anomaly detection benchmarks demonstrate that EvoSAE consistently outperforms recent state-of-the-art FSAD methods across different shot settings. Visualizations of the learned atoms further reveal their semantic interpretability in real-world anomaly detection scenarios.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
2D anomaly detection, few shot learning, continual test-time adaptation, sparse autoencoder

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_LgQ4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes EVoSAE, which uses BatchTopK SAE to decompose DINOv2 dense patch features into sparse "semantic atoms" and construct atom level normal memory. During testing, soft normal weights are constructed based on reconstruction errors, and SAE is continuously fine tuned to allow memory to evolve on the flow test samples.

Critical strengths

The proposed method is simple and is easy to follow, and show the promising results on datasets.

Critical weaknesses

Poor clarity and rigor. In the paper, there are many symbols that are inconsistent, such as Eq (1) and line 141 and Eq. (5) and line 200. "x" is not uniform. In Eq. (6), what is g_theta? It is not explained. Besides, there are many format issues that can affect readability, such as lines 198-199, lines 206-207, and so on. Besides, in References, there are multiple same references introduced, such as lines 547-553, lines 604-610, and so on. Overall, many minor issues lead to a big problem.

Unfair comparison. In the paper, the DINOv2-G is used, which is a large model. However, the results of baselines are from the original papers, where baselines' feature extraction is based on DINOv2-B, DINOv2-L, or other backbones. So, different backbones affect results. From the Appendix, it is found that DINOv2-B or DINOv2-L for the proposed method is not superior to existing methods. Thus, it is recommended to use the same backbones to conduct experiments, for fair comparison. Overall, the proposed method is simple but relies on pre-trained networks.

Memory overhead. The proposed method constructs normal memory that consists of the learned decoder directions of an SAE. Please report the memory storage compared with existing feature-level-based memory construction methods to show the advantage.

Questions for authors

See weakness. The main issues include poor clarity and rigor and potential unfair comparison. Besides, it is recommended to report memory overhead due to the claimed non-feature-level memory construction. The rating will be reconsidered during rebuttal.

4 comments
AuthorsOpenReview ↗

Rebuttal by Authors

We thank Reviewer LgQ4 for your insightful comments and suggestions! We appreciate your positive reviews on the simple idea and promising results. Below are our point-to-point clarifications to address your concerns. Feel free to discuss with us if you have further questions!

W1: Poor clarity and rigor.

We sincerely thank the reviewer for your careful reading and constructive feedback. We will carefully correct the notation inconsistencies, typos, references, and related descriptions in the revised manuscript.

W2: Unfair comparison

We sincerely apologize to the reviewer for the confusion caused by the incorrectly reported baseline results. This was an oversight on our part. Since SubspaceAD is the most closely related and competitive baseline, we followed its experimental configuration, including the DINOv2-G backbone and random seeds. However, we directly adopted the results of AnomalyDINO and other baselines from SubspaceAD without double checking. To address this issue, we have now included the results of AnomalyDINO evaluated with DINOv2-G and corrected the corresponding comparisons. We will update Table 1 accordingly once the remaining baseline evaluations are completed.

SetupMethodMVTec-AD Image AUROC/AUPRMVTec-AD Pixel AUROC/PROVisA Image AUROC/AUPRVisA Pixel AUROC/PRO
1-shotAnomalyDINO98.1/99.296.9/93.992.9/93.897.9/92.9
SubspaceAD98.0/99.097.6/93.793.3/93.298.3/93.4
EvoSAE (Ours)98.4/99.397.2/95.994.8/94.498.9/95.9
2-shotAnomalyDINO98.5/99.397.2/94.493.7/94.498.0/93.3
SubspaceAD98.1/99.197.8/94.094.1/93.798.4/93.5
EvoSAE (Ours)98.7/99.597.5/96.395.3/94.999.0/96.3
4-shotAnomalyDINO98.7/99.597.4/94.894.8/95.298.3/94.4
SubspaceAD98.4/99.398.0/94.294.5/93.898.5/94.1
EvoSAE (Ours)99.1/99.797.8/96.595.6/95.199.1/96.6

W3: Memory overhead

Following the reviewer's suggestion, we report the normal memory storage and peak GPU memory usage below.

1) Normal Memory Storage. AnomalyDINO stores dense patch features, resulting in the largest memory footprint. In contrast, SubspaceAD uses a compact PCA subspace and achieves the lowest storage cost. EvoSAE employs an SAE-based atom-level memory, reducing storage by 33.3% compared with AnomalyDINO, although requiring more storage than SubspaceAD.

2) Peak GPU Memory. AnomalyDINO achieves the lowest peak GPU memory usage through feature matching without parameter updates. In contrast, SubspaceAD requires additional computation for PCA-based reconstruction, and EvoSAE incurs memory overhead from gradient-based SAE adaptation. EvoSAE uses slightly more peak GPU memory than SubspaceAD (28.04 vs. 25.59 GiB).

3) Performance Trade-off. Despite its additional adaptation overhead, EvoSAE achieves the best results in 3 out of 4 metrics, demonstrating the effectiveness of its evolving atom-level memory for few-shot anomaly detection.

Table: Efficiency and detection performance comparison on MVTec-AD under the 4-shot setting. Best results are highlighted in bold.

MethodNormal Memory (MiB) ↓Peak GPU Memory (GiB) ↓Image AUROC ↑Image AUPR ↑Pixel AUROC ↑Pixel PRO ↑
AnomalyDINO432.287.8798.799.597.494.8
SubspaceAD11.5925.5998.499.398.094.2
EvoSAE (Ours)288.1928.0499.199.797.896.5

Thank you for your effort and response. The response has addressed some of my concerns. The proposed method is simple-effective, but it still incurs increased memory overhead despite the proposed atom-level memory.

Besides, please upload the new PDF version, since the current PDF is still the previous one, which makes me hard to tell the clarity and rigor of the paper.

We sincerely thank you for your response and valuable suggestions, which have helped us improve the quality of our submission.

We would like to clarify two points:

  • During the rebuttal period, we conducted extensive additional experiments, some of which required substantial computational time. Therefore, we did not update our paper at our first rebuttal. We have now uploaded a revised version that corrects typos, mathematical notation inconsistencies, formatting issues, and unclear descriptions. All modifications are highlighted in blue for ease of reference. We would greatly appreciate it if you could review the updated manuscript.

  • In fact, Our atom-level memory requires less storage than the feature-based memory of AnomalyDINO (288.19 MiB vs. 432.28 MiB), achieving a 33.3% reduction. Note that SubspaceAD is not a feature memory-bank-based method; instead, it employs a compact PCA subspace to reconstruct test features, resulting in a smaller memory footprint. Although our method requires additional GPU memory for test-time adaptation, it achieves a favorable trade-off between memory efficiency and anomaly detection performance.

Thank you for your response. The newly uploaded PDF addresses most of my concerns. However, the overhead (memory and even inference time) is still a major concern. The memory overhead should have been much smaller as expected, due to atom-level memory construction. But it still incur over hundreds of MiB. Besides, the performance of the proposed method mainly depends on pre-trained feature extractors (e.g., DINO series) like many existing FSAD methods. Thus, in this case, the proposed method may lack competitiveness if it has no edge in computational efficiency.

Reviewer_owoKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper addresses few-shot industrial anomaly detection. It argues that feature-level memories are too coarse because foundation-model neurons are polysemantic, and stores normality at the level of dictionary atoms instead. A BatchTopK sparse autoencoder is trained on multi-scale DINOv2 patch features of the support images, and its decoder directions serve as the normal memory (Sec. 3, 4.1). At test time each patch gets a soft normality weight from its reconstruction error (Sec. 4.2), the weighted patches fine-tune the SAE (five steps per image, App. A.1), and the adapted parameters are carried to the next image instead of being reset (Sec. 4.3). On MVTec-AD, VisA and MPDD at 1/2/4 shots, the method is best in 28 of 36 cells (Table 1). Table 2 isolates the mechanism on VisA: image AUROC is 91.4 with a frozen SAE, 92.3 with per-image adaptation and 95.0 with continual adaptation (95.3 for the full model).

Critical strengths

S1. Table 2 separates persistence from per-image adaptation, which the methods the paper compares against do not do. INP-Former and FastRef adapt to each test image independently. Here the variants run under the same protocol (91.4 frozen, 92.3 reset per image, 95.0 continual on VisA), so the 2.7-point gap between the last two is aggregate evidence that carrying normality forward helps. On MPDD the image-level AUROC of the continual variant is 1.3 points lower, but the experiment is the right one to run.

Critical weaknesses

W1. Table 1 is not a like-for-like comparison. EvoSAE updates its parameters on the evaluation stream (Sec. 4.3), and the paper does not say whether the baselines were rerun under the same condition. Backbone size is also unclear. The caption says "with DINOv2-G", and Table A11 applies DINOv2-G to AnomalyDINO and SubspaceAD, yet the 1-shot AnomalyDINO entry in Table 1 (96.6) is the number its original paper reports for its default ViT-S backbone (Damm et al., WACV 2025). Sec. 5.1 says only that baseline numbers were taken from the original papers or from rerun official code, and the backbone of the test-time adaptation baselines is not stated. In Table A10, VisA 1-shot image AUROC moves 4.6 points from DINOv2-S to DINOv2-G, while the gap between EvoSAE and the strongest baseline in each cell is at most 2.5 points (median 0.4 across the 36 cells). Missing entries fall mostly on the adaptation baselines: 20 of the 38 "-/-" cells belong to the four test-time adaptation methods.

W2. The claimed mechanism, improvement that accumulates along the stream, is never measured directly. No table or figure reports performance by position in the stream; every number is an aggregate over the full pass. Quantitative adaptation on/off results appear only in Table 2 (2-shot) and Table A11 (4-shot MVTec), and Fig. 3 is qualitative, so the effect is never quantified in the 1-shot setting the paper emphasizes. Where it is measured, the frozen SAE wins some cells. In Table A11 it reaches 98.1 pixel AUROC at 471 ms against 97.8 at 1,009 ms for the full method, and 98.1 would be the best MVTec-AD pixel AUROC in Table 1. In addition, no validation split is defined, and tau = 1.5 is chosen by a sweep scored on MVTec-AD and then reused for VisA and MPDD (App. A.3).

W3. The few-shot streaming setting already has prior work that is neither cited nor compared against. FOADS (Wei et al., 2024, arXiv:2403.18201) starts from a few normal images and keeps updating on an unlabeled test stream without storing past samples. Without such a method in Table 1, it is unclear what the SAE memory adds over existing online few-shot detectors.

Questions for authors

Q1. Could you report image AUROC by position in the stream (first 10%, middle, last 10%), state the streaming order, and show how much the metrics change under shuffling? The MPDD result in Table 2 (81.6 continual vs 82.9 reset) suggests the curve may not be monotone. Does the stream include the anomalous test images? If so, what happens when defective parts arrive in a long consecutive run, as with a machine fault? That case breaks the assumption that test samples are "predominantly normal" (Sec. 1). Eq. 8 weights patches relative to the median error of the same image, so small defects should stay suppressed, but a defect covering a large part of the image shifts that median, and a recurring defect whose error drops with each update (Fig. 3) would gain weight. Sec. 5.4 shows how much such contamination can cost: MPDD image AUROC drops from 82.7 (frozen SAE, Table 2) to 65.5 with lambda = 0 (Table 4).

Q2. Sec. 2 cites no prior use of SAEs for anomaly detection. SPG (arXiv:2604.02871, April 2026) trains an SAE on patch tokens of a frozen foundation model and uses its dictionary for zero-shot anomaly detection on MVTec-AD and VisA. The setting differs, but the idea of an SAE dictionary over frozen patch features is the same. What is new relative to it?

Q3. Could you rerun FOCT, FastRecon, INP-Former and FastRef on the same DINOv2-G features to fill the empty cells of Table 1, and add MuSc as a transductive reference, since Eq. (5) adopts its neighbourhood aggregation?

Q4. How many support draws and seeds are behind Table 1, and what is the standard deviation? Margins of 0.1-0.2 points cannot be interpreted without it.

7 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/4)

We thank Reviewer owoK for your insightful comments and suggestions! We appreciate your positive reviews on the persistent updating and test-time comparision. Below are our point-to-point clarifications to address your concerns. Feel free to discuss with us if you have further questions!

W1(1): EvoSAE updates its parameters on the evaluation stream (Sec. 4.3), and the paper does not say whether the baselines were rerun under the same condition.

  • First, We would like to clarify that continual adaptation to streaming test data is a key motivation and contribution of EvoSAE, and we develop the persistent updating and soft spatial weighting (Sec. 4.1) to this end. Previous methods often overlooked this issue; they neither conducted experiments under this setting nor designed specialized modules to enable the model to possess this continuous learning capability. Therefore, we report their results under the original test setting.
  • Second, We respectfully maintain that it is not our responsibility to redesign existing baselines for a setting they were not originally intended to address. Furthermore, we argue that the baseline model may not be directly applied to these settings because memory leakage is a key challenge; this is why we carefully designed the soft weighting and spatial averaging modules in EvoSAE.

W1(2): Backbone size is also unclear.

We sincerely apologize to the reviewer for the confusion caused by the incorrectly reported baseline results. This was an oversight on our part. Since SubspaceAD is the most closely related and competitive baseline, we followed its experimental configuration, including the DINOv2-G backbone and random seeds. However, we directly adopted the results of AnomalyDINO and other baselines from SubspaceAD without double checking. To address this issue, we have now included the results of AnomalyDINO evaluated with DINOv2-G and corrected the corresponding comparisons. We will update Table 1 accordingly once the remaining baseline evaluations are completed.

SetupMethodMVTec-AD Image AUROC/AUPRMVTec-AD Pixel AUROC/PROVisA Image AUROC/AUPRVisA Pixel AUROC/PRO
1-shotAnomalyDINO98.1/99.296.9/93.992.9/93.897.9/92.9
SubspaceAD98.0/99.097.6/93.793.3/93.298.3/93.4
EvoSAE (Ours)98.4/99.397.2/95.994.8/94.498.9/95.9
2-shotAnomalyDINO98.5/99.397.2/94.493.7/94.498.0/93.3
SubspaceAD98.1/99.197.8/94.094.1/93.798.4/93.5
EvoSAE (Ours)98.7/99.597.5/96.395.3/94.999.0/96.3
4-shotAnomalyDINO98.7/99.597.4/94.894.8/95.298.3/94.4
SubspaceAD98.4/99.398.0/94.294.5/93.898.5/94.1
EvoSAE (Ours)99.1/99.797.8/96.595.6/95.199.1/96.6

W1(3) & Q3: the backbone of the test-time adaptation baselines is not stated.

We thank the reviewer for this valuable suggestion. In our original submission, we reported the results of existing test-time baselines from their respective papers without aligning their backbones with DINOv2-G. To ensure a fair comparison, we have now re-evaluated these baselines using the same DINOv2-G backbone as EvoSAE and report the updated results below. We are unable to reproduce the results of FOCT and FastRef with DINOv2-G, as their implementations have not been publicly released.

SetupMethodMVTec-AD Image AUROC/AUPRMVTec-AD Pixel AUROC/PROVisA Image AUROC/AUPRVisA Pixel AUROC/PRO
1-shotFastRecon88.3/94.295.7/86.474.7/77.194.7/74.3
INP-Former97.8/99.097.1/93.194.1/95.297.2/90.5
FOADS75.7/89.287.0/67.374.2/81.494.8/77.1
EvoSAE (Ours)98.4/99.397.2/95.994.8/94.498.9/95.9
2-shotFastRecon92.2/96.396.7/88.978.9/81.396.5/78.9
INP-Former98.2/99.197.3/93.595.0/95.398.0/92.4
FOADS75.4/89.586.5/67.577.6/84.195.8/80.3
EvoSAE (Ours)98.7/99.597.5/96.395.3/94.999.0/96.3
4-shotFastRecon94.7/97.697.3/91.281.8/83.997.3/82.4
INP-Former98.7/99.397.3/93.695.0/94.897.7/91.1
FOADS80.0/91.786.6/66.678.8/85.194.9/79.0
EvoSAE (Ours)99.1/99.797.8/96.595.6/95.199.1/96.6

W1(4): In Table A10, VisA 1-shot image AUROC moves 4.6 points from DINOv2-S to DINOv2-G, while the gap between EvoSAE and the strongest baseline in each cell is at most 2.5 points (median 0.4 across the 36 cells).

The above corrected results demonstrate that EvoSAE with DINOv2-G consistently achieves the best performance across most evaluation metrics.

W1(5): Missing entries in Table.1

We have included the results of FastRecon and INP-Former in the table provided in our response to W1(3). We are unable to reproduce FOCT and FastRef due to the unavaliable codes.

AuthorsOpenReview ↗

Rebuttal by Authors (2/4)

W2 (1) & Q1(1): No table or figure reports performance by position in the stream.

  • Following your suggestion, we further investigate how EvoSAE's performance evolves with the proportion of processed test samples. Directly evaluating different positions in the stream would introduce bias, as each position corresponds to different test images. To ensure a fair comparison, we perform persistent updating on the first (ρ\rho%) of test samples, freeze the memory thereafter, and evaluate performance on the full test set. We additionally shuffle the test sequence using three random seeds (1, 2, 3) and report the mean and standard deviation to assess sensitivity to test order.

  • The results demonstrate that 1) persistent updating consistently improves performance over the frozen baseline (0%), validating the benefits of test-time adaptation; 2) on MVTec-AD, EvoSAE achieves its best performance with 10%–50% updates, suggesting that the normal memory can be effectively refined using a relatively small number of reliable test samples. Further updates provide diminishing returns as the memory gradually saturates; 3) in contrast, VisA generally benefits from higher updating ratios, potentially due to its greater visual diversity and the need for more samples to capture the underlying normal distribution. Overall, these findings demonstrate that EvoSAE can effectively leverage streaming test data, with the optimal updating ratio depending on dataset characteristics.

DatasetUpdate RatioImage AUROCImage AUPRPixel AUROCPixel PRO
MVTec-AD0%98.1299.2797.9495.83
10%98.57±0.09399.48±0.03298.17±0.03196.18±0.093
50%98.88±0.05299.55±0.01597.90±0.18296.27±0.101
90%98.85±0.06199.52±0.02197.85±0.13496.24±0.099
100%98.83±0.08899.51±0.02797.80±0.19096.29±0.059
VisA0%91.3991.4098.7695.19
10%91.91±0.24792.49±0.14698.87±0.04595.63±0.159
50%93.70±0.31193.49±0.39498.82±0.10495.59±0.350
90%95.10±0.60895.10±1.52599.00±0.02896.31±0.021
100%95.27±0.57994.86±0.50999.03±0.02296.33±0.048

W2 (2): Quantitative adaptation on/off results appear only in Table 2 (2-shot) and Table A11 (4-shot MVTec).

Following your suggestion, we report the ablation results on the 1-shot MVTec dataset. The full EvoSAE achieves the best results in 10 out of 12 metrics, further validating the effectiveness of our persistent updating strategy.

DatasetSetupImage AUROCImage AUPRPixel AUROCPixel PRO
MVTec-ADFrozen SAE97.498.997.895.3
Query-specific TTA (w/ SC)97.899.097.895.5
EvoSAE (Ours)98.499.497.295.9
VisAFrozen SAE89.489.298.494.0
Query-specific TTA (w/ SC)90.590.198.494.6
EvoSAE (Ours)94.894.498.995.9
MPDDFrozen SAE78.277.697.593.6
Query-specific TTA (w/ SC)78.377.997.694.0
EvoSAE (Ours)78.778.498.094.7

W2(3): Where it is measured, the frozen SAE wins some cells in Table. A11

We acknowledge that persistent updating does not guarantee the best performance across all tasks and metrics. However, we emphasize that continual learning from streaming test data is a critical yet largely overlooked challenge in the anomaly detection community, despite its practical importance in real-world industrial scenarios, where data arrive continuously. EvoSAE is specifically designed to address this challenge. Importantly, EvoSAE outperforms the frozen SAE baseline on 3 out of 4 metrics in Table A11 and outperforms the frozen SAE baseline on 9 out of 12 metrics in Table 2. These findings provide strong empirical evidence for the effectiveness of our proposed continual adaptation framework.

W2(4): In addition, no validation split is defined, and tau = 1.5 is chosen by a sweep scored on MVTec-AD and then reused for VisA and MPDD (App. A.3).

τ\tau is the temperature parameter controlling the decay rate of normality weights with respect to reconstruction errors. As discussed in Appendix A.3, τ∈[1.0,2.0]\tau \in [1.0, 2.0] generally yields stable performance. While dataset-specific tuning on a validation set may further improve performance, we simply fix τ=1.5\tau=1.5 across all datasets to ensure consistency and avoid dataset-specific optimization, which is not the focus of this work.

AuthorsOpenReview ↗

Rebuttal by Authors (3/4)

W3: Missing FOADS results.

Following your suggestion, we reproduce the results of FOADS with the DINOv2-G backbone. we will add the comparision and discuss the difference between these two models in our revision.

1) Few-shot Normality Modeling. FOADS models normal features using Neural Gas clustering and covariance estimation, which may be less reliable with extremely limited support samples (1–4 shots). Notably, its original experiments primarily adopt a 10-shot setting.

2) Reliable Online Adaptation. FOADS relies on hard distance-based thresholds to select normal features for updating, which may be sensitive to inaccurate normality estimates under few-shot conditions. In contrast, EvoSAE incorporates soft spatial weighting to leverage local spatial evidence, improving the reliability of normal sample selection.

SetupMethodMVTec-AD Image AUROC/AUPRMVTec-AD Pixel AUROC/PROVisA Image AUROC/AUPRVisA Pixel AUROC/PRO
1-shotFOADS75.7/89.287.0/67.374.2/81.494.8/77.1
EvoSAE (Ours)98.4/99.397.2/95.994.8/94.498.9/95.9
2-shotFOADS75.4/89.586.5/67.577.6/84.195.8/80.3
EvoSAE (Ours)98.7/99.597.5/96.395.3/94.999.0/96.3
4-shotFOADS80.0/91.786.6/66.678.8/85.194.9/79.0
EvoSAE (Ours)99.1/99.797.8/96.595.6/95.199.1/96.6

Q1(2): Does the stream include the anomalous test images?

  • Yes, the test stream contains both normal and anomalous images. This motivates our spatially calibrated soft normality weighting module (Sec. 4.2, Eqs. 6–9), which assigns higher weights to normal patches and suppresses anomalous patches based on reconstruction errors and local spatial evidence.
  • For consecutive anomalous images, Eqs. 6, 8, and 9 are designed to assign near-zero weights to patches with high reconstruction errors, thereby suppressing their gradient contributions during persistent updating and mitigating the risk of memory contamination, even when defective samples arrive consecutively.

Q1(3): Eq. 8 weights patches relative to the median error of the same image, so small defects should stay suppressed, but a defect covering a large part of the image shifts that median.

  • We agree with that the median value ctc_t in Eq. 8 may affect patch-level normality weighting, particularly when anomalies occupy a large image region. To investigate this, we calculate the anomaly area ratio (the proportion of anomalous pixels) and examine the three categories with the largest ratios.
  • The results reveal that EvoSAE struggles to improve localization performance on metal_nut (14.5%) and transistor (12.0%), potentially because large anomalous regions shift the median error and cause some anomalous patches to receive excessively high normality weights.
  • This is an interesting finding. A potential solution is to decrease ctc_t in Eq. 8 when prior knowledge of anomaly characteristics is available. We will discuss this limitation in the revised manuscript and leave its further investigation for future work.

Table: Category-wise comparison between Frozen SAE and EvoSAE on MVTec-AD under different few-shot settings. The percentage in parentheses denotes the anomaly area ratio, calculated as the number of anomalous pixels divided by the total number of pixels.

SetupCategory (Anomaly Ratio)MethodImage-level AUROC/AUPRPixel-level AUROC/PRO
1-shotmetal_nut (14.5%)Frozen SAE100.0/100.098.2/95.8
EvoSAE100.0/100.089.5/91.0
transistor (12.0%)Frozen SAE98.9/98.694.7/82.1
EvoSAE99.9/99.985.4/85.1
tile (9.8%)Frozen SAE100.0/100.098.0/95.3
EvoSAE100.0/100.098.4/96.2
2-shotmetal_nut (14.5%)Frozen SAE100.0/100.098.2/95.7
EvoSAE100.0/100.090.5/91.8
transistor (12.0%)Frozen SAE100.0/100.095.9/87.5
EvoSAE100.0/100.091.7/87.2
tile (9.8%)Frozen SAE100.0/100.098.1/95.2
EvoSAE100.0/100.098.4/95.9
4-shotmetal_nut (14.5%)Frozen SAE100.0/100.098.2/95.4
EvoSAE100.0/100.091.3/92.2
transistor (12.0%)Frozen SAE100.0/100.096.4/89.7
EvoSAE100.0/100.094.1/88.2
tile (9.8%)Frozen SAE100.0/100.097.9/94.9
EvoSAE100.0/100.098.3/95.8
Reviewer_e17AOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

EvoSAE is a few-shot industrial anomaly detection method that extracts multi-layer features with a frozen DINOv2-G encoder and trains a sparse autoencoder on multi-scale patch features from a small set of augmented normal images, and features a test time adaption.

Critical strengths
  1. EvoSAE is SoTA on MVTec-AD and VisA
  2. Persistent, residual-weighted test-time adaptation is a useful design question.
Critical weaknesses
  1. The main difference towards existing works is not fully analyzed or ablated. EvoSAE's persistent updates and soft spatial weighting are distinct from existing methods but the present comparisons do not establish which component produces the claimed benefit.

  2. Lack of variance for few-shot experiment. This might be a new criteria recently, where most research teams have enough resources to evaluate the model many times, especially for few-shot exp.

Questions for authors
  1. What test-image order was used, is that affected performance ? How was that affected performance?
  2. Why was rotation chosen as the primary augmentation for the normal support images? Please report a controlled ablation with and without rotation augmentation.
3 comments
AuthorsOpenReview ↗

Rebuttal by Authors (1/2)

We thank Reviewer e17A for your thoughtful feedback and constructive suggestions! We appreciate your positive reviews on clear motivation and effective designs. Below are our detailed responses to your remaining questions.

W1: The main difference towards existing works is not fully analyzed or ablated.

  • Mathmatically, as you suggested, persistent updates and soft spatial weighting are distinct from existing methods:

    1), Persistent updating enables the detection model to continuously update its normal memory using high-confidence normal test samples. This not only helps mitigate the distribution shift between training and testing data, but also accommodates real-world industrial scenarios where test samples arrive in a continuous stream.
    2), A key challenge in persistent updating is reliably identifying normal test samples, as incorporating anomalous samples may contaminate the normal memory and degrade detection performance. Soft spatial weighting estimates patch-level normality by incorporating local spatial evidence, thereby reducing the influence of isolated residual fluctuations and improving the reliability of samples selected for memory updates.
    3), These two components work together to enable the proposed model to continuously optimize itself during testing and demonstrate strong lifelong learning capabilities.

  • Empirically, we conduct the suggested ablation studies under the 2-shot setting across three benchmarks, where PU and SC denote persistent updating and soft spatial weighting, respectively. Note that SC cannot be evaluated independently, as it is specifically designed to support PU. The results show that 1) both modules contribute to improving the baseline performance, and 2) On MPDD, the frozen baseline achieves slightly higher image-level AUROC and AUPR, whereas EvoSAE achieves the best pixel-level AUROC and PRO, demonstrating the benefits of persistent updating for fine-grained anomaly localization. Importantly, EvoSAE achieves the best results in 9 out of 12 metrics across three benchmarks, demonstrating the overall effectiveness of the proposed components.

DatasetSetupImage AUROCImage AUPRPixel AUROCPixel PRO
MVTec-ADw/o PU; w/o SC98.199.297.995.8
w PU; w/o SC98.799.597.496.1
w PU; w SC (Ours)98.799.597.596.2
VisAw/o PU; w/o SC91.491.498.795.2
w PU; w/o SC95.094.798.996.0
w PU; w SC (Ours)95.394.999.096.3
MPDDw/o PU; w/o SC82.784.697.994.9
w PU; w/o SC81.682.598.195.2
w PU; w SC (Ours)82.583.698.295.4

W2: Lack of variance for few-shot experiment.

Thank you for the valuable suggestion. In our experiments of the submission, we follow previous works and report the mean value of results with 5 different seeds (42,43,44,45,46). We here report the mean and standard deviation value of subspaceAD and our EvoSAE to have a quick comparision. Please refer to our revision for details. From the results, we find that EvoSAE generally exhibits low variance across random seeds while maintaining competitive or superior performance over SubspaceAD, demonstrating its robustness and reproducibility in few-shot anomaly detection.

SetupMethodMVTec-AD Image AUROC/AUPRMVTec-AD Pixel AUROC/PROVisA Image AUROC/AUPRVisA Pixel AUROC/PROMPDD Image AUROC/AUPRMPDD Pixel AUROC/PRO
1-shotSubspaceAD97.1±0.9 / 98.6±0.497.5±0.1 / 94.5±0.293.2±0.8 / 93.0±0.898.2±0.1 / 95.5±0.175.3±0.8 / 74.9±0.597.6±0.2 / 93.9±0.1
EvoSAE (Ours)98.4±0.6 / 99.3±0.397.2±0.2 / 95.9±0.194.8±0.4 / 94.4±0.398.9±0.0 / 95.9±0.178.7±0.6 / 78.4±0.498.0±0.2 / 94.7±0.1
2-shotSubspaceAD97.5±0.7 / 98.8±0.497.8±0.1 / 94.9±0.193.8±0.4 / 93.4±0.798.3±0.0 / 95.7±0.181.3±0.5 / 82.4±0.298.0±0.1 / 95.1±0.1
EvoSAE (Ours)98.7±0.4 / 99.5±0.297.5±0.1 / 96.3±0.195.3±0.3 / 94.9±0.499.0±0.0 / 96.3±0.182.5±0.4 / 83.6±0.298.2±0.1 / 95.4±0.1
AuthorsOpenReview ↗

Rebuttal by Authors (2/2)

Q1:What test-image order was used.

In our experiment, we used the default file name sorting order. Following your suggestion, we conducted experiments by randomizing the test order (setting different seed values, e.g., seed=1, 2, 3) and reported the results. The results show that EvoSAE is robust to test-image ordering, with low standard deviations across shuffled sequences and performance closely matching the original results. This suggests that the effectiveness of persistent updating does not depend strongly on a particular test-image order.

DatasetImage AUROCImage AUPRPixel AUROCPixel PRO
MVTec-ADshuffer98.83±0.08899.51±0.02797.80±0.19096.19±0.059
Reported98.799.597.596.3
VisAshuffer95.25±0.17994.78±0.10999.03±0.02296.32±0.048
Reported95.394.999.096.3

Why was rotation chosen.

  • Following SubspaceAD, we adopt random rotations (0°–345°) to increase the diversity of normal support samples while preserving their appearance characteristics. Following your advice, we further compare rotation with three alternative augmentation strategies: horizontal flipping (p=0.5), vertical flipping (p=0.5), and random affine transformations. The results under the 4-shot setting on MVTec-AD are presented below.

  • The results show that 1) random rotation consistently outperforms the alternative augmentations for both methods, validating its effectiveness in enriching normal support representations; and 2) under random rotation, EvoSAE outperforms SubspaceAD on three of four metrics.

MethodAugmentationImage AUROCImage AUPRPixel AUROCPRO
SubspaceADHorizontal flip96.898.097.293.4
SubspaceADVertical flip96.397.997.293.2
SubspaceADRandom affine96.397.597.493.0
SubspaceADRandom rotation98.499.398.094.2
EvoSAEHorizontal flip96.397.897.295.1
EvoSAEVertical flip96.797.897.395.4
EvoSAERandom affine96.397.797.595.4
EvoSAERandom rotation99.199.797.896.5
AuthorsOpenReview ↗

Have Our Initial Responses Addressed Your Concerns?

Dear Reviewer e17A,

We sincerely thank you for your time and constructive suggestions, which have helped us improve the quality of our submission.

Following your suggestions, we have conducted additional experiments and provided detailed responses to your comments. We hope that these new results and clarifications have adequately addressed your concerns. We have also uploaded a revised manuscript that addresses typographical errors, inconsistent mathematical notation, formatting issues, and unclear descriptions. All revisions are highlighted in blue to facilitate your review.

As the rebuttal period is approaching its deadline, we would greatly appreciate any further feedback on whether our responses have resolved your concerns. Please do not hesitate to let us know if any questions or concerns remain.

Thank you again for your valuable time and consideration.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.