EvoSAE: Evolving Sparse Autoencoder for Few-Shot Industrial Anomaly Detection

Few-shot anomaly detection (FSAD) aims to identify abnormal samples from only a limited number of normal examples, gaining increasing attention due to its effectiveness and flexibility for new industrial scenarios. However, it is usually hard for existing FSADs to discover sufficient knowledge from scarce reference samples. Moreover, most approaches focus on feature-level representation while overlooking the polysemanticity of neurons in foundation models. In the paper, we introduce EvoSAE, a self-improving FSAD framework built upon sparse autoencoders (SAE). Moving beyond feature-level memory construction, EvoSAE establishes an atom-level memory by disentangling features as sparse overcomplete directions. These semantic atoms provide more fine-grained representations of normal patterns and enhance the compositional capability for modeling unseen features. Importantly, we softly weight test patches according to their reconstruction errors, assigning larger weights to patches that are more likely to be normal. The weighted patches are then used to fine-tune the SAE during inference, enabling its semantic atoms to progressively adapt to the test distribution. Extensive experiments on three industrial anomaly detection benchmarks demonstrate that EvoSAE consistently outperforms recent state-of-the-art FSAD methods across different shot settings. Visualizations of the learned atoms further reveal their semantic interpretability in real-world anomaly detection scenarios.