acceptodds
Under review as a conference paper at ICLR 2027

ADPretrain++: Dense Dual-Task Anomaly Representation Pretraining for Industrial Anomaly Detection

Abstract

Unsupervised industrial anomaly detectors such as PatchCore, PaDiM and CFLOW score images and pixels with frozen features from generic backbones, which were never trained to separate normal from anomalous patterns. Residual anomaly-representation pretraining narrows this mismatch, but it optimises a single global objective on one feature space, even though image-level detection and pixel-level localisation require different evidence. We propose ADPretrain++, which pretrains residual features on an industrial dataset with real defect masks through a shared learnable-reference projector followed by an image adapter and a pixel adapter. The pixel adapter is trained with a dense norm-map loss and prototype shaping, and the image adapter with top- multiple-instance learning and descriptor shaping. The pretrained features replace the backbone features of an unchanged detector. With a DINOv3-L backbone and averaged over four benchmarks, the features raise PatchCore by 4.8 points of image-level area under the ROC curve (I-AUROC) and 7.3 points of area under the per-region overlap curve (AUPRO), raise PaDiM by 1.1 and 1.6 points, and leave CFLOW essentially unchanged. Ablations attribute the gains to the dual-task objectives and the shared projector, and evaluation-time routing with PatchCore shows that the image adapter serves image scores while the pixel adapter serves localisation maps.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Industrial anomaly detection, Anomaly representation pretraining, Feature adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7mCaOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces ADPretrain++, an improved method of ADPretrain, aimed at providing specialized pre-trained representations for the anomaly detection field. Compared to ADPretrain, ADPretrain++ mainly makes two improvements: Firstly, it sets the encoder to be learnable, offering a higher parameter capacity compared to the fixed encoder in ADPretrain. Secondly, it introduces two additional branches, namely the image adapter and the pixel adapter. These two branches are specifically designed for image-level detection and pixel-level localization, respectively, achieving superior performance compared to the shared projector for their respective tasks. The pixel adapter is trained with a dense norm-map loss and prototype shaping, the image adapter incorporates top-k multi-instance learning and descriptor shaping. Experimental results on multiple downstream datasets and anomaly detection methods demonstrate the improvement of pre-trained representations obtained by ADPretrain++ over ADPretrain.

Critical strengths
  1. The problem addressed in this paper is of great value: constructing dedicated pre-trained representations for the anomaly detection field, thereby empowering all anomaly detection methods in the entire field. ADPretrain++ further demonstrates that residual features can serve as the fundamental representation in the anomaly detection field.

  2. The pre-trained features constructed by ADPretrain++ have achieved consistent improvements over ADPretrain on multiple downstream datasets and downstream anomaly detection methods, further advancing the progress in the direction of anomaly representation pre-training.

Critical weaknesses
  1. Is Lora used in every layer of the encoder, or is it only applied in the layers that output features? In line 151, it mentions "the last four blocks". If it refers to the last four blocks of the entire encoder, this would mean that the features output from the previous three levels are still based on a fixed encoder without Lora.

  2. Setting the encoder to learnable or using LoRa in ADPretrain can lead to training failures. However, using LoRa in ADPretrain++ achieved good results. How was this achieved? Is there a specific method to avoid training failures? Have you ever tried setting the entire encoder to learnable? What was the effect?

  3. Compared to ADPretrain, ADPretrain++ introduces an additional 4 losses, making a total of 6 losses. There are too many losses, causing the method complex, is it possible to simplify them? In Table 2, some losses actually do not bring much effect, and could be considered for removal. The problem with having 6 losses is that the weighting factor λ\lambda for each loss cannot be adjusted at all, requiring extensive hyperparameter experiments. However, the paper does not provide any hyperparameter experiments for the weight parameters of the losses.

  4. Although the paper emphasizes that separating the image and pixel paths is a more optimal approach, in practice, only the representations from these two paths are used simultaneously in PatchCore, while only the pixel path is employed in other methods. Therefore, the experimental support for the core idea in the paper is insufficient. Additionally, if image-level detection and pixel-level localization utilize two different feature paths, do these downstream AD methods need to model and train twice with different features? In this case, the two feature paths actually have little significance.

  5. The experimental verification using only DINO-v3-L as a backbone is insufficient. In ADPretrain, verification was conducted on a total of 5 backbones. As a pre-training representation task, the core value is to provide better pre-training representations. Therefore, pre-training is necessary on all current mainstream backbones used for anomaly detection to provide good representations. Additionally, the current input resolution is 224x224, but some of the latest AD methods use higher resolutions such as 448x448. Therefore, it is best to provide a larger model than DINO-v3-L and pre-training representations at a higher resolution.

  6. In Table 2, the effect of using only one route is already quite good. So what is the significance of using two routes? AUPRO even decreases.

Questions for authors

The questions have been written in the Weaknesses part.

Reviewer_SJGvOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presents ADPretrain++, an extension of ADPretrain for industrial anomaly detection. The method aims to improve the transferability of pretrained anomaly representations by introducing task-specific supervision for image-level anomaly detection and pixel-level anomaly localization. It combines normal-reference residual features, a shared learnable-reference projector, and two task-specific adapters. The pixel branch is optimized using dense mask supervision and prototype-based objectives, while the image branch employs top-k multiple-instance learning and descriptor-level objectives.

The method is pretrained on Real-IAD and evaluated as a feature replacement for five existing anomaly detectors across five industrial anomaly detection benchmarks. The experiments demonstrate improvements over generic backbone features and the original ADPretrain, particularly for PatchCore. Additional ablations and feature-routing experiments investigate the contributions of individual components and objectives.

Critical strengths

The paper evaluates the pretrained representations across five industrial benchmarks and five downstream anomaly detectors, while keeping the downstream scoring mechanisms unchanged. This is a meaningful experimental setting for studying transferable feature quality. The improvements, particularly for PatchCore, suggest that task-oriented anomaly pretraining can benefit existing detection pipelines without requiring modifications to their scoring algorithms.

The objective ablations and evaluation-time routing experiments provide interesting observations about the different roles of image-level and pixel-level supervision. The variation in gains across downstream detectors is also valuable, as it suggests that representation improvements interact with the detector's scoring mechanism. These findings provide a potentially useful basis for understanding anomaly representation transfer.

Critical weaknesses
  1. The central motivation is insufficiently established, and the claimed contribution is not fully supported by the analysis.

  2. The method appears primarily incremental, with insufficient analysis to establish a broader methodological contribution.

Questions for authors
  1. What concrete evidence supports the claim that image-level detection and pixel-level localization require separate feature representations?

The paper identifies the use of a single global objective as a major limitation of ADPretrain. However, Table 2 shows that a shared representation trained with the proposed objectives performs comparably to the full dual-route architecture. Could the authors clarify the specific limitation that feature separation addresses, and provide evidence of task interference or incompatible optimization requirements within the shared representation?

  1. How should the relative contributions of the task-specific objectives and the dual-route architecture be interpreted?

The ablation results suggest that the additional objectives account for most of the improvements, whereas the architectural separation provides relatively limited or mixed benefits. Could the authors clarify which component constitutes the central contribution and explain why the dual-route design is preferable to a shared representation trained with the same objectives?

  1. What determines whether a downstream detector benefits from the proposed representation pretraining?

The improvements differ substantially between PatchCore and CFLOW. The paper suggests that this may relate to how strongly a detector relies on the geometry of its input feature space, but the current evidence does not directly establish this explanation. Could the authors provide more targeted analysis to support this interpretation? Clarifying this relationship would considerably strengthen the general significance of the work.

Reviewer_71K2OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ADPretrain++, a dual-task representation pretraining framework for industrial anomaly detection. Building upon ADPretrain, the method introduces a learnable-reference projector and two task-specific adapters to separately optimize image-level anomaly detection and pixel-level anomaly localization. Experiments on five benchmarks demonstrate that the proposed features can improve the performance of several existing anomaly detectors.

Critical strengths
  1. Well-motivated approach. The paper identifies the different requirements of image-level detection and pixel-level localization, providing a reasonable motivation for task-specific feature learning.
  2. Good applicability. The proposed pretrained features can be integrated into various existing anomaly detectors without modifying their architectures or detection mechanisms.
  3. Comprehensive experiments. The authors evaluate the method on five benchmarks and five anomaly detectors, including ablation studies, hyperparameter sensitivity analysis, and cross-backbone comparisons.
Critical weaknesses
  1. Insufficient justification for the dual-branch design. The ablation results show that a single shared branch achieves comparable or even slightly better overall performance than the proposed dual-branch architecture, raising questions about the necessity of separate task-specific adapters.
  2. Inconsistent improvements across detectors. While the method achieves notable improvements with PatchCore, the gains on CFLOW are marginal, and performance decreases are observed in some cases. This suggests that its effectiveness may depend on the downstream detector.
  3. Limited analysis of pretraining data dependency. Although the method demonstrates cross-dataset generalization, all experiments rely on Real-IAD as the pretraining source. It would be valuable to investigate whether similar improvements can be achieved using alternative pretraining datasets or with fewer pixel-level anomaly annotations.
  4. Insufficient experimental analysis. The experiments are conducted with only one random seed per configuration, without reporting statistical variations. Additionally, qualitative visualizations of anomaly localization results would help better understand the strengths and failure cases of the proposed method.
Questions for authors
  1. Could you further clarify under what conditions the dual-branch design is actually needed? Also, would its advantages become more evident on larger-scale or more challenging datasets?
  2. Could you provide a more detailed analysis explaining the reasons behind the performance differences across detectors?
  3. How ADPretrain++ performance degrades when gradually reducing the amount or density of pixel-level anomaly annotations.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.