acceptodds
Withdrawn from ICLR 2027

AgenticAD: Learning to Route, Observe, and Detect for Sample-Adaptive Industrial Anomaly Detection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-class industrial anomaly detection faces strong sample-wise variation in detector preference, defect scale, and appearance. We introduce AgenticAD, a sample-adaptive route–observe–detect framework for supervised control of frozen anomaly detectors. Supervision is used only to train the lightweight routing and observation controllers (using pixel masks for pixel-level control); the anomaly detectors remain frozen. Stage I routes each image to one of four complementary experts using task-specific utilities derived from all expert outcomes. Stage II performs a bounded observe–detect loop conditioned on the initial score, heatmap, and observation history. At each step, the controller either stops or applies a heatmap-guided crop or photometric transformation; a generic local detector analyzes the resulting view and aligns its evidence to the original image. Fixed aggregation rules convert detector outputs into final scores. On IND-COM, comprising five industrial datasets and 78 object categories, Stage I improves image-level AUROC/AP from for category-best routing to , and Stage II further reaches . For pixel localization, Stage I reaches AUROC/AP, and Stage II reaches , compared with for category-best routing. Overall, AgenticAD offers a practical way to adapt detector selection and inspection effort to the diversity of real-world industrial products and anomalies.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Industrial Anomaly Detection, Agentic Expert Routing, Sample-Adaptive Routing, Multi-Step Visual Reasoning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_WNnSOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes AGENTICAD, a two-stage framework for sample-adaptive industrial anomaly detection. Stage I learns a lightweight controller to select one of four frozen anomaly detection experts based on image features. Stage II employs another controller to adaptively select image transformations or stop further inspection, using AnomalyCLIP as a frozen local detector. Both controllers are trained with full-information supervision derived from labeled anomaly data.

The authors construct IND-COM by combining five existing industrial anomaly detection datasets and adopting a supervised-controller training protocol. The method achieves 91.34% image-level AUROC and 92.45% pixel-level AUROC. While the results demonstrate the potential of adaptive inspection, I have concerns about the supervision-matched comparisons, the contributions of individual components, and the reliability of certain baseline results.

Critical strengths

The authors provide an insightful observation that different anomaly detection experts exhibit distinct strengths across anomaly types, even within the same object category, as illustrated in Figure 1(A).

Critical weaknesses
  1. Limited supervision-matched comparisons and insufficient evidence of the advantage over simpler supervised alternatives.

A major concern is the experimental protocol. AGENTICAD trains its controllers using 7,316 labeled images, including 5,299 anomalous images, and additionally uses pixel-level annotations for localization. In contrast, conventional UAD baselines are trained using only normal images. Although the authors acknowledge this difference, comparisons against unsupervised methods cannot adequately establish the advantage of the proposed framework under comparable supervision.

Moreover, the improvement of Stage I over strong unsupervised baselines appears relatively modest on MVTec AD. As shown in Table 2, Stage I achieves 94.56% image-level AUROC, only 0.90 percentage points higher than Dinomaly (93.66%), while remaining below Dinomaly2 (97.20%). Even the complete Stage II framework (96.35%) does not outperform Dinomaly2 on this dataset. These results raise questions about whether the performance gains justify the substantial additional labeled training data required by the proposed framework.

Since substantial anomaly supervision is already available, comparisons with lightweight supervised classifiers or segmentation models trained using the same annotations would be important for demonstrating the practical value of the proposed control-based approach.

  1. Incomplete component analysis and concerns about baseline reliability.

Although Table 5 includes fixed-observation baselines, the contribution of Stage I routing to the complete system remains insufficiently isolated. In particular, it is unclear whether applying the same Stage II observation controller directly to a strong individual detector, such as Dinomaly, could achieve comparable performance. The generality of Stage II is also unclear because AnomalyCLIP is the only generic local detector evaluated. Furthermore, Tables 2 and 3 do not report the standalone performance of AnomalyCLIP.

I am also concerned about the unexpectedly low results of DeCo-Diff. Table 2 reports only 44.71% image-level AUROC on MVTec AD, while the official DeCo-Diff repository reports approximately 99.3% under its original evaluation setting. Although the evaluation protocols differ, such a substantial discrepancy requires careful verification of the implementation, anomaly-score direction, preprocessing, and evaluation procedure.

Questions for authors

Q1. Supervision-matched comparisons. Given that AGENTICAD uses thousands of labeled anomalous images and pixel-level masks, could the authors compare against simple supervised classification and segmentation baselines trained using the same annotation budget? How much of the improvement is attributable to the proposed adaptive control mechanism rather than access to additional anomaly supervision?

Q2. Contribution of Stage I and generality of Stage II. What performance would be achieved by applying the same Stage II training and inference strategy directly to Dinomaly, without Stage I routing? Could the authors also evaluate Stage II using another frozen local detector instead of AnomalyCLIP? Reporting the standalone image-level and pixel-level performance of AnomalyCLIP would further clarify its contribution.

Q3. DeCo-Diff baseline. Could the authors explain the very low DeCo-Diff performance in Tables 2 and 3, especially on MVTec AD? The reported values differ substantially from those in the original work and official implementation. Please clarify the implementation, checkpoint, preprocessing, anomaly-score direction, and evaluation settings.

Reviewer_QDgiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes AGENTIC AD, a framework for industrial anomaly detection that combines sample-adaptive expert routing with adaptive visual observation. It first selects one anomaly detection model for each input image, then decides whether to perform additional observations, such as cropping or adjusting brightness and contrast. The framework uses lightweight controllers while keeping the detection models frozen. The results show clear improvements in image-level anomaly detection, while the gains in pixel-level localization are relatively small.

Critical strengths
  1. Interesting framework. The combination of expert routing and adaptive observation provides a flexible way to use existing anomaly detection models.

  2. Good Performance. The proposed method achieves clear improvements in image-level AUROC and AP on the evaluated benchmark.

  3. Easy to follow.

Critical weaknesses
  1. Fairness of Comparisons. Most individual anomaly detection baselines are trained using only normal images, while the proposed controllers are trained with image-level labels and pixel-level annotations. This difference in supervision makes it difficult to determine whether the performance improvements come from the proposed framework or the additional supervision. Comparisons under similar supervision settings would provide a fairer evaluation of the proposed method.

  2. Limited Efficiency Evaluation. The proposed framework requires training two controllers and may perform multiple additional inference calls through AnomalyCLIP. However, the computational overhead is not sufficiently evaluated in terms of training time, inference latency, memory usage, and throughput. Although the average number of detector calls is reported, a more comprehensive efficiency analysis is needed to demonstrate the practical benefits of the framework.

  3. Missing Simple Multi-Expert Aggregation Baselines. The proposed method achieves notable improvements in image-level anomaly detection, but it remains unclear whether similar improvements could be obtained through simple aggregation of multiple experts. Comparisons with conventional score averaging or other simple ensemble strategies, considering both detection performance and inference cost, would help justify the necessity of the learned expert routing mechanism.

  4. Insufficient Analysis of the Adaptive Observation Strategy. The proposed Observe–Detect Loop relies on a limited set of predefined image transformations, but it remains unclear whether its improvements primarily come from adaptive action selection or simply from additional detector calls. Comparisons with fixed observation strategies, random action selection, and rule-based policies under the same inference budget would better demonstrate the effectiveness of the learned controller.

  5. Limited Justification for Multi-Step Decision Learning. Although the proposed Observe–Detect Loop performs sequential decisions, the action controller is trained using utilities computed from the immediate outcome of each action. This raises the question of whether such single-step supervision can effectively optimize multi-step observation strategies, especially when the benefits of an action may only become apparent after subsequent observations. Further analysis would help justify the proposed training strategy.

Questions for authors
  1. Fair Comparison. How much of the performance improvement comes from the proposed framework rather than the additional supervision? Comparisons with supervised anomaly detection methods or under matched supervision settings would help clarify this issue.
  2. Computational Efficiency. What is the end-to-end computational cost of AgenticAD in terms of inference latency, throughput, memory usage, and training time? How does its accuracy-efficiency trade-off compare with conventional anomaly detectors?
  3. Multi-Expert Aggregation. How does the proposed routing mechanism compare with simple multi-expert aggregation strategies, such as score averaging or rank-based fusion? Can these simpler approaches achieve comparable image-level detection performance, and what advantages does adaptive routing offer?
  4. Adaptive Observation. How much of the improvement in Stage II comes from learned action selection rather than additional detector calls? Does the learned policy outperform fixed or rule-based observation strategies under the same inference budget?
  5. Multi-Step Decision Learning. Since the action controller is trained using the immediate utility of each action, how does the proposed objective account for actions whose benefits may only become apparent after subsequent observations?
Reviewer_vAzvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AgenticAD learns which anomaly detector to use for each image and whether further inspection is needed. It selects crops or brightness and contrast adjustments while keeping the detectors fixed. Experiments on five industrial datasets show larger improvements in image-level detection than in localization.

Critical strengths
  1. The method makes use of several existing detectors while running only one initially. Keeping the detectors fixed also limits training to small controllers.
  2. The image-level gains are substantial. Category-averaged AUROC improves from 86.78 for category-best routing to 91.58.
Critical weaknesses
  1. Some numbers do not match the evaluation described in the paper. EfficientAD’s overall pixel results in Table 3 cannot be obtained by averaging its subset results as described in Section 4.1. In Table 6, the normal and anomalous groups imply 0.858 calls per image, but the reported overall average is 0.969.
  2. The paper lacks a comparison with a standard classifier trained on the same labeled data. A comparison with SegAD [1], which learns to combine detector outputs, is also missing. These comparisons would help show what routing adds.
  3. It is unclear how much the extra inspection steps help. A comparison with a controller trained to stop after one extra inspection is missing. It also does not test whether using results from earlier steps improves performance.

[1] Aimira Baitieva, David Hurych, Victor Besnier, Olivier Bernard. Supervised Anomaly Detection for Complex Industrial Images. CVPR, 2024.

Questions for authors
  1. How were training examples created for the second and third inspection steps?
  2. What are the inference time and peak memory, including feature extraction?
Reviewer_AahoOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes AgenticAD, a controller framework on top of frozen anomaly detectors for multi-class industrial anomaly detection. Its premise is that both the best detector and the appropriate way of observing an image vary from sample to sample. Stage I uses a small MLP on frozen Qwen2.5-VL-3B visual features to route each image to one of four frozen experts (Dinomaly, EfficientAD, PatchCore, UniAD). Stage II is a second MLP that, for at most three steps, either stops or applies a heatmap-centred crop or a brightness/contrast adjustment. Each new view is scored by a frozen AnomalyCLIP, and the results are fused with the Stage I output by fixed averaging. Both controllers are trained with "full-information supervision": soft targets are built from all counterfactual outcomes computed offline. The controllers are trained on a labeled portion of the official test pools of five industrial datasets (78 categories) and evaluated on the remaining portion. The paper reports improvements over individual experts and over category-level expert selection at both image and pixel level.

Critical strengths

S1. A well-motivated question. Figure 1 shows that the best expert changes across defect types, even within a category. The per-sample oracle far exceeds any single expert. This makes "how to exploit complementary detectors" a meaningful question for multi-class inspection.

S2. Lightweight training on top of frozen detectors. The only trainable components are two small MLP controllers. All anomaly detectors, the visual encoder and AnomalyCLIP stay frozen. The framework can therefore reuse existing detectors without retraining or modifying them, and the training cost is low. This is attractive for practical deployment, where detectors are often already in place and expensive to retrain.

Critical weaknesses

W1. The learned router is not shown to be better than a simple category lookup, and it is never tested where a lookup would fail. In industrial inspection the product category is almost always known at deployment. The natural alternative to a learned router is therefore a category-to-expert lookup table, which the paper includes as "category-best". Comparing the two fairly requires per-category metrics, since pooling scores across 78 categories mixes in cross-category effects. Under this comparison (category-averaged results, Table 7), Stage I improves over category-best by only about 1 AUROC point (87.83 vs 86.78), and it is worse on three of the five datasets. On seen categories, then, the router offers little over a lookup table. The router's distinctive value would have to come from generalization: new categories with no lookup history, or new defect types within a known category. This is exactly what the experiments do not test. The controllers are trained and evaluated on the same categories and the same defect types. There is also a tension with the paper's premise. The method needs a large amount of labeled anomalous data covering the test defect types. Once such data are available, the problem is no longer unsupervised AD. The paper should explain why routing among unsupervised detectors is preferable to using these labels directly.

W2. The role of Stage II is unclear; its stated mechanism is not supported. Stage II provides most of the reported image-level gain (84.75 to 91.34 AUROC), so it carries the paper's main result. The stated mechanism (Fig. 1C) is that small or low-contrast defects need a crop or photometric adjustment to be seen. If that were what Stage II does, localization should improve clearly. In fact, the pixel-level gain is only +0.37 AUROC. The ablation (Table 5) shows only that learned action selection beats fixed actions. It does not show that the gain comes from better observation. The final score is an average of the expert score and AnomalyCLIP scores, and the controller sees image features from which the label may be predictable. Stage II can therefore act as a learned way of combining scores, which conflicts with the claim that no learned fusion is used. To support the claimed mechanism, the paper needs, for example: the gain broken down by defect size; Stage II without the visual feature in its state; and a comparison with a learned score combiner over the same views.

W3. Other details.

  • Several results are internally inconsistent. In Table 3, random routing outperforms every individual expert under a per-image-averaged metric, which should not be possible. In Table 5, "Random Obs." and "Fixed Bright. down" report identical numbers. The numbers in Figure 1B do not match Table 2.
  • The choice of Qwen2.5-VL-3B as the router's feature extractor is not motivated, and no alternative backbone (e.g., CLIP, DINOv2, or the experts' own features) is compared. A 3B vision-language encoder is a heavy component for a "lightweight" router, and its effect on routing quality is unknown.
Questions for authors
  1. Generalization of the router (decisive). Please evaluate under leave-one-dataset-out, leave-category-out, or leave-defect-type-out settings. Does the router beat a category lookup where the lookup is unavailable or wrong?

  2. Router vs. lookup on seen categories. Given that the category is known at deployment, what practical benefit does the router offer over category-best on seen categories?

  3. What Stage II actually does. Please report the gain broken down by defect size, Stage II without the visual feature in its state, and a learned score combiner over the same candidate views. If the combiner matches Stage II, in what sense is the observe-detect loop the source of improvement?

  4. Details. Please explain the inconsistencies in Tables 3 and 5 and Figure 1B, and justify the choice of Qwen2.5-VL-3B as the feature extractor.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.