- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper compares 30 industrial anomaly detection configurations across multi-class, few-shot, and zero-shot settings on MVTec-AD and VisA. It reports detection and localization scores alongside throughput, GPU memory, parameter counts, and FLOPs. It also examines one-shot ranking stability and the cost of retaining reference features. The goal is to provide a more useful basis for selecting detectors under deployment constraints.
Critical strengths
- The comparisons have practical value and provide useful information for deployment. Methods with similar detection performance can differ substantially in inference speed and GPU memory usage, which can affect the choice of method in practice.
- The paper provides concrete evidence of the distinction between detection and localization.
- The few-shot comparison is reasonably designed. All methods use the same support sets across the five one-shot runs, making small performance differences between methods easier to interpret.
Critical weaknesses
- The efficiency measurements are not documented in enough detail to assess their reproducibility and comparability. The timer boundaries, GPU synchronization, numerical precision, and number of timed repetitions remain unclear. Since the paper’s main contribution relies on comparing deployment costs, these details are essential for interpreting the reported differences.
- Although the paper broadens the coverage of methods and adds paired few-shot experiments, both of which are useful, much of the analysis still focuses on describing differences between the selected configurations. The additional contribution over existing benchmarks[1] therefore needs clearer justification. Grouping methods by how they compute anomaly scores is mainly descriptive. Measurements showing how different scoring procedures affect computational cost would make its value more convincing.
- The deployment and budget conclusion for PromptAD is not fully supported by the reported timing. The paper excludes PromptAD from throughput comparisons because its timing omits post-processing, yet Section 4.4 uses it as an example satisfying the 1,000 MB and 50 images/s budget. It is therefore unclear whether the complete pipeline meets this requirement.
[1] Xie et al. IM-IAD: Industrial Image Anomaly Detection Benchmark in Manufacturing. IEEE Transactions on Cybernetics, 2024.
Questions for authors
- Where does the timer start and stop for each method? Are reference retrieval, CPU–GPU transfers, and post-processing included consistently? Please also clarify how GPU synchronization and repeated timing measurements are handled.
- Does PromptAD still meet the 50 images/s threshold in Section 4.4 when post-processing is included? If complete timing is unavailable, could the authors revise the deployment-budget example?
- For MetaUAS, can the authors confirm whether the reported VisA accuracy was measured at the same input resolution as the throughput? If not, which accuracy result should be paired with the reported efficiency?
- Could the authors provide the five paired run-level differences behind the ranking reversals in Section 4.3?