- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates whether in-domain training is sufficient for fine-grained multimodal industrial anomaly understanding (MM-IAU). The authors argue that supervised specialization of a single MLLM improves certain semantic capabilities but does not solve shared fine-grained perceptual weaknesses. They analyze MMAD and show that trained specialists still lag behind human experts in defect localization and anomaly detection, motivating a division-of-labor strategy. The authors propose SiGMA, a training-free spatially grounded multi-agent framework that combines heterogeneous frozen MLLM agents with a reference-based defect expert. The framework includes a multimodal searcher for retrieving domain knowledge and normal references, a defect expert for producing calibrated anomaly evidence, and a reliability controller for adaptive aggregation based on source competence and query-level evidence quality. Experiments on MMAD show that SiGMA achieves 85.2% average accuracy, outperforming existing trained specialists and large-scale MLLMs.
Critical strengths
Instead of only proposing another model, the paper identifies an important bottleneck: fine-grained industrial perception remains weak even after domain training. The analysis showing trained methods still underperform base models on anomaly detection is interesting and potentially valuable.
The proposed combination of heterogeneous MLLMs with a dedicated defect expert is conceptually reasonable. Ablations show that the defect expert mainly contributes to anomaly detection and localization, while MLLM diversity improves semantic tasks.
Critical weaknesses
-
The paper argues that in-domain training is insufficient for fine-grained industrial anomaly understanding, but the conclusion is drawn from one benchmark. A stronger claim requires broader experimental evidence.
-
Although the authors report that a three-agent system reaches 84.4% accuracy, SiGMA still requires multiple MLLM calls and external visual expert processing. 57640_Is_In_Domain_Training_En The practical advantage over a carefully designed single-model system is not fully established.
Questions for authors
How does SiGMA perform when the reliability controller cannot access the entire unlabeled test batch and must operate in an online streaming scenario?
Have the authors evaluated whether the estimated source reliability transfers across different industrial domains or product categories?
How sensitive is the performance to the choice of the defect expert (e.g., DINOv3/RAD)? Would replacing it with another anomaly detector substantially change the conclusions?
Compared with simply using a strong vision anomaly detector plus an LLM-based explanation module, what is the unique advantage of the proposed multi-agent aggregation strategy?
5 comments
Response to Reviewer uBwh (2/4)
W2: The practical advantage over a carefully designed single-model system is not fully established.
A: We thank the reviewer for this fair comment. We provide evidence that SiGMA outperforms carefully designed single-model systems in two aspects, namely at a matched model size and in generalization. First, at a similar total scale, single large MLLMs of 72B/76B given domain knowledge still fall at least 5.0 below SiGMA on MMAD. Second, the strongest trained specialist, JUDO [3], given exactly the same retrieved context and defect-expert evidence as SiGMA, still falls 2.9 below SiGMA on MMAD. Moreover, its training does not generalize, as on the unseen MANTA [1] it falls below its own untrained base model and 6.3 below SiGMA.
(a) Single models of similar scale. The 9 checkpoints of SiGMA total about 70B parameters. We therefore compare with single large MLLMs of similar size on MMAD, each given domain knowledge.
| Model | Params | Avg.(%) | AD | Cls. | Loc. |
|---|---|---|---|---|---|
| InternVL2-Llama3-76B | 76B | 76.2 | 66.7 | 71.0 | 60.6 |
| Qwen2.5-VL-72B-Instruct | 72B | 80.2 | 71.6 | 73.9 | 69.1 |
| SiGMA | ~70B (9 × ≤9B) | 85.2 | 82.6 | 79.2 | 79.6 |
As shown in the table, at a similar total size SiGMA outperforms the strongest 72B model by 5.0 on average, by 11.0 in anomaly detection and by 10.5 in defect localization. The advantage therefore does not come from more parameters, but from dividing the labor among heterogeneous agents and a defect expert.
(b) A carefully designed trained specialist. We gave JUDO, the strongest trained specialist in Table 1, exactly the information SiGMA uses: first the same retrieved context, and then additionally the same defect-expert overlay and spatial hint.
| Setting | MMAD avg. (official, %) | MANTA item acc. (%) |
|---|---|---|
| Qwen2.5-VL-7B (JUDO's base model), same retrieved context | 76.9 | 73.1 |
| JUDO, same retrieved context | 81.8 | 70.0 |
| JUDO + same defect-expert evidence | 82.3 | 71.8 |
| SiGMA | 85.2 | 78.1 |
As shown in the table, the same defect evidence improves JUDO by only 0.5, and SiGMA still outperforms it by 2.9 on MMAD. On the unseen MANTA, the gap widens to 6.3, and JUDO falls 3.1 below its own untrained base model, so the benefit of its in-domain training does not transfer (W1). We will add these comparisons to the CR.
[1] MANTA: A Large-Scale Multi-View and Visual-Text Anomaly Detection Dataset for Tiny Objects. CVPR 2025.
[2] SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment. ACM MM 2025.
[3] JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA. ICLR 2026.
Response to Reviewer uBwh (3/4)
Q1: How does SiGMA perform when the controller cannot access the full unlabeled test batch and must operate online?
A: We thank the reviewer for this practical question. SiGMA operates effectively online without access to the full test batch. Its controller uses unlabeled test data only to estimate the task-wise reliability of each source and the distribution of anomaly-map strength, and both can equally be estimated from previously seen images. Specifically, when MMAD is replayed as an online stream, with each image answered using statistics from the most recent 512 images and re-estimated every 256 images, accuracy decreases by only 0.2 compared with the full batch, averaged over 10 arrival orders. When the statistics are estimated once from the first 512 images and then frozen, the decrease is only 0.1.
What the controller estimates. Without any label, the controller estimates two statistics from unlabeled test data: (i) the task-wise reliability of each source, and (ii) the distribution of anomaly-map strength, which turns each map into its evidence score t (the within-task percentile). In the paper, both are estimated from the full test batch.
Online protocol. We replayed MMAD as a stream of images, using the frozen model outputs. When an image arrives, all its questions are answered with statistics from previous images only, and only then is the image added to the history, so no question influences its own weights. The statistics use the last w images and are updated every u images. For the first images, before any history exists, the controller uses uniform voting, and these images are included in the score. We tested a random image order and a product-blocked order, in which all images of one product arrive before the next product starts, so each new product begins with statistics from other products only.
| Protocol | MMAD avg. (%) | Δ vs. batch |
|---|---|---|
| Full unlabeled batch (paper) | 85.2 | — |
| Streaming, random order, w = 512, u = 256 (mean of 10 orders, SD < 0.1) | 85.0 | −0.2 |
| Streaming, product-blocked order (mean of 10 orders, SD < 0.1) | 85.0 | −0.2 |
| Statistics frozen after the first 512 images (mean of 3 seeds; scored on the remaining images) | 85.1 | −0.1 |
| Range over all 12 (w, u) settings, w ∈ {128,…,1024}, u ∈ {64, 256, 512} | 84.9 – 85.1 | −0.3 – −0.1 |
As shown in the table, the change is −0.2 in both orders (95% bootstrap CI over images: [−0.3, −0.1]), and it stays between −0.3 and −0.1 across all 12 choices of w and u. SiGMA-Agent (see our response to Reviewer 787k, W1 & Q1) likewise estimates reliabilities online, from previously answered questions only.
Q2: Does the estimated source reliability transfer across industrial domains or product categories?
A: We thank the reviewer for this insightful question. The estimated reliability transfers across both industrial domains and product categories. MMAD itself spans 4 industrial sources and 38 product categories, from industrial parts and electronic boards to food and retail goods, which allows us to test this transfer directly. Specifically, we hold out one source (e.g., VisA) or one product (e.g., bottle) at a time, estimate the controller statistics only on the remaining data, and apply them unchanged to the held-out one, so the target is never seen during estimation. The overall accuracy remains 85.2 and the accuracy on each held-out source changes by at most 0.1.
Protocol. We hold out targets in two ways. In leave-one-source-out, each of the four sources is held out in turn, and the controller statistics of Q1 (source reliabilities and the evidence-score distribution) are estimated on the other three sources only. In leave-one-product-out, each product is held out in turn and the statistics are estimated on all other products. In both cases, the statistics are frozen and applied to the held-out target without any update. Since DS-MVTec reuses MVTec-AD images, the two are always held out together.
| Where the statistics are estimated | MMAD avg. (%) |
|---|---|
| All of MMAD, including the target (paper) | 85.2 |
| Other sources only (leave-one-source-out) | 85.2 |
| Other products only (leave-one-product-out) | 85.2 |
The table below shows the leave-one-source-out result for each held-out source.
| Held-out source | Statistics from the other sources (%) | Statistics from all of MMAD (%) |
|---|---|---|
| MVTec-AD | 93.8 | 93.8 |
| MVTec-LOCO | 80.3 | 80.2 |
| VisA | 82.6 | 82.6 |
| GoodsAD | 84.0 | 84.1 |
As shown in the tables, estimating the statistics without the target leaves the overall accuracy unchanged, and changes the accuracy on each held-out source by at most 0.1. Thus, the statistics can be estimated once on existing data and reused for new domains or products.
Response to Reviewer uBwh (1/4)
W1: The conclusion is drawn from one benchmark.
A: We thank the reviewer for this important point. To verify that our conclusion holds beyond MMAD, we froze the agent team and all hyper-parameters and evaluated SiGMA on MANTA [1], an additional benchmark of tiny objects from five domains, each captured from five viewpoints, which was never used in any design choice. Without any training or tuning, SiGMA outperforms all reported methods, improving over the strongest trained specialist SAGE [2] by 4.6, while the trained JUDO [3] even falls below its own untrained base model. A plausible reason is that SFT and RL fit the model weights to the training distribution, which limits their generalization under distribution shift, whereas SiGMA keeps all MLLMs frozen and re-estimates source reliabilities label-free on each new benchmark. This supports the main finding of our paper beyond MMAD. Moreover, on MMAD itself, SiGMA is insensitive to all key hyper-parameters, with performance varying by at most 0.5. This indicates that SiGMA's performance stems from the framework itself without the need for careful hyper-parameter tuning on MMAD.
(a) Additional unseen benchmark. MANTA [1] focuses on tiny objects, such as grains, pills and small parts, from five domains (agriculture, electronics, groceries, mechanics, medicine). Each image stitches five viewpoints, and five binary items are asked about each object. Besides item accuracy over all 10,000 items, we report the stricter all-five-correct rate, where all five items about an object must be answered correctly. We froze the team, the readers, all hyper-parameters and the decoding budgets, and no label was read before scoring.
As shown in the table, SiGMA improves over SAGE [2] by 4.6, over the fine-tuned InternVL2 by 6.8, and over GPT-4o by 14.4. The gain is even larger under the all-five-correct metric, where SiGMA improves over SAGE by 13.5. In contrast, in-domain training does not transfer: given the same context, JUDO falls 3.1 below its own untrained base model, Qwen2.5-VL-7B, and remains below it in item accuracy even with the same defect evidence.
(b) Sensitivity on MMAD. On a stratified subset of 1,024 images (4,981 questions) with the full 8 base agents and 3 readers, we vary each key hyper-parameter while fixing the others at their defaults.
As shown in the table, every value stays within 0.5 of the default. The largest change comes from switching off evidence weighting entirely (β = 0), and w and p_low change the result by at most 0.05. The reported results therefore do not rely on finely tuned hyper-parameters. We will report these results in the camera-ready version (CR).
[1] MANTA: A Large-Scale Multi-View and Visual-Text Anomaly Detection Dataset for Tiny Objects. CVPR 2025.
[2] SAGE: A Visual Language Model for Anomaly Detection via Fact Enhancement and Entropy-aware Alignment. ACM MM 2025.
[3] JUDO: A Juxtaposed Domain-Oriented Multimodal Reasoner for Industrial Anomaly QA. ICLR 2026.