acceptodds
Withdrawn from ICLR 2027

WISD-AD: Real World Semi Supervised Benchmark for Defect Detection of Soft Magnetic Wound Inductors

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing public datasets for industrial defect detection generally face two major limitations. Cross-domain benchmarks such as MVTec AD and VisA cover diverse but scattered categories and scenarios, which differ from practical product-level inspection needs, and performance on several commonly used benchmarks has gradually approached saturation. Although domain-specific datasets such as NEU-DET and GC10-DET are closer to real industrial scenarios, their data splits and evaluation protocols are usually inconsistent across studies, hindering fair comparison. More importantly, most existing datasets focus mainly on unsupervised or fully supervised settings, whereas a unified 2D benchmark for semi-supervised settings remains lacking. To address this gap, we propose WISD-AD, a real-world dataset for surface defect detection of soft-magnetic wound inductors. It contains five defect categories with practical challenges such as small defects, weak textures, and class imbalance, and supports fully and semi-supervised classification and detection with unified data splits and evaluation protocols. In addition, we further propose MD as a benchmark framework for WISD-AD. Extensive experiments reveal that existing methods still struggle under sparse annotations, and further show that representation quality before self-training critically affects subsequent pseudo-label reliability. M²D³ substantially alleviates this limitation, providing a strong reference for annotation-efficient defect inspection.

Primary area:
datasets and benchmarks
Keywords:
Semi-Supervised Benchmark, Semi-Supervised Learning, Defect Detection, Soft-Magnetic Wound Inductors

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_uFMKOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes WISD-AD, a dataset for surface defect detection of soft-magnetic wound inductors, together with M²D³, a semi-supervised framework that first performs feature distillation from a frozen DINOv3 and EUPE into the student backbone, and then conducts semi-supervised self-training using pseudo-labels generated by an EMA teacher. The central claim is that improving the initial representation enhances the effectiveness of subsequent self-training.

Critical strengths

The construction of a real-world industrial dataset with standardized evaluation protocols is of practical value.
Notable gains under low annotation: with only 1% labeled data, pseudo-label self-training alone achieves 13.31 mAP50, dual-teacher distillation alone achieves 36.85 mAP50, and the full method achieves 44.60 mAP50.

Critical weaknesses

Dataset.​ The dataset contains only 2,521 images, considerably fewer than MVTec AD and VisA, which limits its scale as a benchmark. How comprehensively do the five categories—Crack, Collapse Block, Missing Weld, Solder Spike, and Lifted Wire—cover the defect spectrum of this product? The two claimed characteristics, a standardized semi-supervised protocol and a long-tailed defect distribution, are not in themselves meaningful contributions, since both can be trivially reproduced from existing uniformly distributed datasets through simple sampling.
Methodological novelty.​ The teacher–student architecture with an EMA teacher is a conventional technique in semi-supervised detection. The proposed Heterogeneous Distillation distills features from a frozen DINOv3 and EUPE into the student backbone; the motivation for distilling representations from different feature spaces into a single model needs to be analyzed and justified.
Experiments.​ Defect detection is an active research area, and the comparison on WISD-AD should include methods published within the last two years. M²D³ also lacks a generalizability evaluation on existing datasets.

Questions for authors

The authors need to respond to weaknesses.

Reviewer_of2KOpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces WISD-AD, a real-world 2D dataset for surface defect inspection of soft-magnetic wound inductors, together with a benchmark protocol and a reference method. The dataset contains 2,521 production-line images (1440×10801440\times1080) covering five defect categories (Crack, Collapse Block, Missing Weld, Solder Spike, Lifted Wire), annotated with image-level class labels and bounding boxes. Images from the same production batch appear only in either the training or the evaluation split, and fixed labeled subsets are provided at six annotation ratios (1%, 5%, 10%, 15%, 20%, and 25%) for both classification and detection. The authors also propose \MD{}, a two-stage framework that first distills features from two frozen foundation models (DINOv3 and EUPE) into a ResNet-50 or YOLO11-n student and then performs EMA teacher--student pseudo-label self-training. Experiments show that existing supervised and semi-supervised methods degrade sharply under sparse annotation, while \MD{} achieves the best results at all annotation ratios (e.g., 44.60 mAP50 with 1% labels vs.\ 34.38 for the strongest semi-supervised baseline). The appendix further provides ablations, an analysis of robustness to labeled-subset sampling, a pseudo-label reliability analysis, transfer experiments on NEU-DET, and a deployment-efficiency analysis.

Critical strengths

S1. Careful split design for leakage prevention and reproducibility. Training and evaluation splits are isolated by production batch, all methods share a fixed evaluation set, labeled subsets are obtained by class-stratified sampling and released as fixed indices, and the ground truth of unlabeled samples is never accessed during training (Sec.3.1, App.B). On industrial data, leakage caused by near-duplicate samples within a production batch is a common source of inflated results, and low-label comparisons are highly sensitive to which samples are labeled; these design choices therefore directly improve the reliability and comparability of the results.

S2. Addresses a practical gap. Public product-level production-line data that support both classification and detection under multiple annotation budgets are scarce, and such a benchmark is of direct value to industrial practitioners. The evaluation also yields a useful empirical observation: with their official recommended configurations, DETR-style detectors nearly collapse under extremely sparse labels (RT-DETR 0.16 and DINO 1.09 mAP50 at 1%, Table 6).

S3. Large gains in low-label detection. With 1% labels, \MD{} outperforms the strongest semi-supervised baseline by 10.22 mAP50 (44.60 vs.\ 34.38), and by 14.25 with 5% labels (68.17 vs.\ 53.92) (Table 6); the same trend holds on NEU-DET (+6.6 and +7.3 over the strongest competing method at 1% and 5%, respectively; Table 11). With 25% labels, \MD{} reaches 88.25 mAP50, only about 2.5 points below YOLO11 trained on the full 80% training pool (90.79), suggesting that it could substantially reduce annotation costs in practice.

S4. No additional inference cost, which is meaningful for practical deployment. The foundation-model teachers, projection heads, and EMA teacher are used only during training; the deployed model is still the YOLO11-n student, with identical parameters and FLOPs (2.6M, 6.5 GFLOPs) and nearly identical latency (3.87 vs.\ 3.80 ms) (Table 12, App. F). By contrast, Unbiased Teacher requires 39.5M parameters and 204.1 GFLOPs, yet reaches only 26.37 mAP50 with 1% labels. Confining the extra cost to the training stage is important for production lines with throughput or edge-hardware constraints.

Critical weaknesses

W1. Narrow data scope. The data cover only a single product and a single imaging setup, with about 2.5K images and five classes, and contain only defective images (App.~B explicitly states that no defect-free category is included). On real production lines, the vast majority of parts are defect-free and the false-alarm rate on good parts is a key metric, which this benchmark cannot measure. Likewise, the semi-supervised setting is simulated by hiding the labels of defective images, whereas real unlabeled production data would be dominated by normal samples. Both points are somewhat at odds with the ``real-world production-line inspection'' framing.

W2. Limited novelty of the benchmark contribution. (a) The main distinguishing feature in Table 2, the "standardized semi-supervised protocol'', is a property of the evaluation protocol rather than of the data itself: the authors apply the same protocol directly to NEU-DET in App. E, and the cited KolektorSDD2 paper (Bo\v{z}i\v{c} et al., 2021) already evaluates mixed supervision with varying numbers of pixel-level annotated samples. (b)Table~2 omits GC10-DET and DeepPCB, the closest analogues discussed in Sec. 2.2. GC10-DET itself exhibits marked class imbalance, so the "long-tailed defect distribution" row is not well supported, and the "fine-grained annotations'' row is checked for every dataset.

W3. The comparisons do not isolate \MD{}'s contribution from external foundation-model priors. (a)\MD{} distills from two ViT-B/16 foundation models (Table 7), but the initialization or pretraining of the baselines is not reported. The ablation shows that the gains come almost entirely from distillation: for detection with 1% labels, distillation alone raises mAP50 from 12.23 to 36.85, whereas self-training alone reaches only 13.31 (Table 9), far below existing semi-supervised detectors (Efficient Teacher 29.17, STEP-DETR 34.38; Table 6). Tables 5 and 6 therefore mainly compare ``with vs.\ without foundation-model distillation'' rather than the semi-supervised algorithms themselves. (b) For classification, although \MD{} can exploit foundation-model priors, its improvement over SemiVisBooster is less than 1 point at every annotation ratio (Table 5).

W4. Internal inconsistencies in the reported statistics. (a)Table 6 reports RT-DETR at 1% labels as 0.16±0.440.16 \pm 0.44 (3 seeds). For three non-negative mAP values with a mean of 0.16, the sample standard deviation cannot exceed 0.16×3≈0.280.16 \times \sqrt{3} \approx 0.28, so this value cannot be correct. (b) The same runs are reported with different standard deviations in different tables: ViT-Base is 63.76±0.1663.76 \pm 0.16 in Table 3 but ±0.36\pm 0.36 in the 40% column of Table 5; Swin-Tiny is ±0.23\pm 0.23 vs.\ ±0.33\pm 0.33; YOLO11 is ±0.23\pm 0.23 in Table 4 but ±0.21\pm 0.21 in the 80% column of Table 6. (c) The ablation baseline in Table 8 (supervised ResNet-50 with 1% labels) is 39.54%, whereas Table 5 reports 49.57% for ResNet-50 with 1% labels. Relative to the latter, the gain of the full method drops from +14.48+14.48 to +4.45+4.45 points. (d) Sec. 3.1 states that classification results are averaged over five cross-validation folds, but Tables 3 and 5 report mean ±\pm standard deviation over 3 random seeds.

W5. The evaluation design does not adequately support the conclusions about benchmark difficulty and method superiority. (a) Under full supervision, modern detectors already reach 90.5--93.9 mAP50 (Table 4), and \MD{}'s mAP50:95 (53.97) is essentially the same as YOLO11's (53.92), suggesting that its advantage lies mainly at the loose IoU threshold; yet the semi-supervised results (Table 6) report only mAP50. (b) The detection test set contains about 505 images, of which only 18 are Missing Weld (derived from Table 1), so the variance across seeds cannot reflect the sampling uncertainty of the test set. (c) The rarest class, Missing Weld, is in fact among the easiest classes for detection (93--97 AP for modern detectors, Table 4), indicating that the difficulty stems mainly from inter-class similarity and low saliency rather than from the long-tailed distribution emphasized in the paper.

W6. Limited methodological novelty, insufficient evidence for the role of the self-training stage, and missing key details. (a) Multi-teacher distillation from foundation models (e.g., AM-RADIO, Ranzinger et al., 2024) and EMA-teacher pseudo-labeling (Mean Teacher, Unbiased Teacher, Efficient Teacher) are established techniques, and the observation that stronger representations improve subsequent semi-supervised learning is well known (e.g., SimCLRv2, Chen et al., 2020). The related work section does not discuss these lines of research. (b) In Fig. 9, for every initialization, the pseudo-label mAP50 after round 9 is lower than after round 1 (\MD{}: 69.03% →\rightarrow 67.55%, with a peak of 69.73% at round 5); that is, iterative self-training does not improve pseudo-label quality. Nevertheless, the detection configuration uses 10 rounds (Table 7), while Fig. 9 shows only 9 rounds and does not state the corresponding annotation ratio. If Fig. 9 corresponds to 1% labels, the pseudo-label mAP50 of about 58--61% for COCO/ImageNet initialization is hard to reconcile with the test mAP50 of only 12--13 for the non-distilled variants in Table 9. (c) The number of distilled layers NN, how YOLO11-n features are spatially aligned with the ViT patch grid, and how the resolution gap between distillation (256×256256\times256) and detection (640×640640\times640) is handled are not specified.

Questions for authors

Q1. Which images are used for the label-free Stage I distillation? Do they include the 20% evaluation set of the detection task or the 60% test set of the classification task? App. B states that the held-out evaluation set is never used for model training, but does not explicitly mention Stage I. Q2. For each baseline in Tables 3--6, what initialization (random, ImageNet, COCO, or a foundation model), number of training epochs, input size, data augmentation, and class-rebalancing strategy were used? Did any baseline use a foundation-model backbone? Q3. Could you report (i) Efficient Teacher (detection) and SemiVisBooster (classification) initialized with the same distilled backbone, and (ii) the results of linear probing or fine-tuning DINOv3 and EUPE with 1% and 5% labels? These are the most direct control experiments for separating \MD{}'s design contribution from its priors. Q4. Could you explain the RT-DETR value of 0.16±0.440.16 \pm 0.44, the different standard deviations reported for the same runs in different tables, and the gap between the Table 8 baseline (39.54%) and ResNet-50 with 1% labels in Table 5 (49.57%), and provide per-seed raw results? Q5. Does the classification protocol use five-fold cross-validation or three random seeds? How are the fixed labeled subsets mapped onto the folds? Q6. Regarding Fig. 9: which annotation ratio does it correspond to? Is the pseudo-label mAP50 computed before or after confidence thresholding? Given that quality declines after round 5, why are 10 self-training rounds still used? Do the labeled and unlabeled training images come from the same production batches? If so, this could inflate the pseudo-label quality measured on the unlabeled pool.

Reviewer_QGYkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces WISD-AD, a dataset of 2,521 wound-inductor images covering five defect categories, with classification and bounding-box annotations and predefined annotation budgets. It also proposes M²D³, which combines feature distillation from DINOv3 and EUPE with EMA-based pseudo-label self-training. Classification and detection comparisons across annotation budgets report improved performance, particularly under limited supervision. Feature visualizations provide additional qualitative evidence of improved representations.

Critical strengths
  1. M²D³ strengthens the student’s representations through foundation-model distillation before introducing pseudo-label self-training. This directly addresses the dependence of self-training on initialization quality and provides a reasonable approach to learning from scarce annotations.
  2. Separate projection heads align student features with the feature spaces of DINOv3 and EUPE, allowing both teachers to supervise a shared backbone. The framework accommodates classification and detection models, making the design applicable to both defect recognition and localization.
Critical weaknesses
  1. The selection of student feature layers and the spatial alignment of teacher and student features are not clearly described. Moreover, the reported comparisons do not isolate the benefits of individual teachers and their combination. Consequently, it remains unclear whether the gains specifically support the claimed complementarity of the two teachers.
  2. The described filtering uses a predefined confidence threshold, without explaining how confidence differences across defect categories are handled. EMA stabilizes parameter updates but does not explicitly correct systematic pseudo-label errors. Given the dataset’s class imbalance and visually similar defects, class-wise pseudo-label precision and coverage are needed to assess whether rare categories receive reliable supervision.
Questions for authors
  1. Which student feature layers are used for distillation, and how are the teacher and student features spatially aligned when their resolutions differ?
  2. Under matched training conditions, what are the respective contributions of each teacher, their combination, and pseudo-label self-training? These comparisons would help assess the claimed complementarity of the teachers.
  3. How is the pseudo-label confidence threshold selected? Can the authors provide class-wise precision and coverage of the retained pseudo labels, particularly for rare defect categories?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.