- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes a new method and a new dataset to address the challenging problem of cross-SKU defect detection in industrial settings. The authors point out that neither existing methods nor datasets have adequately addressed this problem. To this end, they introduce a new multi-SKU defect detection dataset consisting of shampoo bottles, with training and test sets containing disjoint SKUs. Based on this new dataset, the authors propose a defect detection method designed to improve cross-SKU generalization. By combining pretrained Swin-B and Grounding DINO models with several newly proposed modules, the method achieves some improvements in detection accuracy on the new dataset.
Critical strengths
- Cross-SKU defect detection is indeed an important and challenging problem in real-world industrial production. Therefore, the paper's focus on this problem is highly valuable.
- The proposed method appears to be theoretically sound. In addition, the paper is clearly written, well-organized, and logically coherent.
Critical weaknesses
-
The test set should include both new and previously seen SKUs. In real-world applications, new and existing SKUs will inevitably coexist. Therefore, the model should not focus solely on new SKUs at the expense of its accuracy on SKUs encountered during training. However, this paper evaluates performance only on new SKUs, making the evaluation incomplete.
-
Although cross-SKU generalization is an important problem, the differences among SKUs can vary substantially across product categories. The proposed method is evaluated only on shampoo bottles, along with experiments on GC10-DET that simulate different SKUs. Given the limited diversity of product categories, I do not believe these experiments provide sufficient evidence that the proposed method can also achieve good cross-SKU detection performance on other types of products.
-
According to Table 2, the proposed method improves AP by only 1.1 points over MM-GDINO-T, while increasing the number of parameters by 46%. I do not find this result particularly convincing, nor does it adequately demonstrate the superiority of the proposed method. Furthermore, Table 3 shows that even after removing all the newly proposed modules, the model still outperforms MM-GDINO-T. This suggests that the improvement may be primarily due to the increased number of parameters rather than the proposed method itself.
Questions for authors
Please focus on addressing the issues raised in the Weaknesses section, particularly by improving the rationale behind the training and test set split and clarifying how the superiority of the proposed method is demonstrated.
We thank the reviewer for highlighting the need for a more complete evaluation of seen and unseen products and a careful same-backbone comparison. We have incorporated these clarifications into the revised manuscript in red.
Q1: Test set should include seen and unseen SKUs, not only unseen SKUs.
A1: We thank the reviewer for this helpful suggestion. We have extended the evaluation to include seen SKUs, unseen SKUs, and their combined test set, and repeated the experiments with two training seeds. We retain the original 10 unseen SKUs as the unseen test set, comprising 2,624 images. Within the original 38 training SKUs, we construct new training, validation, and seen-test subsets with a target ratio of approximately 8:1:1 via random sampling, yielding 8,199, 1,145, and 1,151 images, respectively. The combined test set contains 3,775 images from both seen and unseen SKUs.
Table 1: Seen, unseen and combined tests under the revised split. Avg. denote averaged over two seeds.
ProDMM improves AP over the same-backbone baseline on seen SKUs, unseen SKUs, and the combined test set. The improvements occur in both training runs across all three evaluations. These results demonstrate that ProDMM improves generalization to new SKUs while also enhancing detection on existing SKUs.
Q2: Shampoo bottles and simulated GC10-DET may not sufficiently cover product-category diversity.
A2: We agree that a broader evaluation across more product categories would prove generalizability, and we have added this as an explicit limitation in the revised manuscript. To the best of our knowledge, commonly used public surface-defect datasets lack the product-SKU annotations required for an evaluation with separate training and test SKU identities. We therefore construct GC10-DET-SKU from the widely used GC10-DET dataset as a complementary appearance-based evaluation. This dataset extends the analysis from shampoo bottles to metallic surfaces with different textures. ProDMM achieves 37.8 AP, compared with 35.8 AP for MM-GDINO-T.
ProDMM uses an input-dependent soft assignment over training SKU prototypes to condition feature modulation, together with defect-aware cross-attention. The current results support its effectiveness on held-out bottle SKUs and on the appearance groups constructed from GC10-DET. We recognize broader product coverage as an important direction and will pursue evaluations on additional product families with recorded SKU identities through further industrial collaborations.
Q3: The contribution of the proposed components relative to increased model capacity.
A3: We appreciate the reviewer’s concern regarding the trade-off between model complexity and performance gain. The Baseline in the table below is the direct reference for evaluating our proposed components. Its 15.1 AP establishes the starting point of our component analysis, the proposed components further improve AP from 15.1 to 16.0, with approximately 8.5% additional parameters relative to baseline.
Table 2: Parameter statistics of the proposed components.
Beyond the overall gain, the component results show improvements among configurations with similar model sizes. Introducing prototype conditioning raises AP from 15.5 to 15.8. Adding defect-aware cross-attention further increases AP to 16.0 and AP75 from 8.0 to 8.9. These results support the contribution of SKU-conditioned modulation and defect-aware interaction. We further compare the Baseline and ProDMM under the revised split in Table 1 above. ProDMM improves unseen-SKU AP for both seeds, from 13.8 to 14.6 and from 13.5 to 13.9, respectively. The improvements provide additional evidence of the effectiveness of our method across both seen and unseen product appearances.