acceptodds
Under review as a conference paper at ICLR 2027

Prototype-Guided Dynamic Domain Modulation and a Large-Scale Multi-SKU Benchmark for Shampoo Bottle Surface Defect Detection

Abstract

Industrial shampoo-bottle surface defect detection must operate across many stock keeping units (SKUs), where differences in bottle geometry, material, label layout, reflectance, and imaging conditions can substantially alter the appearance of the same defect. Existing industrial defect datasets rarely provide both instance-level defect annotations and product-variant identities, making it difficult to study SKU-aware detection under realistic manufacturing conditions. We introduce Bottle-SKU, a large-scale multi-SKU benchmark for shampoo bottle surface defect detection. Bottle-SKU contains 13,119 images, 20,419 instance-level bounding boxes, seven defect categories, and 48 bottle SKUs, enabling evaluation of both shared defect recognition and SKU-specific appearance variation. To address the challenge of SKU-specific appearance variation, we propose Prototype-Guided Dynamic Domain Modulation (ProDMM). ProDMM constructs a visual prototype for each SKU from training images and predicts a soft SKU distribution for each input. The resulting prototype-conditioned representation dynamically generates low-rank Query/Value modulation for the visual backbone and residual conditioning for defect-text-to-visual attention, while preserving the pretrained Grounding DINO detection pathway. The model is trained jointly with detection, SKU-domain classification, and modulation-regularization objectives. Experiments on Bottle-SKU and the public GC10-DET-SKU metallic surface defect dataset demonstrate the effectiveness of prototype-guided domain conditioning for industrial surface defect detection.

Primary area:
datasets and benchmarks
Keywords:
Surface defect detection, object detection, domain adaptation, prototype learning, vision-language models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 2 reviews, above 27% of the others.
Reviewer_2cDcOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a new method and a new dataset to address the challenging problem of cross-SKU defect detection in industrial settings. The authors point out that neither existing methods nor datasets have adequately addressed this problem. To this end, they introduce a new multi-SKU defect detection dataset consisting of shampoo bottles, with training and test sets containing disjoint SKUs. Based on this new dataset, the authors propose a defect detection method designed to improve cross-SKU generalization. By combining pretrained Swin-B and Grounding DINO models with several newly proposed modules, the method achieves some improvements in detection accuracy on the new dataset.

Critical strengths
  1. Cross-SKU defect detection is indeed an important and challenging problem in real-world industrial production. Therefore, the paper's focus on this problem is highly valuable.
  2. The proposed method appears to be theoretically sound. In addition, the paper is clearly written, well-organized, and logically coherent.
Critical weaknesses
  1. The test set should include both new and previously seen SKUs. In real-world applications, new and existing SKUs will inevitably coexist. Therefore, the model should not focus solely on new SKUs at the expense of its accuracy on SKUs encountered during training. However, this paper evaluates performance only on new SKUs, making the evaluation incomplete.

  2. Although cross-SKU generalization is an important problem, the differences among SKUs can vary substantially across product categories. The proposed method is evaluated only on shampoo bottles, along with experiments on GC10-DET that simulate different SKUs. Given the limited diversity of product categories, I do not believe these experiments provide sufficient evidence that the proposed method can also achieve good cross-SKU detection performance on other types of products.

  3. According to Table 2, the proposed method improves AP by only 1.1 points over MM-GDINO-T, while increasing the number of parameters by 46%. I do not find this result particularly convincing, nor does it adequately demonstrate the superiority of the proposed method. Furthermore, Table 3 shows that even after removing all the newly proposed modules, the model still outperforms MM-GDINO-T. This suggests that the improvement may be primarily due to the increased number of parameters rather than the proposed method itself.

Questions for authors

Please focus on addressing the issues raised in the Weaknesses section, particularly by improving the rationale behind the training and test set split and clarifying how the superiority of the proposed method is demonstrated.

1 comment

We thank the reviewer for highlighting the need for a more complete evaluation of seen and unseen products and a careful same-backbone comparison. We have incorporated these clarifications into the revised manuscript in red.

Q1: Test set should include seen and unseen SKUs, not only unseen SKUs.

A1: We thank the reviewer for this helpful suggestion. We have extended the evaluation to include seen SKUs, unseen SKUs, and their combined test set, and repeated the experiments with two training seeds. We retain the original 10 unseen SKUs as the unseen test set, comprising 2,624 images. Within the original 38 training SKUs, we construct new training, validation, and seen-test subsets with a target ratio of approximately 8:1:1 via random sampling, yielding 8,199, 1,145, and 1,151 images, respectively. The combined test set contains 3,775 images from both seen and unseen SKUs.

Table 1: Seen, unseen and combined tests under the revised split. Avg. denote averaged over two seeds.

MethodSeedSeen APSeen AP50Unseen APUnseen AP50Combined APAP50
Baseline340724.658.013.836.718.647.3
Baseline6424.757.713.534.918.345.7
ProDMM340725.759.114.638.619.548.7
ProDMM6425.357.813.936.219.046.8
BaselineAvg.24.657.813.735.818.546.5
ProDMMAvg.25.558.514.337.419.347.7

ProDMM improves AP over the same-backbone baseline on seen SKUs, unseen SKUs, and the combined test set. The improvements occur in both training runs across all three evaluations. These results demonstrate that ProDMM improves generalization to new SKUs while also enhancing detection on existing SKUs.

Q2: Shampoo bottles and simulated GC10-DET may not sufficiently cover product-category diversity.

A2: We agree that a broader evaluation across more product categories would prove generalizability, and we have added this as an explicit limitation in the revised manuscript. To the best of our knowledge, commonly used public surface-defect datasets lack the product-SKU annotations required for an evaluation with separate training and test SKU identities. We therefore construct GC10-DET-SKU from the widely used GC10-DET dataset as a complementary appearance-based evaluation. This dataset extends the analysis from shampoo bottles to metallic surfaces with different textures. ProDMM achieves 37.8 AP, compared with 35.8 AP for MM-GDINO-T.

ProDMM uses an input-dependent soft assignment over training SKU prototypes to condition feature modulation, together with defect-aware cross-attention. The current results support its effectiveness on held-out bottle SKUs and on the appearance groups constructed from GC10-DET. We recognize broader product coverage as an important direction and will pursue evaluations on additional product families with recorded SKU identities through further industrial collaborations.

Q3: The contribution of the proposed components relative to increased model capacity.

A3: We appreciate the reviewer’s concern regarding the trade-off between model complexity and performance gain. The Baseline in the table below is the direct reference for evaluating our proposed components. Its 15.1 AP establishes the starting point of our component analysis, the proposed components further improve AP from 15.1 to 16.0, with approximately 8.5% additional parameters relative to baseline.

Table 2: Parameter statistics of the proposed components.

Configuration (cumulative)Total parametersBottle-SKU AP
Baseline232.98M15.1
\quad + DMT251.72M15.5
\quad + Prototype251.98M15.8
\quad + DCA (Full)252.68M16.0

Beyond the overall gain, the component results show improvements among configurations with similar model sizes. Introducing prototype conditioning raises AP from 15.5 to 15.8. Adding defect-aware cross-attention further increases AP to 16.0 and AP75 from 8.0 to 8.9. These results support the contribution of SKU-conditioned modulation and defect-aware interaction. We further compare the Baseline and ProDMM under the revised split in Table 1 above. ProDMM improves unseen-SKU AP for both seeds, from 13.8 to 14.6 and from 13.5 to 13.9, respectively. The improvements provide additional evidence of the effectiveness of our method across both seen and unseen product appearances.

Reviewer_5U8JOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses a key bottleneck in industrial automated surface defect detection: model performance degradation when manufacturing lines switch between different Stock Keeping Units (SKUs), where substantial variations in geometry, material, label layout, and illumination occur. To resolve this challenge, the authors present the following core contributions. The Bottle-SKU Benchmark Dataset: fills a gap in existing industrial surface defect datasets, which predominantly focus on single metallic surfaces or local patches without joint instance-level annotations and explicit product-variant identities. The Prototype-Guided Dynamic Domain Modulation (ProDMM) constructs offline visual prototypes for each training SKU and predicts a soft SKU probability distribution for each input image during inference, using the resulting prototype-conditioned representation to dynamically generate low-rank Query/Value residual updates for the visual backbone (DMT module). The Domain Cross-Attention (DCA) injects prototype-conditioned visual context into defect-text interactions, allowing the model to adapt text-visual features to product-dependent appearance changes while preserving the shared, pretrained Grounding DINO detection pipeline.

Critical strengths

1 Clear and Industry-Relevant Motivation: Accurately identifies a major real-world bottleneck in multi-SKU manufacturing lines: defect semantics are shared across products, whereas appearance priors shift per product variant. The proposed SKU-disjoint evaluation directly simulates actual line-switching deployment scenarios.
2 High-Quality, Challenging Benchmark: The newly introduced Bottle-SKU dataset is large-scale, professional, and well-annotated (containing over 10k full-object images across 48 SKUs). The SKU-disjoint split prevents data leakage via interpolation, offering a solid benchmark for future research.
3 Efficient Design: By freezing the large vision-language backbone (Swin-B / BERT) and introducing lightweight dynamic modulation along with DCA, the method leverages strong pretrained priors (Grounding DINO) without incurring the high computational cost of full fine-tuning for every new SKU. Evaluates performance extensively on Bottle-SKU as well as on a pseudo-SKU split of the public GC10-DET dataset (GC10-DET-SKU). Comprehensive ablation experiments validate the individual contributions of each component, modulation width, prototype support size, and loss weights.

Critical weaknesses

1 The introduction fails to elucidate how the SKU domain issue is addressed, and the methodological contributions are insufficiently highlighted. Furthermore, there is a lack of necessary analysis regarding the extent to which the SKU domain impacts performance.

2 In the flowchart of Method in Figure 2, the abbreviations and full names should be consistent and in line with the main text. If the innovation mainly lies in the framework, it is recommended to draw the framework more clearly and highlight the key points. For example, the variable symbols related to the process in the paper should be included in the figure.

3 The feature heatmaps processed by the DMM module can be compared with those from the detector without the DMM module, which would be more beneficial to demonstrate its effectiveness.

4 While parameter-efficient, the underlying Grounding DINO baseline is heavy (252.68M parameters). Generating dynamic parameters and computing multi-head DCA introduce additional memory footprint and latency, which may hinder deployment on resource-constrained industrial edge devices.

Questions for authors

See the weakness.

1 comment

We thank the reviewer for recognizing the industrial motivation and the benchmark, and for suggestions to clarify the method and its practical cost. We have incorporated these clarifications into the revised manuscript in red.

Q1: SKU-domain handling, contributions, and impact unclear.

A1: We thank the reviewer for this comment. We have strengthened the methodological explanation in the manuscript. ProDMM addresses this challenge through prototype-conditioned, layer-wise Swin modulation and defect-aware cross-attention. To quantify the observed seen–unseen performance difference, we retrained the Baseline and ProDMM under the revised random split using different seeds. As shown in the table below, under this split, performance across SKU domains drops when the model is evaluated on SKU domains not seen during training and the mean seen-to-unseen AP gap is about 11 points. ProDMM improves AP over the same-backbone baseline on seen SKUs, unseen SKUs, and the combined test set.

Table 1: Seen- and unseen-SKU performance under the revised split. Avg. denotes the mean across two seeds.

MethodSeedSeen APUnseen AP
Baseline340724.613.8
Baseline6424.713.5
BaselineAvg.24.613.7
ProDMM340725.714.6
ProDMM6425.313.9
ProDMMAvg.25.514.3

ProDMM treats each training SKU as an appearance prototype and conditions a shared defect detector on a soft mixture of these prototypes. Concretely, ProDMM (i) builds a prototype bank from seen SKUs, (ii) infers a soft SKU posterior for each input, (iii) converts the prototype mixture into image-conditioned low-rank QLer/VLer residuals that modulate the frozen Swin backbone layer by layer, and (iv) uses DCA to inject prototype-conditioned visual evidence into defect-text cross-attention. The frozen detection pathway preserves shared defect semantics, while the prototype-conditioned residuals and DCA adapt the appearance-dependent computation. The soft mixture is essential for unseen SKUs, it represents an unseen product as a combination of seen appearance factors, enabling smooth transfer among visually related SKUs instead of forcing the input into one seen domain.

Q2: Fig. 2 abbreviations, framework, and variables unclear.

A2: We are grateful for this helpful suggestion. We have revised Figure 2 and the associated notation to improve clarity. All abbreviations used in Figure 2 now match those in the main text, and each abbreviation is defined at its first occurrence in both the figure caption and the main text. We have added the variable symbols used in the paper (e.g., PuP_u, hdomh_{\mathrm{dom}}, PP) directly into Figure 2, so that the formal definitions in the figure and the text are now fully aligned. We believe the revised Figure 2 is now clearer and more consistent with the main text.

Q3: Heatmaps with and without DMM should be compared to demonstrate its effectiveness.

A3: We sincerely thank the reviewer for this constructive suggestion. We agree that comparing feature heatmaps with and without our proposed module is a direct way to demonstrate its effectiveness. We have added paired heatmap comparisons between the Baseline and ProDMM on the same Bottle-SKU test images in Appendix E, covering various bottle appearances. ProDMM allocates highly concentrated attention to small defects compared to the Baseline, including worn areas on the upper label and lower bottle surface. We believe these visualizations clearly demonstrate the effectiveness of the DMM module in suppressing appearance noise and localizing defects on SKUs.

Q4: Extra memory/latency hinders edge deployment.

A4: We thank the reviewer for highlighting the computational requirements of industrial edge deployment. We recognize the importance of reducing this cost for resource-constrained devices. Our measurements on an NVIDIA A800 show an average processing time of 359.65 ms/image for ProDMM and 345.69 ms/image for the baseline, corresponding to approximately 4.0% additional latency, which is well within the acceptable cycle time for most industrial production lines and is a small price to pay for the significant accuracy gains on unseen SKUs. The measurements reflect performance on server hardware, whereas deployment on industrial edge devices requires further optimization. In future work, we plan to investigate structured pruning of the backbone and attention modules, as well as quantization to reduce model storage and inference memory.

Table 2: Inference cost.

MethodAverage time (ms/image)GPU memory (GiB)
DOSOD44.730.94
PET-DINO524.901.12
MM-GDINO-T196.691.09
YOLO-World129.900.96
WeDetect176.011.58
Baseline345.691.50
ProDMM359.651.75

Comments on the paper

1 comment

We thank the reviewer for recognizing the importance of the SKU benchmark and for the constructive questions about deployment, generalization, and computational cost. We have incorporated these clarifications into the revised manuscript in red.

Q1: Can the authors report false positives per image and precision at an operating threshold calibrated for industrial inspection?

A1: We thank the reviewer for raising this point. At the default threshold 0.25, precision is 38.3% with 67.7% recall. At a threshold of 0.60, the model achieves 81.5% precision and 27.3% recall at an FPPI <= 0.10.

Table 1: Threshold sensitivity on the Bottle-SKU test set.

ThresholdPRFPPI
0.2538.367.71.75
0.4064.750.10.44
0.5075.336.70.19
0.6081.527.30.10

To calibrate the operating point on a validation set, we retrain and evaluate ProDMM under a revised random split with two seeds (3407, 64). We preserve the original 2,624-image unseen-SKU test set in Bottle-SKU and partitioning the original training set into 8,199 train / 1,145 seen-SKU validation / 1,151 seen-SKU test images. The following table reports the mean fixed-threshold results over both seeds.

Table 2: ProDMM under the revised split, the metric averaged over two seeds.

ThresholdPRFPPI
Seen-SKU validation
0.2548.269.41.17
0.4073.256.30.32
0.5081.645.90.16
0.6087.836.70.08
Seen-SKU test
0.2548.268.71.15
0.4070.954.40.35
0.5079.745.30.18
0.6086.136.60.09
Unseen-SKU test
0.2539.763.11.54
0.4061.448.20.49
0.5069.537.90.27
0.6075.829.40.15

Q2: Can the method be evaluated on another factory, product family, or temporally separated production batch?

A2: We thank the reviewer for this question on cross-environment generalization. Bottle-SKU covers all bottle variants currently used by our partner factory and encompasses a diverse range of bottle types. To date, our industrial collaboration has centered on this factory, and no other factories are currently involved. We plan to expand collaborations with additional factories to evaluate independently collected product families and temporally separated production batches.

Q3: How does constructing GC10-DET-SKU by clustering features from all images affect the independence and representativeness of the held-out split?

A3: We appreciate this concern. To our knowledge, public surface-defect datasets do not provide product-SKU identities supporting a SKU-disjoint protocol, so we build GC10-DET-SKU as a complementary appearance-based evaluation. We first extract fixed pretrained visual features from the complete GC10-DET image collection and cluster them. Images with a larger visual-semantic gap are assigned to different clusters, producing clear differences in surface appearance. We then hold out complete visual clusters as pseudo-SKUs. The extracted visual clusters are visually independent and serve as representative pseudo-SKUs corresponding to different appearances within the same product category. GC10-DET-SKU complements Bottle-SKU by extending the analysis from shampoo bottles to metallic-surface inspection.

Q4: How does ProDMM handle a newly introduced SKU whose appearance is substantially different from all stored prototypes?

A4: We thank the reviewer for this insightful question. The mean prototype is extracted as an aggregated representation of the bottles belonging to the same SKU, and thus encodes the appearance information of that SKU. As noted at the end of Section 3.3, "This soft aggregation retains uncertainty among visually similar SKUs instead of committing to a hard prototype assignment.", which allows appearance information from the same product but with different appearances to be recoupled at inference. Specifically, when a newly introduced SKU still falls within the general category of bottles, it can essentially be regarded as a combination of existing SKUs. In this sense, its appearance can be represented through soft aggregation over stored prototypes rather than requiring a dedicated hard prototype, which mitigates the risk of failing to capture substantial within-SKU variation.

Q5: Can the authors report end-to-end inference latency and memory usage against Grounding DINO and comparable detectors?

A5: We appreciate this comment. As shown in the table below, we report the average processing time per image and the maximum recorded GPU memory. Here, Baseline denotes Grounding DINO, and all experiments were conducted on an NVIDIA A800. ProDMM requires 359.65 ms/image and 1.75 GiB of GPU memory. ProDMM is faster than PET-DINO but uses more memory. We will consider pruning, quantization, and caching to accelerate inference as future work.

Table 3: Inference cost.

MethodAverage time (ms/image)GPU memory (GiB)
DOSOD44.730.94
PET-DINO524.901.12
MM-GDINO-T196.691.09
YOLO-World129.900.96
WeDetect176.011.58
Baseline345.691.50
ProDMM359.651.75

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.