acceptodds
Under review as a conference paper at ICLR 2027

Instance-Dependent Noise Robust Hashing for Multi-Label Image Retrieval

Abstract

Supervised hashing has achieved remarkable progress in image retrieval, yet its performance heavily relies on high-quality labels and is therefore vulnerable to label noise. Existing robust hashing methods mainly address class-conditional noise (CCN), while instance-dependent noise (IDN), which better reflects real-world noise patterns, remains unexplored in multi-label hashing retrieval. We identify two fundamental corruptions of pairwise supervision caused by multi-label IDN: additive IDN introduces spurious semantic similarities, causing false attraction, whereas subtractive IDN destroys genuine semantic correlations, resulting in false repulsion. To address them, we propose Instance-Dependent Noise Robust Hashing (IDNRH). It consists of two complementary components: Sample-to-Label Truncation (SLT), which uses a GMM to partition potentially noisy samples, reconstructs label confidence through multi-view neighbor consensus, and truncates unreliable labels to mitigate false attraction; and Repulsion-Filtered Soft Contrastive Learning (RFSCL), which suppresses false repulsion using shared-neighbor similarity while adaptively modulating attraction with soft label similarity. Extensive experiments on widely used datasets and a real-world noisy dataset demonstrate that IDNRH consistently outperforms state-of-the-art hashing methods under multi-label IDN.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
hashing, label noise, partial-label

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_XpTNOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2(+1)
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper investigates the problem of multi-label image hashing retrieval under instance-dependent label noise. The authors categorize the impact of label noise on pairwise supervision into two types: false attraction caused by erroneously added labels, and false repulsion caused by missing ground-truth labels, and propose the IDNRH framework.

The method consists of two main modules: Sample-to-Label Truncation (SLT) partitions samples via a GMM based on classification losses, reconstructs label confidence by incorporating dual-view neighbor information, and truncates low-confidence positive labels in suspicious samples; Repulsion-Filtered Soft Contrastive Learning (RFSCL) leverages the Jaccard similarity of the truncated labels to modulate the attraction strength, and exploits shared neighbor relationships to mitigate potential false repulsion. The model employs a frozen CLIP ViT-B/32 backbone and jointly optimizes a classification loss, a pairwise loss, and a quantization loss to train the hash head. Experiments are conducted on three synthetic noisy datasets and one real-world Web noisy label dataset.

Critical strengths
  1. The paper addresses a practically relevant problem with a clear design rationale: label additions and omissions cause false attraction and repulsion, motivating label truncation and repulsion filtering.

  2. IDNRH reports competitive results across datasets, noise settings, and code lengths, including a backbone-controlled CLIP comparison and evaluation on real Web-label noise. These results suggest practical value, subject to the evaluation concerns below.

Critical weaknesses
  1. Novelty relative to closely related work is unclear. Robust Contrastive Cross-modal Hashing with Noisy Labels (ACM MM 2024) already uses a loss-based two-component GMM for noisy-label hashing; Semantic-Consistent Bidirectional Contrastive Hashing for Noisy Multi-Label Cross-Modal Retrieval (AAAI 2026) has already constructed soft contrastive learning based on label overlap, performing repulsion and attraction. This paper likewise uses a GMM to distinguish noisy samples and constructs soft contrastive learning based on label overlap, with LrepL_{\mathrm{rep}} and LattL_{\mathrm{att}}, but neither cites nor compares against these works, leaving its incremental contribution unclear.

  2. Table 3 lacks a clear comparison protocol and contains an apparent metric inconsistency. It is unclear whether NACR and SLT share the same training samples, noisy labels, and mixed IDN setting, or how the single-modality adaptation was performed. Under matched data and aggregate-count metrics, Appendix B.3 implies ∣A∣∣O∣=FTR⋅PR(1−P)\frac{|A|}{|O|}=\frac{\mathrm{FTR}\cdot P}{R(1-P)} Yet the inferred ratios differ: 2.365 versus 1.078 on NUS-WIDE, and 1.586 versus 3.110 on MS COCO. These discrepancies require clarification before Table 3 can reliably support SLT’s advantage.

  3. The evaluation provides limited support for fine-grained ranking and noise robustness. The relevance definition and mAP cutoff are unspecified. If relevance means sharing at least one clean label, binary metrics cannot distinguish degrees of semantic overlap. Moreover, actual addition/deletion rates and proportions of corrupted samples are unreported. Algorithm 2’s nominal noise parameter controls CLIP-score percentiles rather than actual corruption rates, which are further affected by label restoration. These omissions obscure both ranking quality and corruption severity.

Questions for authors
  1. What are the specific contributions beyond NRCH and SCBCH? Can adapted baseline or component-replacement comparisons be provided under matched backbone and supervision settings?

  2. Do NACR and SLT use identical data and noise settings in Table 3? Please clarify the protocol and provide raw counts such as ∣A∣|A|, ∣O∣|O|, ∣M∩A∣|M\cap A|, and ∣M∩O∣|M\cap O| to resolve the metric discrepancy. For each synthetic setting, report the proportions of clean, addition-only, deletion-only, and mixed-error samples, plus post-restoration addition/deletion rates. Can separate-noise experiments and a clean-label reference also be provided?

  3. Can nDCG@K be reported over all queries using clean-label-based graded relevance and an explicit gain function?

  4. How much does training improve over pretrained CLIP? Please include frozen CLIP continuous-feature retrieval and a same-length hashing baseline using a fixed projection without labels or task-specific training, to separate the benefits of pretraining, hash learning, and noise handling.

9 comments
AuthorsOpenReview ↗

Response to Reviewer XpTN (Q1/W1 1/2)

We thank the reviewer for the constructive comments and positive assessment. We respond to each concern point by point below.

Q1/W1: What are the specific contributions beyond NRCH and SCBCH? Can adapted baseline or component-replacement comparisons be provided under matched backbone and supervision settings?

We appreciate the reviewer for highlighting these closely related works. We will cite NRCH and SCBCH in the revision.While GMM-based sample partitioning and soft contrastive learning are established techniques, the novelty of IDNRH lies in its targeted treatment of two distinct pairwise supervision errors induced by multi-label IDN, rather than the individual use of these techniques.

1. Differences from NRCH (ACM MM 2024).

NRCH employs GMM-based dynamic noise separation to select reliable training samples. However, its sample-level partitioning does not explicitly distinguish clean and corrupted labels within a partially corrupted instance.

In contrast, IDNRH introduces a finer-grained sample-to-label strategy. Specifically, we apply exponential moving average (EMA) smoothing to per-sample BCE losses before GMM estimation, and use the clean-component posterior probability to guide multi-view neighbor-based label confidence reconstruction. Subsequently, class-specific adaptive thresholds selectively remove unreliable positive labels from potentially noisy samples while preserving their remaining useful supervision.

Thus, unlike sample-level selection in NRCH, IDNRH transforms coarse sample reliability into fine-grained label-level correction, which is particularly important when correct and incorrect labels coexist within the same multi-label instance.

2. Differences from SCBCH (AAAI 2026).

Although SCBCH also employs bidirectional soft contrastive learning, its attraction and repulsion terms differ from those of our RFSCL. We clarify this distinction by comparing their respective formulations.

Its attraction term (Eq. 5) is formulated as:

\frac{1}{n^2} \sum_{i=1}^{n}\sum_{j\ne i} \exp(\xi-S_{ij})M_{ij} -\frac{1}{n}\sum_{i=1}^{n}S_{ii}.$$ where $M_{ij}=\mathbb I(R_{ij}>0)$ is a binary positive-pair mask. Therefore, although SCBCH includes partially overlapping samples as soft pairs, its label-based attraction weighting is binary rather than determined by the degree of label overlap. Its repulsion term (Eq. 7), on the other hand, employs continuous label similarity: $$\mathcal L_{\mathrm{rep}}^{\mathrm{SCBCH}} = \frac{1}{2n^2} \sum_{i=1}^{n}\sum_{j\ne i} \sum_{d=1}^{2} \exp\left( N_{ij}^{d}(1-R_{ij}) \right).$$ where $N_{ij}^{d}$ denotes the margin-adjusted feature similarity in each retrieval direction. Our RFSCL employs continuous label similarity in **both attraction and repulsion**: $$\mathcal L_{\mathrm{att}}^{\mathrm{IDNRH}} = \frac{1}{n^2} \sum_{i=1}^{n}\sum_{j=1}^{n} R_{ij}\exp(\xi-S_{ij}^{w}).$$ $$\mathcal L_{\mathrm{rep}}^{\mathrm{IDNRH}} = \frac{1}{n^2} \sum_{i=1}^{n}\sum_{j=1}^{n} (1-R_{ij})(1-\bar C_{ij}) \exp(S_{ij}^{w}).$$ Here, $S_{ij}^{w}$ denotes the feature similarity between two samples in the weak view, and $\bar C_{ij}$ denotes the normalized shared-neighbor similarity. This formulation introduces two important differences from SCBCH. **First, unlike the binary positive-pair mask in SCBCH, our attraction term directly uses continuous label similarity ($R_{ij}$) to determine attraction strength.** Consequently, samples with stronger semantic overlap receive stronger attraction, while those with weaker overlap receive correspondingly weaker attraction. This preserves fine-grained semantic relationships and helps mitigate false attraction caused by additive IDN. **Second, beyond using label similarity to regulate repulsion, we introduce shared-neighbor similarity ($\bar C_{ij}$) as complementary structural evidence**. Under subtractive IDN, genuine positive labels may be missing, resulting in underestimated label overlap and potentially incorrect repulsion between semantically related samples. High shared-neighbor similarity indicates potential semantic relevance even when observed label overlap is low. The factor $(1-\bar C_{ij})$ therefore selectively suppresses such potentially unreliable repulsion while retaining repulsion between samples with little neighborhood overlap.
AuthorsOpenReview ↗

Response to Reviewer XpTN (Q1/W1 2/2)

3. Backbone-controlled comparison with adapted NRCH and SCBCH.

To further validate the methodological advantages under matched backbone and supervision settings, we extend the backbone-controlled comparison by including NRCH and SCBCH.

Since both are originally cross-modal hashing methods, we adapt them to image-only retrieval while preserving their respective training mechanisms:

  1. We use a frozen CLIP ViT-B/32 to replace the original fixed image features and set the input dimension to 512.
  2. We retain the two independent hash branches, but feed the same image's CLIP feature to the branch originally used for text. We preserve the original losses, NRCH's DNS, SCBCH's reliability weighting, and their respective optimizers.
  3. During retrieval, both queries and gallery images are encoded using the main image branch.

We follow the same backbone-controlled experimental protocol, using matched training settings in terms of data splits, noisy-label annotations, backbone, 128-bit hash codes, noise levels (20%, 40%, and 60%), and evaluation protocol, while retaining each method's native training strategy.

As shown in Table 1, IDNRH achieves the best mAP across all nine dataset-noise settings. These results demonstrate that the advantage of IDNRH is not merely attributable to the backbone or supervision setting, but also comes from its specifically designed noise-robust learning mechanisms.

Table 1. Backbone-controlled comparison with adapted NRCH and SCBCH (mAP, %).

MethodReferenceDataset20%40%60%
Bi-halfAAAI'21NUS-WIDE66.666.666.6
MS COCO54.554.554.5
MIRFLICKR-25K74.174.174.1‾\underline{74.1}
SDCBMVC'23NUS-WIDE62.762.762.7
MS COCO58.158.158.1
MIRFLICKR-25K72.372.372.3
HHCHTIP'24NUS-WIDE62.262.262.2
MS COCO59.059.059.0
MIRFLICKR-25K73.973.973.9
DIORTKDE'24NUS-WIDE73.571.265.8
MS COCO68.963.951.7
MIRFLICKR-25K75.872.471.4
SEGANeurIPS'25NUS-WIDE76.272.365.7
MS COCO69.9‾\underline{69.9}64.860.5‾\underline{60.5}
MIRFLICKR-25K83.880.073.0
NDBETMM'26NUS-WIDE69.565.656.9
MS COCO62.456.252.3
MIRFLICKR-25K77.573.467.6
NRCHACM MM'24NUS-WIDE75.773.567.5‾\underline{67.5}
MS COCO69.9‾\underline{69.9}65.259.5
MIRFLICKR-25K85.280.072.5
SCBCHAAAI'26NUS-WIDE76.8‾\underline{76.8}74.2‾\underline{74.2}65.3
MS COCO71.366.1‾\underline{66.1}55.9
MIRFLICKR-25K86.2‾\underline{86.2}80.3‾\underline{80.3}71.6
IDNRHOursNUS-WIDE77.975.469.8
MS COCO71.367.862.8
MIRFLICKR-25K87.182.675.1

Thank you for the response. Your answers effectively addressed my key concerns regarding the paper's core arguments and novelty, and the additional comparison results with closely related methods are highly convincing. Based on this, I am inclined to give a positive score. However, I still have a few remaining concerns: providing more diverse evaluation metrics for comparison in the multi-label setting would significantly strengthen my confidence in the conclusions. Additionally, access to detailed training logs would further solidify my final rating.

Reviewer_hSDgOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses multi-label instance-dependent label noise (IDN) in supervised image hashing, identifying additive noise as causing false attraction and subtractive noise as causing false repulsion. It proposes IDNRH, which combines Sample-to-Label Truncation (SLT) for reliable label reconstruction and truncation with Repulsion-Filtered Soft Contrastive Learning (RFSCL) to suppress false repulsion while preserving soft semantic attraction. Experiments show consistent improvements over state-of-the-art hashing methods under multi-label IDN.

Critical strengths
  • This motivation of this paper is clear and supported well. It is easy to follow.
  • This work focuses on instance-dependent noise, which is more challenging and less studied. The IDNRH method is sound solving the the challenging task.
  • The experiments on four benchmark datasets verify the effectiveness of the IDNRH method. Its empirical study is convincing.
Critical weaknesses
  • Could you explain why this task is more challenging than the previous one? Please use one or two examples.
  • It is suggested to clarify why your method can solve this task? What is the key advantage of your method.
  • I notice that you learn thresholds t_k in (9), have you verified its advantage? If I predefine or fix the t_k, what is the performance?
  • How is mAP calculated, and how many samples are retrieved when calculating mAP? It seems not well defined.
  • Why we need to use R_ij and C_ij to represent similarity in Sec 4.2?
  • What is the sensitivity of the IDNRH method to the regularization parameters, alpha and beta?
Questions for authors

Please kindly address the concerns in Weakness part.

2 comments
AuthorsOpenReview ↗

Response to Reviewer hSDg

We thank the reviewer for the constructive comments and positive assessment. We respond point by point below.

Q1/W1: Could you explain why this task is more challenging than the previous one? Please use one or two examples.

Compared with conventional class-conditional noise (CCN), multi-label instance-dependent noise (IDN) is more challenging because corruption depends on individual image content and may affect only some labels in an instance. Thus, correct and incorrect labels can coexist, making sample-level noise identification insufficient.

Moreover, additive and subtractive IDN corrupt pairwise relations in opposite ways. As shown in Fig. 1, an erroneously added "Stop Sign" label to Sample A creates false semantic overlap and causes false attraction. Conversely, the missing "Cat" label in Sample B removes genuine semantic overlap and causes false repulsion. Both misguide hash-code learning and impair retrieval.

Q2/W2: Why can your method solve this task? What is its key advantage?

The key advantage of IDNRH is that it jointly addresses label-level and pairwise-relation corruption. SLT combines GMM-based sample partitioning with multi-view neighbor consensus to reconstruct label confidence and selectively remove unreliable positive labels, mitigating false attraction while preserving clean supervision.

RFSCL further uses shared-neighbor similarity to suppress false repulsion caused by missing labels, while soft label similarity preserves graded semantic attraction. These complementary modules improve the robustness of pairwise supervision. The ablation studies in Table 2 and Appendix B.2 validate their effectiveness.

Q3/W3: I notice that you learn thresholds tkt_k in Eq. (9). Have you verified their advantage? What is the performance if tkt_k is predefined or fixed?

The threshold tkt_k is adaptively determined, rather than learned. It accommodates differences in label-confidence distributions across categories and noise levels that a fixed threshold may not capture.

We conducted additional experiments on MS COCO using 128-bit hash codes under 20%, 40%, and 60% noise, comparing the adaptive threshold with four fixed thresholds:

Threshold (mAP, %)20%40%60%
tk=0.2t_k=0.269.8‾\underline{69.8}65.9‾\underline{65.9}60.9
tk=0.4t_k=0.468.965.560.8
tk=0.6t_k=0.668.265.861.1‾\underline{61.1}
tk=0.8t_k=0.867.465.860.5
Adaptive (IDNRH)71.367.862.8

Compared with the best fixed-threshold setting, the adaptive threshold improves mAP by 1.5, 1.9, and 1.7 percentage points at 20%, 40%, and 60% noise, respectively. We will include these results in the revised manuscript.

Q4/W4: How is mAP calculated, and how many samples are retrieved when calculating mAP?

mAP is calculated over the entire retrieval database, not a fixed Top-K subset. For each query, all database images are ranked by ascending Hamming distance. AP is computed from the precision at the ranks of relevant images, and mAP is the mean AP over all queries.

Thus, each query is evaluated against the full database: 193,734 images for NUS-WIDE, 117,218 for MS COCO, 18,015 for MIRFLICKR-25K, and 193,734 for NUS-WIDE-Web-21. We will clarify this protocol in the revised manuscript.

Q5/W5: Why do we need to use RijR_{ij} and CijC_{ij} to represent similarity in Sec. 4.2?

RijR_{ij} and CijC_{ij} capture complementary pairwise information. RijR_{ij} measures semantic similarity via the Jaccard overlap of truncated labels and provides graded attraction. However, subtractive IDN may remove genuine labels and underestimate RijR_{ij}, causing false repulsion. To address this, CijC_{ij} measures the Jaccard similarity between neighborhood sets and provides complementary structural information. When label overlap is low but shared-neighbor similarity is high, repulsion is suppressed to avoid false negatives.

Thus, RijR_{ij} preserves graded semantic attraction, while CijC_{ij} filters potentially unreliable repulsion, allowing RFSCL to handle both types of pairwise-relation corruption.

Q6/W6: What is the sensitivity of IDNRH to the regularization parameters, α\alpha and β\beta?

We conducted a sensitivity analysis in Appendix B.7 (Fig. 10). Under 60% IDN with 128-bit codes, we evaluate α=0.3,0.5,0.7,0.9,1.1\alpha = 0.3, 0.5, 0.7, 0.9, 1.1 and β=0.1,0.2,0.3,0.4,0.5\beta = 0.1, 0.2, 0.3, 0.4, 0.5 on three datasets. The results show stable performance: when varying either parameter individually, the mAP fluctuation remains within 1.0 percentage point on each dataset. We use α=0.7\alpha=0.7 and β=0.3\beta=0.3 as the default settings.

Thank you to the author for addressing my confusion and concerns. I keep the positive score.

Reviewer_u2FbOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

IDNRH learns hash codes for image retrieval from noisy multi-label annotations. Here, the errors depend on image content and can create false matches or hide real ones. To limit these errors, SLT identifies potentially noisy samples and removes positive labels with little support from neighboring images. RFSCL then uses the remaining labels to weight attraction between images. For relationships that these labels may miss, shared neighbors provide a reason to reduce repulsion. The experiments include three datasets with synthetic noise and NUS-WIDE-Web-21 with real Web tags. In the reported results, IDNRH achieves higher retrieval scores than the compared hashing methods.

Critical strengths

The method has a clear rationale for the multi-label setting. An image can have one wrong tag and several correct ones. SLT can retain useful information from that image by filtering positive labels individually. That still leaves matches hidden by missing tags. RFSCL brings in shared neighbors as another clue and gives potentially related pairs less weight as negatives.

The evaluation covers several datasets, noise settings and hash lengths, as well as real Web tags. Several baselines use the same CLIP backbone as IDNRH in additional comparisons. In the component tests, the full version scores better than those that leave out label filtering or the shared-neighbor check.

Critical weaknesses

In the PDF viewer I used, some symbols in Figures 2, 3 and 8 appear as boxes.

Both the synthetic noise and IDNRH’s frozen features rely on CLIP. The backbone-matched comparisons keep the same noise setup, leaving robustness to other noise mechanisms unclear.

Questions for authors

Does IDNRH still outperform the baselines when a non-CLIP model generates the noise? Does this advantage also hold when the methods use the same non-CLIP backbone?

The classifier is trained with reconstructed soft labels, while the GMM uses BCE losses computed against the observed noisy labels. How reliable is the clean/noisy split as the rate of missing positive labels increases?

3 comments
AuthorsOpenReview ↗

Response to Reviewer u2Fb (1/2)

We thank the reviewer for the constructive comments and positive assessment. We respond to each concern point by point below.

W1: Some symbols in Figures 2, 3, and 8 are displayed as boxes in the PDF.

We apologize for the inconvenience. The issue is not reproducible across the PDF readers we tested and may be related to font-rendering compatibility. In the revision, we will re-export the figures with embedded fonts and replace unsupported symbols with standard ones.

Q1: Does IDNRH still outperform the baselines when a non-CLIP model generates the noise? Does this advantage also hold when the methods use the same non-CLIP backbone?

We conduct two additional experiments to examine the effects of noise generation models and visual backbones.

Experiment 1: BLIP-1-based noise generation.

We replace CLIP with BLIP-1 [1] for noise injection while retaining the CLIP hashing backbone. IDNRH and representative baselines are evaluated on MS COCO with 128-bit codes under three nominal noise levels (20%, 40%, and 60%).

Table 1 reports the actual noise statistics. The nominal noise level is a percentile-based threshold parameter, not the actual corruption rate. Since BLIP-1 and CLIP have different image-text score distributions, identical nominal settings do not guarantee identical additive and subtractive rates. Here, the additive rate is the proportion of originally negative image-label pairs incorrectly flipped to positive, the subtractive rate is the proportion of originally positive pairs incorrectly flipped to negative, and the sample contamination rate is the proportion of images with at least one corrupted label.

Table 1. Actual noise statistics of BLIP-1-based noise injection (%). Each entry denotes Additive / Subtractive / Sample contamination.

Dataset20%40%60%
MIRFLICKR-25K0.86 / 19.31 / 48.495.57 / 37.55 / 86.6319.27 / 56.67 / 97.95
MS COCO0.11 / 19.84 / 37.401.25 / 38.68 / 76.747.16 / 56.09 / 96.90
NUS-WIDE0.83 / 12.41 / 30.822.75 / 25.76 / 54.677.60 / 42.75 / 68.86

Despite the differences in actual corruption rates, the generated datasets exhibit substantial noise. For example, MS COCO's sample contamination rate reaches 96.90% at the 60% setting.

Table 2. mAP (%) on MS COCO with BLIP-1-generated noise and a CLIP backbone.

Method20%40%60%
Bi-half54.554.554.5
SDC58.158.158.1
HHCH59.059.059.0
DIOR67.463.758.8
NDBE63.257.148.9
SEGA70.165.357.5
IDNRH70.866.763.2

IDNRH achieves the best mAP at all three noise levels, exceeding the strongest baseline by 0.7, 1.4, and 4.2 percentage points, respectively. This confirms its effectiveness under non-CLIP-generated noise.

Experiment 2: Backbone-controlled comparison using VGG19.

We further replace CLIP with VGG19 for IDNRH and the selected baselines. All methods use the same BLIP-1-generated MS COCO datasets, 128-bit codes, and three noise levels.

Table 3. mAP (%) on MS COCO with BLIP-1-generated noise and a unified VGG19 backbone.

Method20%40%60%
Bi-half57.357.357.3
SDC60.660.660.6
HHCH57.457.457.4
DIOR60.455.448.9
NDBE65.258.850.5
SEGA68.063.358.9
IDNRH69.465.261.1

IDNRH consistently achieves the best mAP, outperforming the second-best method by 1.4, 1.9, and 0.5 percentage points, respectively. This demonstrates that its advantage also holds with a non-CLIP backbone.

AuthorsOpenReview ↗

Response to Reviewer u2Fb (2/2)

Q2: The classifier is trained with reconstructed soft labels, while the GMM uses BCE losses computed against the observed noisy labels. How reliable is the clean/noisy split as the rate of missing positive labels increases?

Thank you for raising this important concern.

To specifically investigate the reliability of GMM-based sample partitioning when positive labels are missing, we conduct an additional experiment using subtractive-only IDN on MS COCO with 128-bit hash codes under three noise levels (20%, 40%, and 60%).

Following the loss-mixture modeling strategy adopted in DivideMix [2], we fit a two-component GMM to the per-sample loss distribution and use the posterior probability of the clean component as the continuous reliability score. We then evaluate its ability to distinguish clean and noisy samples using the area under the receiver operating characteristic curve (AUROC). For this offline evaluation, a sample is considered noisy if at least one of its originally positive labels is removed; otherwise, it is considered clean. The ground-truth noise status is used only for evaluation and is not involved in training.

AUROC measures the ranking capability of a score across all possible classification thresholds. An AUROC of 0.5 corresponds to random discrimination, whereas an AUROC of 1.0 indicates perfect discrimination. Equivalently, in our setting, AUROC measures the probability that a randomly selected clean sample receives a higher clean posterior probability than a randomly selected noisy sample.

The results are reported in Table 4.

Table 4. AUROC (%) of GMM-based clean/noisy discrimination under subtractive-only IDN.

Noise level20%40%60%
GMM (AUROC ↑)81.877.672.3

As shown in Table 4, the AUROC gradually decreases as the subtractive noise level increases, indicating that identifying clean samples becomes more challenging when more positive labels are missing. Nevertheless, the AUROC remains above 72% even under 60% noise, demonstrating that the GMM-based partition still provides meaningful discriminative information under substantial subtractive noise.

References

[1] J. Li, D. Li, C. Xiong, and S. Hoi. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. Proceedings of the International Conference on Machine Learning, PMLR, 162:12888–12900, 2022.

[2] J. Li, R. Socher, and S. C. H. Hoi. DivideMix: Learning with Noisy Labels as Semi-Supervised Learning. Proceedings of the International Conference on Learning Representations, 2020.

Thank you for the clear response and the additional experiments. The results with BLIP-generated noise and a unified VGG19 backbone address my main concern about the dependence on CLIP. The GMM analysis also provides useful evidence that the reliability scores remain informative as more positive labels are missing. I appreciate the authors’ efforts to address my questions.

Overall, the response strengthens my confidence in the paper, and I maintain my weak-accept recommendation.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Revised Manuscript Uploaded

Dear Reviewers and Area Chair,

We have uploaded a revised version of our manuscript to address the valuable feedback received during the discussion period.

The revision incorporates the clarifications and improvements outlined in our responses, including additional citations to closely related methods and supplementary experiments, such as the nDCG@100 evaluation for fine-grained retrieval ranking.

To keep the manuscript concise, some supplementary experiments reported in our rebuttal have not been incorporated into the revised PDF. Their detailed results and analyses remain available in our OpenReview responses.

We sincerely thank the reviewers for their constructive feedback and the Area Chair for their time and effort in overseeing the review process. The reviewers' comments have helped us further improve the overall quality of our paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.