acceptodds
Under review as a conference paper at ICLR 2027

Beyond Sample Reliability: Relation Identity Determines Noise Harm in Supervised Contrastive Learning

Abstract

In supervised contrastive learning (SupCon), noisy labels alter not only individual supervision but also cross-sample positive relations. We ask which of these relations should be trusted. Through controlled noise placement and quantity-matched interventions, we find that sample difficulty can help identify corrupted supervision but does not consistently predict its representation harm, while changing which relations are modified can produce substantially different outcomes under the same intervention budget. Motivated by this observation, we propose Posterior-Guided Relation Correction (PRC), which learns class posteriors from noisy data and uses their pairwise agreement to reallocate normalized positive target mass among observed-positive relations while preserving instance positives. We further characterize this reweighting at the gradient level: the effect of a relation perturbation depends jointly on its assigned target mass and the current gradient geometry, and posterior-guided reweighting is locally beneficial only when the support score aligns with relation utility. Experiments across synthetic corruption, varying noise rates, and real-world noisy-label benchmarks show consistent representation improvements. Further analysis reveals a clear boundary: strong relation diagnosis does not necessarily support reliable intervention, especially when recovering missing positive relations.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Learning with Noisy Labels, Supervised Contrastive Learning, Robust Representation Learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ofFuOpenReview ↗
Rating
1(−1)
Confidence
3
Informativeness
2
Presentation
1(−1)
Soundness
1
Read review
Paper summary

Authors modify SupCon's loss to be label noise-aware by weighting per-anchor loss terms where the weights are derived from similarities (inner product) of softmaxed predictions of a model trained on the noisy data in a normal way. Experiments with synthetic and real world datasets consistently show improvement over other methods even if it's mostly around +0.5% in absolute terms.

Critical strengths
  1. Proposition is easy to understand and implement: a loss weighting modification per anchor where weights are based on a trusted network's softmaxed outputs' similarities.

    • why it is consequential to your overall assessment: it is a new*, interesting way to address the problem (*: analogues may be found in combating label noise in supervised normal learning, e.g., Patrini et al.'s CVPR 2017 paper "Making Deep Neural Networks Robust to Label Noise" or the like but its adaptation to its contrastive part is nontrivial and authors make the distinction between samplewise and pairwise very clear.)
  2. Experiments cover a good range: synthetic and real-world datasets, and quite a few other recent methods from the literature.

    • why it is consequential to your overall assessment: it's good to show emprical support for the proposition.
Critical weaknesses
  1. Section 4.1, i.e., the derivation of sijs_{ij} is arguably the most important part of the paper, yet it is not justified sufficiently in my opinion (the trust it is given might be too risky). What if hϕh_\phi overfits the noise? What if it underfits the clean samples? Training on the noisy data in a normal way makes it fallible for these questions. Yes, the emprical support suggests it's fine but what effect does it have on those results? Line 803 mentions 15 epochs: how did the authors choose this, for example? Similarly, ρκ\rho_\kappa's design is explained for its properties but it is not clear why one should not do it any other feasible way but have 1−s1-s instead of −s-s, for example: any specific constructive reason? Furthermore, in either case, if I need to first disjointly train a model on the noisy data in a traditional way (i.e., no contrasts), why should I not use label noise-tolerant techniques while I'm at it and be done with it? For example, one can integrate Zhang et al.'s ICLR 2021 paper "Learning with Feature-Dependent Label Noise: A Progressive Approach" into that first network (and their algorithm is amenable for that) and they report 74.02% for Clothing1M and 85.28% for Food-101N highly comparable with your results but without 2-stage training or any augmentation or contrastive learning. Note that I'm not arguing in a general sense why one should prefer traditional learning versus contrastive; my argument is due to your pipeline having 2 stages already (actually 3 stages if we count the linear classifier at the end) and why one should not take a shortcut at the first stage which does no contrastive learning.

    • why it is consequential to your overall assessment: the disjoint nature with a normal way of training a model on the noisy data seems risky and unjustified in that one may desert contrastive learning at that point
  2. While the experiments cover a wide range, all of them are about image classification. Yes, the vast majority of the literature only ever experiments with images and they are important but it does not mean they should be the only ones. Even a synthetic experiment on, say, audio signals or tabular data or ... could help see the applicability of the method. After all, there is nothing visual-specific in the proposition. Note that one does not have to do augmentation in supervised contrastive learning: one has label information now; augmentation is mainly a leftover from its self-supervised counterpart where it is the main ingredient.

    • why it is consequential to your overall assessment: label noise concerns datasets other than of image nature and even 1 synthetic experiment on that front could help but is missing.
Questions for authors
  • Why do the authors keep the augmentation (same question could be asked to the SupCon paper): one already has label information (noisy or not does not matter as you are dealing with that and SupCon assumes the noise away)? It is needed and the main driving point in, say, SimCLR and the like as they don't have labels and self-supervise but I don't see the need here. Would also increase applicability domain of your method to not-so-easily-augmentable types of data, e.g., tabular, time series.

  • Line 294: by "unit weight", you mean unit weight before normalization by ∑ωαp\sum \omega_{\alpha p}, right? Or you manually force them to be 1 for augmented siblings?

  • What is a typical range of values for ρκ\rho_\kappa in your experience in the experiments (with κ=1\kappa = 1)?

  • The generalization accuracy improvements in Table 1 seem to be mostly around +0.5%; authors say in Line 365 these are consistent improvements and in fact they are, but do the authors think +0.5% is significant of an improvement? (I'm not implying it isn't but rather seeking an explanation more than stating the improvement itself from the authors.)

  • On the flip side, there does not seem to be a mention of Limitations: are there any particular dataset or some other configurations where you see the proposed model not prevail?

  • Line 489 mentions "proof drafting" but there is no proof in the paper (Appendix included), which is fine of course but why mention that then?

  • Figure 2's caption reads "converts endpoint posteriors into pairwise semantic support": what do the words "endpoint" and "semantic" mean here? Isn't this saying "derives weights of the loss using predictions of a noisily-trained model"? I think there is use of loaded words for relatively simple phenomena throughout the text that could be revisited.

Reviewer_5sLaOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper studies supervised contrastive learning (SupCon) under noisy labels from a relation-level perspective. Through controlled intervention experiments, authors show that relation-level noise is more critical than sample-level for SupCon. Motivated by this, the authors propose Posterior-Guided Relation Correction (PRC), which learns class posteriors from noisy data and redistributes normalized positive target mass among observed-positive relations according to pairwise posterior agreement; experiments on synthetic corruptions and real noisy datasets demonstrate consistent performance gains. Authors also conduct detailed ablation analysis and give the mathematical derivations.

Critical strengths
  1. The distinction between sample-level and relation-level supervision reliability is well motivated.
  2. The overall experimental design is well-controlled.
  3. The PRC gains on controlled synthetic and Real-world datasets are consistent.
Critical weaknesses
  1. The presentation needs to be improved. Some concepts are introduced very early, while the explanation comes very late.
  2. PRC only reweights observed positives and cannot recover missing positive support; this limits gains in settings where FNs dominate (e.g., CIFAR-100).
  3. The contribution of the auxiliary proxy model and its computational cost require stronger controls.
Questions for authors
  1. How sensitive is PRC to the proxy architecture or size?
  2. What is the compute overhead of PRC (proxy training + per-batch sij computation) relative to Observed SupCon!
  3. For FN, could a very conservative addition of top-k high-precision FN candidates improve PRC? Can the authors report FN top-ranking precision with smaller k?
Reviewer_7YWvOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies the effect of noisy labels in supervised contrastive learning from a relation-level perspective. The main argument is that sample-level reliability is not necessarily a good indicator of how harmful corrupted supervision will be, since a single mislabeled example can introduce multiple incorrect positive and negative relations. The authors investigate this through controlled noise placement and relation-level interventions, and propose Posterior-Guided Relation Correction (PRC), which uses pairwise agreement between class posteriors to reweight observed-positive relations while preserving instance-level positives. The paper also provides a gradient-level analysis of relation reweighting and makes an interesting distinction between identifying corrupted relations and being able to intervene on them effectively. Experiments on CIFAR-10/100, CIFAR-N, Clothing1M, and Food-101N show improvements over several existing approaches.

Critical strengths

The main strength of this paper is its perspective on noisy supervision. I think the distinction between sample reliability and relation reliability is meaningful, particularly in supervised contrastive learning, where a single incorrect label affects multiple pairwise interactions. The authors do a good job motivating why removing an entire sample may be unnecessarily aggressive when some of its relations remain useful.

I also appreciate the controlled experimental design. Separating noise placement from wrong-target assignment, and then comparing interventions under matched budgets, makes the empirical analysis more informative than a standard comparison across noise rates. The oracle relation intervention experiments are especially interesting. The large difference between targeted and randomly selected relation removal provides fairly convincing evidence that the identity of the modified relations matters, not just their number.

Another positive aspect is that the paper goes beyond proposing a weighting heuristic. The gradient-level analysis provides some intuition for why a relation score that identifies corruption well may not necessarily be the best signal for optimization. I found the distinction between relation diagnosability and intervention actionability particularly useful, and the negative results on false-negative recovery add credibility to the discussion.

Finally, the experimental coverage is reasonably broad. The authors include several synthetic corruption settings, real-world noisy-label benchmarks, ablations of different relation scores, and sensitivity analyses. The improvements are fairly consistent, which supports the practical usefulness of the proposed approach.

Critical weaknesses

My main concern is the novelty of PRC itself relative to existing confidence-based and pairwise reweighting methods. The proposed mechanism is quite straightforward: train a classifier on noisy labels, compute the dot product of its posterior distributions, and use an exponential transformation to reweight positive pairs. While the motivation is reasonable, it is not entirely clear how much of the contribution comes from a fundamentally new learning principle versus a relatively simple application of posterior-based similarity weighting. The paper would benefit from a sharper discussion of what distinguishes PRC from existing methods that use soft semantic similarities or confidence-weighted contrastive objectives, particularly WSC and related approaches.

A second concern is the experimental comparison protocol. The authors adapt a large number of existing methods to a unified backbone and a 30-epoch representation-learning setting. I understand the motivation for controlling experimental conditions, but some of these methods rely heavily on specific training schedules or multi-stage optimization procedures. The additional experiments with longer training and the method-faithful PSSCL schedule help address this issue, but they do not completely resolve it for all competing approaches. It remains difficult to judge whether the reported ranking would hold under each method's original training configuration.

I also think the computational overhead deserves more attention. PRC requires training a separate ResNet-18 proxy classifier before contrastive representation learning, and the resulting posteriors remain fixed during the second stage. This is an important practical cost, especially since the main representation encoder is relatively small. The paper reports robustness to proxy-training duration, which is useful, but does not provide a clear comparison of total training time or computational cost against the baselines. It would be helpful to understand whether the accuracy improvements remain attractive under a matched end-to-end compute budget.

Another limitation is that PRC only reweights observed-positive relations. It cannot directly recover missing positive relations, even though the oracle experiments suggest that false-negative relations can be particularly harmful on CIFAR-100. The authors acknowledge this limitation and provide experiments showing why active recovery is difficult. Still, there is a noticeable gap between the broader motivation of relation-level correction and what the proposed method can actually correct. I think the scope of the method should be emphasized more clearly.

Finally, although the theoretical analysis is interesting, its connection to the actual training dynamics is somewhat limited. The gradient identities describe local behavior at a fixed model state, and the authors appropriately avoid claiming a guarantee on final representation quality. The empirical optimization diagnostic helps, but I am not fully convinced that the analysis explains why posterior agreement is consistently preferable to alternative relation scores throughout training. At present, I see the theory as useful supporting intuition rather than a strong theoretical justification for PRC.

Questions for authors
  1. Could the authors clarify the main technical distinction between PRC and existing methods based on soft pairwise similarities or confidence-weighted contrastive supervision? In particular, what part of the proposed mechanism cannot be captured by existing semantic-similarity weighting approaches?

  2. How does PRC compare with the strongest baselines when total computational cost is matched, including the additional proxy-training stage? Reporting GPU-hours or wall-clock training time would help assess the practical trade-off.

  3. The proxy posteriors are computed once and then kept fixed. Have the authors considered updating them during representation learning, or using a pretrained or jointly trained classifier? I would be interested in whether the gains depend on the separate two-stage design.

  4. Since the method does not directly address false-negative relations, especially in settings where the oracle analysis suggests they are important, could the authors discuss more explicitly how much of the available improvement PRC is able to capture?

  5. Could the authors provide more evidence that the protocol-adapted baselines retain their expected behavior under the unified setting? The longer-schedule experiments are helpful, but a broader comparison with method-faithful implementations would strengthen the empirical claims.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.