Contrastive Learning from Crowds

We study the learning from multiple annotations (crowds) problem in classification settings and propose a contrastive learning approach, generalizing its (un)supervised counterparts. Based on _soft labels_, which are normalized histograms of annotations over classes, we exploit the disagreement signal in the annotations to drive the shaping of the representation space. Our key contribution is to write the common (un)supervised contrastive loss functions as a cross-entropy and generalize the target distribution to be the inverted distance over the soft labels. Blurred class information allows to perform easier hard mining without the need for large batch sizes, which we support theoretically and empirically. We provide a theoretical relationship between the risks of the contrastive stage and the downstream linear classification that also uses soft labels. Empirically, we experiment with real-world datasets of various flavors (image classification, sleep staging) to demonstrate the effectiveness of the proposed method.