Label Distribution Learning-based Label Completion for Crowdsourcing

Label completion serves as a preprocessing step to handle the sparse crowdsourced label matrix problem, significantly boosting the effectiveness of the downstream label integration. In recent advances, worker modeling has been proven to be a powerful approach to label completion. However, existing algorithms typically treat the labels annotated by workers as single and deterministic labels, thereby failing to capture the class ambiguity and uncertainty in worker annotation and consequently limiting their performance. To address this limitation, we propose a novel label distribution learning-based label completion (LDLLC) algorithm. Specifically, for each worker, we first design a class membership estimation method to obtain the label distribution of each instance annotated by it and develop a worker similarity estimation method to find its similar workers (including itself). Then, we utilize the instances annotated by its similar workers to augment the instances annotated by it and use the worker similarity to weight the augmented instances. Subsequently, we train a label distribution learning model on the augmented and weighted instances to capture the class ambiguity and uncertainty in worker annotation. Finally, we use the trained model to predict the label distribution of each unannotated instance and complete the missing label. Extensive experiments show that LDLLC significantly outperforms all its competitors.