Instance Difficulty-aware Label Distribution Learning from Crowds

In crowdsourcing scenarios, we can often collect a multiple noisy label set of each instance from different crowd workers and then infer its unknown true label by label integration. Therefore, the performance of label integration is mainly influenced by the noise contained in each instance's multiple noisy label set, which primarily stems from the worker quality and the instance difficulty. However, most existing label integration algorithms mainly focus on modeling the worker quality while ignoring the instance difficulty. To fill this gap, we propose a novel label integration algorithm called instance difficulty-aware label distribution learning (IDLDL). Specifically, we first design a dual label distribution enhancement method to obtain the initial label distribution of each instance. Then, we introduce an instance correction matrix to model the instance difficulty and a label distribution learning model to map the relationship between instances and their label distributions. Thirdly, we optimize this matrix and model to learn the true label distribution of each instance from its initial label distribution under the EM framework. Finally, we use the learned true label distribution of each instance to infer its integrated label. Extensive experiments show that IDLDL significantly outperforms all its competitors.