AgeALBench: Benchmarking Deep Active Learning for Facial Age Estimation with Crowd-sourced Annotations

Age estimation from facial images is a problem of practical importance, with a variety of real-world applications. While deep neural networks have depicted commendable performance for this task, they require large amounts of hand-annotated training data, acquiring which is a time-consuming and labor-intensive process. Active Learning (AL) algorithms automatically identify the salient and exemplar samples from large amounts of unlabeled data, and tremendously reduce human annotation effort in training deep neural network models. However, facial age estimation poses a unique challenge as it is almost impossible to estimate the exact correct age of a person merely from a facial image; moreover, the perceived age is inherently subjective, and different human annotators can have very different perceptions about the age of the same subject. This necessitates a thorough analysis of the performance of AL techniques for facial age estimation, where each image is annotated by multiple annotators to capture annotation variability. In this research, we first collect crowd-sourced age annotations of two benchmark datasets: UTKFace and AAFD; each image is annotated by three annotators, who provide exact age estimates, as well as age range estimates. We then analyze the performance of 13 single annotator, as well as multi-annotator AL algorithms using the collected annotations. To the best of our knowledge, this is the first research effort to analyze the performance of deep AL algorithms (with crowd-sourced annotations) for the task of facial age estimation, where annotators can only provide an approximate label to the queried instances, rather than the exact label. We hope this research will be a step toward bridging the gap between AL and real-world applications, and our findings will be useful for the development of next generation AL algorithms under real-world annotation challenges. Our code and the collected annotations will be open-sourced upon acceptance of our paper.