How reaching a Reliable Ethnicity and Gender Inference (REGI)?

Fairness evaluation in face analysis systems (FAS) uses automatic demographic attribute inference (DAI), which in turn relies on predefined demographic segmentation. However, the validity of fairness auditing hinges on the reliability of the DAI process. We revisit this process by proposing a fully reproducible ethnicity+gender prediction pipeline (REGI) that includes curation of diversified training and test datasets and a transfer-learning-inspired model training strategy. We ground our approach in a per-identity bound on the probability that an image is assigned a different segment than its identity's mean embedding. We show that the decision class this bounds covers, log-likelihood ratios of von Mises-Fisher mixtures, is realized by an RBF-SVM head on a pretrained face recognition encoder. Its margins being driven by the coverage of the training set motivates further our diversified training set WEG${}^{tr}$. We audit this pipeline across three dimensions: accuracy, fairness, and a newly introduced notion of robustness, defined via intra-identity consistency. We compare our approach with seven competitive baselines across multiple datasets and training setups, including pipelines trained specifically for attribute prediction and generalist VLMs. Our pipeline outperforms all baselines for ethnicity, the more challenging attribute, across the three audited dimensions. REGI also ensures an interesting performance-efficiency balance, beating all the dedicated attribute predictors even with smaller computational budgets. Importantly, the newly proposed attribute training dataset generalizes better than existing counterparts, while our new test dataset, more challenging, is composed more suitably for evaluating the three audited dimensions. To promote reproducibility and adoption, we release the dataset metadata, full codebase, pretrained models, and evaluation toolkit.