Noise-Aware Demonstration Selection for Fairness-Sensitive In-Context Prediction

Few-shot prediction with large language models is often treated as a retrieval problem where the goal is to choose relevant demonstrations. This overlooks a key failure mode in fairness-sensitive settings because same-group, different-group, and desensitized demonstrations can all carry noisy labels. We study the relationship between label noise, group identity, and demonstration selection for in-context prediction, with sex and race as sensitive attributes and prompt-only access to black-box large language models. Regression analyses across two prediction tasks quantify how demonstration label noise interacts with group concordance, desensitization, prompt length, and query-group identity. A central finding is that sensitive-attribute concordance does not protect a prompt from label noise (i.e., same-group demonstrations with noisy labels are still associated with reduced target-label correctness). Motivated by these findings, we introduce Fairness- and Noise-Aware Demonstration Selection (FANDS), which combines estimated noise risk with group relation, similarity, and demonstration-order information to improve few-shot prediction. At selection time, a surrogate model estimates the expected performance of candidate demonstration sets without requiring internal large language model scores or repeated exhaustive prompting. Held-out evaluations show that FANDS improves overall predictive performance and fairness metrics relative to baselines. These results show that demonstration label noise is an important factor in fairness-sensitive in-context prediction and that accounting for both noise and group context can improve few-shot predictive performance and fairness.