Attribute Inference on Differentially Private and Fair Learning

Recent fairness-and-privacy frameworks claim to protect sensitive attributes by applying Differential Privacy (DP) during fair optimization. However, existing evaluations often overlook whether released models still leak sensitive information through proxy correlations or prediction outputs. We conduct a systematic evaluation of sensitive-attribute leakage in DP-based fair learning pipelines. Using attribute inference attacks across datasets, models, and privacy budgets, we show that perturbing sensitive attributes alone is often insufficient: structural correlations in non-sensitive features allow models to "re-learn" protected information that remains inferable from released outputs. Across eight datasets and multiple attacker models, our results reveal a gap between formal privacy guarantees and effective protection in downstream models, highlighting the need for evaluations that account for proxy correlations and output-level leakage.