InfoAno: Information-Theoretic Face Anonymization

Deep learning-based face recognition (FR) poses a serious threat to individual privacy by exploiting readily available facial data for unauthorized mass surveillance, undermining the public anonymity essential to fundamental civil liberties. Face anonymization serves as a critical countermeasure by removing identity information while preserving useful attributes for legitimate downstream analysis. However, existing approaches struggle with the core privacy–utility conflict: they either lack principled privacy, leaving identity vulnerable to recovery, or lack principled utility, indiscriminately destroying innocuous attributes. In this paper, we pioneer an information-theoretic perspective on face anonymization. We first formulate face anonymization through mutual information and then theoretically expose the identity recovery vulnerability of adversarial-based methods through two failure modes: illusory adversariality and blind optimization. Guided by our theoretical analysis, we propose InfoAno, an information-theoretic face anonymization model that achieves a principled privacy–utility balance. For privacy, InfoAno employs a stochastic substitution mechanism to eliminate illusory adversariality; for utility, it specifies feasible attributes to eliminate blind optimization. We further unify privacy protection and utility preservation within an information bottleneck framework and optimize the resulting objective via deep variational inference with tractable bounds. Extensive experiments validate the superiority of InfoAno, thwarting identity recovery while faithfully preserving expected attributes.