FAMILIAR: Conjuring the Many Faces of a Virtual Identity via Familiarized Occasion Priors

Face synthesis conjures portraits of people who have never existed. Applications ask more of a virtual person than one portrait: a repertoire of appearances in which identity stays consistent. To create such repertoires, most existing generators treat each identity as a fixed point and add variation by a prescribed rule: perturbing its embedding, borrowing reference styles, or sampling predefined facial parameters. The form of an individual's variation is thus fixed by the method, and data at most fit its parameters. Cognitive research points to why this matters: familiarity with a face is built by learning how that face varies across encounters, and different faces vary along different dimensions. Following this insight, we introduce FAMILIAR, which learns person-specific variation instead of prescribing it. FAMILIAR represents a virtual identity as a distribution over photographic occasions. It first learns the occasion space through identity-conditioned image synthesis, without an attribute inventory, and then learns how that space is occupied for each individual: a conditional normalizing flow shared across identities predicts a full occasion distribution from a recognition embedding alone. A newly sampled identity therefore arrives with its own repertoire, without reference photographs. Occasion statistics differ more between people than within them, and with the generator fixed, another person's occasions lower recognition accuracy despite wider pose spread: useful variation is not interchangeable across identities. FAMILIAR therefore provides state-of-the-art synthetic data for face recognition and, by fixing a source occasion, de-identifies faces while preserving identity-irrelevant facial attributes.