Query-Free Adversarial Face Generation via Reference-Conditioned Composition

Face recognition is widely deployed for identity verification in security-sensitive applications. Verification accepts faces whose representations lie sufficiently close to a reference, defining an acceptance region in a space learned to discriminate identities. Attribute information remains encoded alongside identity even though this objective neither specifies it nor explicitly constrains its variation within the accepted region. Motivated by this coexistence, we study query-free adversarial face generation using a public gallery and accessible recognition models as surrogates for an unseen target model. The goal is to find a representation that yields a face with a chosen race or gender different from the reference, while the target model accepts it as the reference identity. A global approximation of the chosen attribute class produces a candidate representation by projecting the reference representation onto it, but fitting the approximation without the reference obscures the class representations nearest to that reference. We instead form a new representation through reference-conditioned composition, ranking the chosen-class gallery by similarity to the reference and composing the nearest representations. Across independently trained models, gallery representations ranked near the reference tend to remain near it, as do their compositions; aggregating surrogate rankings further improves transfer. Our attack reaches success rates up to 94.3% on AWS Rekognition and 95.0% on Tencent Cloud at default thresholds; at the stricter settings recommended by AWS for law-enforcement use and documented by Tencent at a false match rate of $10^{-5}$, it retains rates up to 55.0% and 80.0%, respectively.