SharpMIA: Critical Sharpness Beyond the Input Hessian for Membership Inference

Membership inference seeks to answer if a given example is in a model's training set. It is both an instrument to measure a model's data retention and an attack vector to leak private data. Nearly every attack against vision models reads the model at the example itself and differs only in what it compares that reading against; the one line of work that moves the reading into the loss geometry takes the trace of the input Hessian to extract curvature, still at the example. We find that the loss carries a membership signal not only at the example but also in the local geometry of its neighborhood. To characterize the geometry and derive a signal, we consider the changes in loss over a finite distance across the example to extract critical sharpness, ${\lambda }_c$—this is effectively how far one must walk from the example before the loss returns to its starting value. A walk needs a direction. Consequently, computing ${\lambda }_c$ needs the target model's gradient. But, we show that it is not necessary. So the signal poses a problem by facilitating a black-box attack under Carlini et al. (2022)'s offline, black-box threat model. Effectively, a walking direction can be averaged from the adversary's access to reference models without the example to approximate ${\lambda }_c$. Interestingly, we find the critical sharpness and prior work's curvature evidence are strong on different examples. So, we fuse both signals together with a difficulty prior we introduce to aid signal fusion. Reproducing 11 prior attacks and evaluating our method, SharpMIA, with standard models and benchmarks, we show that: i) sharpness leaks membership information; and ii) combining it leads to the strongest attack.