When Membership Signals Lie: Per-Sample Membership Inference via Intermediate Checkpoints

Membership Inference Attacks (MIAs) are the primary tool for privacy auditing and machine unlearning evaluation. Population-based MIAs, which typically derive membership scores solely from the target model's output, are computationally cheap and widely adopted under the assumption that they provide a weak but informative signal. Per-sample MIAs such as LiRA, in contrast, account for individual sample hardness but require training tens to hundreds of shadow (reference) models. Challenging the weak-but-informative assumption, we introduce a vulnerability-stratified evaluation protocol and show that population-based MIAs can produce an inverted signal on vulnerable samples—those at greatest risk of privacy leakage—assigning them lower membership scores than typical non-members. Avoiding both the inverted signal of population-based MIAs and the prohibitive cost of per-sample MIAs, we propose Ghost-LiRA, a per-sample membership inference that requires no additional model training. Motivated by the observation that vulnerable samples tend to be memorized in the later stages of training, we find that on these samples, mid-stage checkpoints, naturally saved during training, behave almost as if the sample had been excluded, providing the proxies LiRA needs. Ghost-LiRA thus replaces LiRA's spatial ensemble of shadow models with a temporal ensemble drawn from a single training trajectory. Across four datasets and architectures, Ghost-LiRA matches the privacy-critical [email protected]%FPR of online LiRA with 16–64 shadow models at a roughly $1600\times$ reduction in computational cost. Applied to unlearning evaluation, Ghost-LiRA exposes substantial residual leakage on vulnerable samples—leakage that population-based MIAs fail to detect.