- Rating
- 4
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper argues in favor of compounding the membership signal (curvature of the loss as defined by Ravikumar et al.[1] and the target's loss) by using critical sharpness () which characterizes the curvature of loss near the neighbourhood of a target sample. With empirical evidence, the author(s) establish superiority of - aided SharpMIA w.r.t. SOTA membership inference attacks (MIAs). It differs from the latter in that instead of just using a point estimate, it derives membership information from the geometry of the loss in the locality of the target sample. Additionally, the paper's proposed SharpMIA attack is a black-box attack which does not require access to the model's gradients w.r.t. the target sample for computing a surrogate critical sharpness component (). It can be approximated using reference models. Notably, there already exists research that links critical sharpness to memorization [2] but this is the first work to utilize it for improving MIAs.
[1] Deepak Ravikumar et al. Curvature clues: Decoding deep learning privacy with input loss curvature. NeurIPS (2024).
[2] Moosavi-Dezfooli et al. Robustness via curvature regularization, and vice versa. IEEE CVPR (2019).
Critical strengths
- Computing the critical sharpness does not rely on access to target model gradients for the target samples. The gradients can be approximated from the reference models trained without the target sample. This is cost effective as the reference models can be reused across target samples, amortizing its training compute and avoiding expensive per-sample retraining.
- Demonstrated marked improvement over other SOTA methods (as listed in Table 1, Page 8). SharpMIA using a compound signal comprising of loss curvature signal (raw and calibrated), target model's confidence score, and the critical sharpness signal (raw and calibrated) is clearly superior to the other baselines across diverse models and datasets.
Critical weaknesses
None
Questions for authors
Suggestions
-
Carlini et al. [3] characterized both member and non-member samples as being hard-to-learn and easy-to-learn and Aerni et al. [4] urge for MIAs to be used to evaluate per-sample vulnerabilities. The argument in lines 277–280 also relies on a high-confidence approximation. Could the author(s) stratify the target samples by the their correct-class confidence and report (i) the agreement between and the target-gradient measurement , and (ii) the per-sample SharpMIA performance evaluated using only and ? This would help establish whether lower confidence affects the underlying sharpness signal. Otherwise, the proposed approach is robust to sample difficulty.
-
The attack inherits weaknesses of other reference model-based attacks including:
- You need to train multiple reference models to estimate accurately.
- Adversary is assumed to know the target model architecture, training hyperparameters, and data distribution.
The author(s) should add a Limitations section to acknowledge these inherited weaknesses or add experiments that prove the above arguments to be false.
[3] Carlini et al. Membership Inference Attacks From First Principles. IEEE S&P 2022.
[4] Aerni et al. Evaluations of Machine Learning Privacy Defenses are Misleading. ACM SIGSAC (CCS) 2024.