- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
This work proposes metric-normalized posterior leakage (mPL), which allows to solve some of the issues of metric differential privacy (mDP). In particular, the work focuses on joint-prior Bayesian attackers, where the joint observations of the samples are not independent and this allows an attacker to have a more effective attack in the mDP settings. The work proposes 2 new privacy definitions, which are bounded mPL, probabilistically bounded mPL (PBmPL) and additionally the work defines Adaptive mPL (AmPL) as a new mechanism for these settings. To validate their framework, they evaluate their framework on text embeddings to protect personally identifiable information (PII) and potentially identifiable information (PoII) and on Breast Cancer Wisconsin.
Critical strengths
- The work proposes privacy metrics that overcome some of the limitations of mDP. In particular, the work focuses on the case where the records given to the mechanisms are correlated.
- The introduction of the AmPL framework is a new and interesting privacy preserving strategy.
Critical weaknesses
- It is unclear in which realistic cases this vulnerability practically applies. mDP is often used for geo-indistinguishability or in the text case to protect tokens or attributes. Given the very small reported violation ratios (< ), I am not sure how significant this issue is in practice and if commonly used mDP mechanisms suffer from this issue. Can you give some practical cases in which this new approach is useful and practical?
- The AmPL framework seems to be using no provable protection against joint leakage. From my understanding the AmPL framework doesn’t imply bounded mLP or PBmPL. Additionally, the main privacy evaluations are done using posterior leakage violation ratio, which can be a reasonable additional metric, but I don’t think it is aligned with any standard privacy attacks, such as reconstruction or attribute inference (these appear only in the appendix and are not compared between EM and AmPL). Finally, there is no comparison and discussion with similar notions of privacy leakage and their corresponding mechanism (e.g. Pufferfish).
Questions for authors
- How would this framework perform under realistic attackers? Overall the empirical evaluation is limited to 3 settings and 1 mechanism (the Exponential Mechanism), however, a much wider range of mechanisms exists. Additionally as the current framework doesn’t give provable protection against joint leakage. It would be interesting to understand how it compares with the broader set of empirical defenses.
- What does bounded mPL add beyond a metric Pufferfish instantiation?