- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper considers the question of universal privacy, where we seek a learned representation of an input that preserves privacy across all potential sensitive attributes while retaining utility. The paper argues that (1) Releases that maintain privacy at least as much as all other releases at its utility or better only exist at zero and full sufficiency (using known arguments in a privacy setting); (2) Matching privacy across attackers/attributes at any level below maximum utility carries no information (exactly for continuous data, up to a vanishing margin for finite data); and contains additional analysis for binary vs. multiclass decisions and hard vs. learned quotients. Proofs are supplied, and a variety of experimental settings are used to verify the findings.
Critical strengths
- The paper is structured well, and plots and tables are visually appealing and clear. Related work, both in the specific section and throughout the paper, is thorough (though I cannot verify if citations were missed, as I am not an expert in this field).
- Proofs are supplemented with reproducible numerical verification on random instances; The experimental setup is clear and much information is provided for reproducibility.
Critical weaknesses
- While some scoping is provided, more could be done to situate this stream of information-theoretic inference privacy research in the broader realm of what generally falls under "ML privacy". Does the notion of 'privacy' under this threat model encompass what other related fields would consider a 'private' model?
- Some clarification on experimental setup choices is needed (see Questions).
Questions for authors
- In the experiments, could the authors please share how they came to their choices of model architectures, attackers, and inversion procedure? Is there a risk of attacks not being strong enough in any case, as the theorems concern idealized attackers?
- Are citations needed for the classicial results in Proposition 1 and Theorem 1(a)? Where can more uninitiated readers (like myself) learn more about these results in their original (non-privacy) settings?
- Could the authors please clarify the use of the word 'phase transition' to describe the phenomenon of Theorem 1? Does this phrase have a history of use in information-theoretic contexts? Are there aspects of a physical phase transition that are not present in this setting?
3 comments
Thank you for your thorough response to myself and other reviewers. I have updated my score.
Thank you for taking the time to review our revisions and update your assessment. We sincerely appreciate your thoughtful feedback and constructive suggestions, which have helped us improve the paper. Thank you again for your time and consideration.
Reply to Reviewer jSBF: notion of privacy, attackers, citations, terminology
Thank you for your review.
W1. This notion of privacy. The threat model is attribute inference: an attacker who knows the data distribution sees the release of one input and infers attributes of that input. The guarantee is comparative, as in semantic security. Scope now says what this covers. It does not cover membership inference or extraction of training data. Differential privacy gives individual-level guarantees that can limit those attacks. The result is about being safest among equally useful releases. It does not rule out stand-alone, target-agnostic guarantees based on differential privacy, bounded leakage, computational assumptions or restricted target classes. Related work now discusses Stadler et al. (ICML 2024), who formalise least-privilege learning and establish utility-leakage limits for representations, while we study the ordering of releases and whether a universally safest one exists.
Q1. Architectures, attackers and inversion; strength of the attacks. We applied one rule: a small standard architecture per modality (App. F: an MLP, a two-layer GCN, a two-block CNN and a mean-of-word-embeddings network), and a single attacker and inversion procedure for all releases. The attacker is one MLP, retrained from scratch against each release. The architecture and training hyperparameters are fixed across releases; only the stopping epoch is selected separately for each attribute on a validation split. A trained attacker gives the quantity a practitioner can audit. The original inversion attack is one fixed MLP decoder followed by nearest-neighbour identification in a gallery.
Attacks can be too weak, and this limits what measured leakage levels mean. In Section 5 and Appendix I we attacked continuous releases again with a linear probe, one independent MLP per target, k-NN and gradient-boosted trees, and discrete codes with a contingency table (the Bayes attacker up to estimation error). On continuous releases, validation picks our attacker for 23-70% of (release, attribute) pairs, and one of the two MLPs for 52-92%. With the strongest attacker (chosen by validation per release and attribute), the posterior does not leak more than the embedding on any attribute of 7 of the 9 datasets (3 seeds). On Cora and CiteSeer its mean leakage is lower than the embedding's by 0.042±0.020 and 0.054±0.013 of the entropy (10 seeds, Table 11).
We also added stronger re-identification attacks (Table 8): a larger decoder, a convolutional decoder, and a CNN that learns to map images to releases and matches the observed release in the gallery. The original attack understated the embedding, which is re-identified at 100%, 100% and 96% among 1,000 test images of MNIST, Fashion-MNIST and CIFAR-10 (93%, 61% and 22% in Table 1). Every hard code stays within its certificate ∣c∣/n, and the predicted label stays at 1.0%, its exact Bayes rate. The posterior and VIB have no certificate, and their rates depend on what the attacker knows. A learned attack, which only sees image-release pairs from the attacker split, reaches 1.8-4.5% and 7-17%. An attacker who knows the encoder can compute the release of each gallery image and re-identifies every posterior at full precision. The Table 1 caption now says this, and it is why the paper recommends hard codes.
The posterior and hard codes are post-processings of the embedding, so their true leakage cannot exceed the embedding's, whatever the attacker. Measured levels depend on the attacker, as the paper now states (Table 4, linear probe on MNIST).
Q2. Citations for Proposition 1 and Theorem 1(a). Yes. They now appear at the statements. Proposition 1 is Blackwell's theorem (Blackwell, 1953) applied to attributes. For the original setting, see Le Cam (1964) and Torgersen (1991) on the comparison of experiments. Theorem 1(a) is the minimal sufficiency of the quotient (Bahadur, 1954).
Q3. "Phase transition". We mean a sharp threshold. By Theorem 1, a universally optimal release exists only at utility 0 or I. By Theorem 2, on inputs continuous within each class, the best universally safe utility is 0 for every τ<I and jumps to I at τ=I. On finite data the margin vanishes as the data grow. The term is used in a similar way for the Information Bottleneck (Wu and Fischer, ICLR 2020). Unlike a physical transition, there is no order parameter or critical scaling. Section 4.1 now says this.