- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
The paper measures information disclosed by a forecast (which is the predicted probability distribution) beyond the population posterior (the true probabilities) needed for optimal prediction. This disclosure is measured by excess disclosure which the largest improvement in the best expected payoff from observing the released probabilities over the true ones.
A construction shows that it is possible to infer sensitive information from a model that is perfectly calibrated and almost Bayes-optimal. The paper proves a sharp privacy-utility-tradeoff (excess disclosure vs log-loss regret), and provides a method that attains this bound.
Critical strengths
- The paper demonstrates that perfectly calibrated, near Bayes-optimal probabilities can reveal sensitive information.
- The paper provides a framework to measure excess disclosure and proves a sharp bound on the achievable privacy-utility tradeoff.
- The paper proposes a randomized mechanism that attains the sharp asymptotic bound and provides empirical evidence supporting the effectiveness of randomized releases in reducing sensitive information disclosure.
Critical weaknesses
- The population Bayes risk is uknown in practice, and at , certifying disclosure at most requires a summed task-regret bound of approximately nats (Discussion; Section 7). The experiments provide no such bound, leaving release-size selection disconnected from certification.
- The empirical evaluation is narrower than the theoretical framework. The medical experiments measure reductions in patient-verification AUROC for a selected family of classifiers, rather than the optimal recipient's excess disclosure relative to the population posterior.
Questions for authors
- What observable quantity could guide release size when Bayes regret is unknown? Can the required regret bound be estimated empirically?
- Could the authors compare the empirical excess disclosure observed in their controlled experiments with the theoretical worst-case bounds across different regret levels and release mechanisms?
2 comments
Thank you for the detailed response and revisions.
You have answered my question by introducing a new approach that avoids the unknown Bayes risk. However, your original sampled count method still cannot provide practical privacy certification without knowing the model's regret.
My assessment of the theoretical contributions remains positive, although practical certification remains an important limitation. I therefore maintain my original rating.
Thank you for the review.
1. Observable guidance when Bayes regret is unknown. At n = 256, the guarantee needs a summed task-regret certificate of about 1.95×10−5 nats to bound disclosure by 0.05. Validation loss estimates forecast risk. A useful regret bound also needs information about Bayes risk or model approximation. Appendix A.21 constructs a certificate for a fixed finite predictor class, a probability floor, independent target-population samples, and known approximation and optimization errors.
Proposition A23 supplies an observable alternative: average total-variation distance between a forecast's release law and a record-independent reference bounds excess disclosure. We apply it to selective full-precision release with an erasure fallback. Average acceptance at most 0.05 certifies disclosure at most 0.05, including information from the acceptance flag.
With per-record acceptance capped at 0.25, this policy retains 15.02% and 13.76% of the full-precision logistic recipient's log-loss gain over the prior on BRSET and NIH. The certificate covers all tasks jointly for one uniformly sampled image from the fixed cohort and one fixed model (Section 6.6; Appendix G.1; Table A29). Appendix A.23 gives a prospective version using fixed policies and independent deployment forecasts.
2. Controlled disclosure compared with the bounds. Figure 1 compares optimal bit-inference gain with the finite bounds at six regret levels for counts and angular noise. Count gains lie well below their bounds. Angular-noise gains follow their bounds more closely. Appendix B.2 compares measured gains with the bounds at fixed parameter settings.
Appendix B.3 evaluates the full bounded-decision criterion on a four-state calibrated population. At n = 256, sampled counts give excess disclosure 0.44651 under the full criterion. Restricting the decision to classification of the example's sensitive bit gives an accuracy gain of 0.24969 over the posterior baseline. The count-release upper bound is 1. The optimal incremental prediction cost is 0.001121 nats. Thus, the selected bit understates disclosure, and the worst-case bound is loose on this population.
The medical experiments measure fitted verification AUROC and prediction loss. The unknown posterior may itself permit linkage, so these measurements do not identify excess disclosure or population-optimal prediction cost (Section 6).