- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
The paper studies the problem of reducing group-level accuracy disparities in differentially private learning without introducing additional adaptive clipping mechanisms or privacy-consuming calibration. Its main idea is the combination of an offline configuration-screening method with a group-information score.
Critical strengths
S1. The paper considers important DP-SGD parameters jointly and in particular recognizes that batch size, clipping, noise, and learning rate jointly influence training dynamics and privacy.
S2. The proposed cumulative group-information score, together with the PAN test, provides a solid and systematic way to compare configurations under fixed privacy and epoch budgets.
Critical weaknesses
W1. My main concern is about methodological novelty. The proposed EquiConfig primarily screens existing DP-SGD configurations using an analytically constructed score before applying a conventional privacy-accounted validation procedure. I do not see this a fundamentally new framework. The authors should distinguish their contributions from existing privacy-aware hyperparameter optimization, analytical configuration selection, and fairness-aware DP-SGD tuning methods.
W2. The paper's motivation is that DP-SGD may disproportionately harm underrepresented groups, and EquiConfig seeks configurations that improve the accuracy–disparity trade-off. However, DP-SGD does not necessarily impose a disproportionately larger accuracy loss on minority groups. Whether it does so depends on the dataset, model architecture, group characteristics, privacy budget, and training hyperparameters. The accuracy drop of minority group due to DP-SGD (compared with SGD) could be similar or even less than majority group. The paper conducts evaluation over MNIST, CIFAR-10, SST-2, QNLI, CelebA and Adult, some of which were modified from their original versions. I wondered how we know for sure the chosen dataset, model, and the hyperparameters incurred the disparate impact.
Questions for authors
Please see W2.