- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies one-shot private prototype classification with a frozen public encoder. Class sums and counts are released once with Gaussian noise. Equal sum-noise scale hurts rare classes: noisy means are less accurate and squared distances are inflated. PDC projects noisy means onto the feature ball and corrects distance scores for sum noise.
Critical strengths
- Simple, practical fixes with no extra privacy cost. PDC and PRDC are post-processing of a single zCDP release, so they are easy to adopt and compatible with any frozen encoder.
- Strong empirical gains and careful ablations. The paper isolates projection, correction, public-mean retrieval, and score correction. Gains are large at small budgets and consistent across datasets and imbalance ratios.
Critical weaknesses
- Modest methodological novelty. Projection and distance correction are standard tools. PRDC is essentially shrinkage toward a retrieved public mean. No comparison to a simple James-Stein estimator or other shrinkage baselines.
- Public-bank dependence is a real limitation. PRDC can underperform PDC when the public source is mismatched. The paper shows this but does not offer a reliable way to detect mismatch or fall back safely. Experiments are limited to image features and two public banks.
Questions for authors
- How does PRDC behave when the top-K retrieved public features are irrelevant to the class? Is there a safeguard or diagnostic?
- How sensitive are results to the public development tasks used to select K, the count-budget share, and other hyperparameters?
5 comments
R3.3: a criterion from public information and the release. We select between PDC and PRDC using only the public bank and released statistics, without additional privacy cost (Appendix D.8). Gaussian data fission [1] postprocesses the existing release into two observations with independent noise. We construct both classifiers from one observation and evaluate how often each assigns the class aggregates in the other to their own classes.
Let denote PRDC's average proxy accuracy minus PDC's. We choose
\begin{cases} \mathrm{PRDC},&\widehat\Delta>0,\\ \mathrm{PDC},&\widehat\Delta\le0. \end{cases}$$ These comparisons use the same training sets as Table 1, with new draws of privacy noise shared across methods. At IR 100, with the 20,000-image banks of Appendix D.7, BA (%) averaged over $\rho\in\{.005,.02,.125\}$ is: | Target / public bank | PDC | PRDC | Selection | | --- | --- | --- | --- | | CIFAR-10 / ImageNet | 85.45 | 88.66 | 87.86 | | CIFAR-10 / UECFood | 85.45 | 75.55 | 85.43 | | CIFAR-100 / ImageNet | 36.80 | 41.60 | 41.60 | | CIFAR-100 / UECFood | 36.80 | 27.82 | 36.71 | | FOOD-101 / UECFood | 41.95 | 45.88 | 45.88 | | FOOD-101 / ImageNet | 41.95 | 35.65 | 41.55 | Across the three datasets, both public sources, and both IRs, selection retains 95.8% of PRDC's gains and removes 97.7% of its losses relative to PDC over these budgets. Table 24 reports the full results. On FOOD-101/ImageNet at IR 10 and $\rho=.005$, selection remains 1.17 points below PDC. Developing a more accurate fallback criterion is left to future work. **R3.4: additional tasks and public banks.** We added 20 Newsgroups for text classification, iNaturalist 2018 as a naturally imbalanced image task, and COCO as an additional public bank for CIFAR-10/100 and FOOD-101. The two new tasks support the gains of PDC and PRDC observed on the original datasets (Appendices D.10 and D.11). BA (%) averaged over $\rho\in\{.005,.02,.125\}$ is: | Task | Noisy mean | PDC | PRDC | | --- | --- | --- | --- | | 20 Newsgroups (IR 10) | 21.81 | 32.22 | 44.69 | | 20 Newsgroups (IR 100) | 13.81 | 17.60 | 29.90 | | iNaturalist 2018 | 2.73 | 7.05 | 9.91 | With COCO, PRDC also improves BA over PDC on all three datasets at IR 100, averaged over the same budgets: | Dataset | PDC | PRDC / COCO | | --- | --- | --- | | CIFAR-10 | 85.61 | 88.44 | | CIFAR-100 | 36.63 | 39.21 | | FOOD-101 | 41.93 | 42.84 | These gains support using a general-purpose public collection when retrieval provides examples relevant to the target classes (Appendix D.12). **R3.5: irrelevant neighbors.** PRDC can still outperform PDC after removing public classes matched by name to the target classes. We select 25%, 50%, 75%, or 100% of the target classes and remove all samples from the matched public classes. We compare against random removal of the same number of public samples (Appendix D.7). Results are averaged over $\rho\in\{.005,.02,.125\}$ at IR 100. For FOOD-101 with 50% of the target classes selected, class recall decreases by 1.49 points on the selected classes and 0.59 points on the remaining classes relative to random removal. The changes are smaller on CIFAR-10/100 (Table 23). **R3.6: public development and hyperparameters.** The original submission includes hyperparameter sensitivity results in Appendices D.5 and D.6. We choose $K$ on public development tasks and report the full $K\in\{16,64,256,2048\}$ sweep, with the public choices marked in Figure 7 of the revision. At IR 100, BA averaged over all six budgets varies by 1.25 percentage points across these four values on CIFAR-100 with ImageNet and by 2.84 points on FOOD-101 with UECFood-256. The publicly selected $K$ is within 0.71 and 0.49 points of the best value, respectively. The count-budget share follows Equation (4) and does not depend on the public development tasks. Appendix D.5 varies this share from half to twice its prescribed value while keeping the total privacy budget fixed. Across this range, mean BA changes by less than 0.4 percentage points for each method and dataset. The PRDC weight is determined by the noise variance and released count through Equation (10). **References** [1] James Leiner et al. Data Fission: Splitting a Single Data Point. Journal of the American Statistical Association, 2025.Thank you again for your time and comments. We have posted our responses and uploaded the revised paper. Before the discussion closes, we would appreciate hearing whether our responses address your concerns and which points still need clarification. We would be happy to discuss further.
Thank you for your comments on the method and its evaluation. We believe that addressing the reviewers' comments has substantially improved the paper.
We have uploaded a revised manuscript incorporating the changes described in our responses. All equation, section, table, and figure numbers in our responses refer to this revision.
We welcome further questions or comments. If our responses resolve your concerns, we would be grateful if you could reconsider your score.
R3.1: methodological contribution. Our contribution is to couple prototype regularization and distance correction for classification under class-dependent privacy noise. Recent work continues to develop debiasing methods for functions of private releases [1, 2]. For imbalanced classification, correcting the expected distance penalty still leaves large score fluctuations for small classes, while regularization changes the correction required.
PDC constrains the noisy prototype to the feature ball and calibrates its score correction to the known DP noise scale and the projection factor. PRDC also sets its private–public weighting from the class-specific noise variance and scales the correction accordingly. Section 4 bounds the remaining sum/count fluctuations, separates them from the signed score shifts due to regularization, and gives conditions for preserving or changing the clean class decision. Figures 2 and 3 isolate the effects of regularization, correction, and their combination using the same private release. These simple updates improve classification without further private access or iterative training.
On CIFAR-100 and FOOD-101 at IR 100, averaged over all six budgets, PDC gives 51.1% and 55.9% BA, compared with 33.5% and 38.5% for Projection only and 19.1% and 26.3% for Correction only (Table 13).
The improvements are substantial against both private learning and shrinkage baselines. On CIFAR-10, CIFAR-100, and FOOD-101 at IR 100, averaged over ρ∈{.005,.02,.125}, PRDC improves BA over DP-SGD+LA by 4.30–5.72 percentage points (Table 1). In the additional comparison in R3.2, PDC exceeds James–Stein by 10.64–12.17 points and soft thresholding by 16.18–19.69 points, with distance correction applied to both comparators. PRDC further improves PDC by 0.64–4.73 points in that comparison.
Recent work uses public data to improve differentially private learning [3, 4]. Access to a public bank is thus a realistic assumption for PRDC. To address accuracy loss from a mismatched source, we additionally select between PDC and PRDC using the public bank and released statistics, without additional privacy cost (R3.3, Appendix D.8). Across three datasets, both public sources, and IRs 10 and 100 at the above budgets, this rule retains 95.8% of PRDC's gains and removes 97.7% of its losses relative to PDC.
R3.2: shrinkage comparison. The submitted ablation already includes projection-based shrinkage (Projection only). We added comparisons with James–Stein and coordinatewise soft thresholding [5]. All methods share new draws of privacy noise on the training sets of Table 1, with public banks screened as in Appendix C.2. We apply distance correction to both baselines and select the soft-thresholding parameter on public data. BA (%) averaged over ρ∈{.005,.02,.125} is:
PDC and PRDC outperform both baselines at IR 100. Appendix D.2 provides the full results and experimental details.
References
[1] Flavio Calmon et al. Debiasing Functions of Private Statistics in Postprocessing. FORC, 2025.
[2] Quentin Hillebrand et al. Privacy by Postprocessing the Discrete Laplace Mechanism. NeurIPS, 2026.
[3] Adam Block et al. Oracle-Efficient Differentially Private Learning with Public Data. NeurIPS, 2024.
[4] Jinseong Park, Yujin Choi, and Jaewook Lee. In-distribution Public Data Synthesis with Diffusion Models for Differentially Private Image Classification. CVPR, 2024.
[5] David L. Donoho. De-noising by soft-thresholding. IEEE Transactions on Information Theory, 1995.