acceptodds
Under review as a conference paper at ICLR 2027

Differentially Private Mixing of Public Datasets Improves Private Learning

Abstract

Many machine learning applications involve sensitive data and therefore require training under differential privacy (DP). However, DP training often degrades model utility. In some cases, first pre-training the model on "public" data before finetuning with DP on the sensitive data can reduce the drop in utility. However, the success of this depends on how relevant the selected public dataset is to the sensitive data. We introduce the first pipeline that privately learns the mixture of several public datasets to pretrain on for a given sensitive downstream task. Our key insight is that we can privately find the best mixture of multiple public datasets by privately learning a low-dimensional linear model. We tested our method on the NIH dataset for X-ray classification and the ENRON email dataset for language modeling. Applying our method to find tailored mixtures of X-ray datasets to pretrain on for diseases in the NIH ChestX-ray14 dataset, we improved macro AUC by up to 0.037 across privacy budgets compared to the baselines, with gains as large as +22.8% relative AUC on Cardiomegaly at . For DP training on the ENRON dataset, pre-training on our mixture of The Common Pile (a collection of public-domain text datasets) decreased test perplexity by 16% relative to the baseline mixtures.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Differential Privacy, Trustworthy Machine Learning, Data Mixing

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_nJzyOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes DP-MixMin, a novel approach to select optimal data mixture for public pretraining before fine-tuning model with DP-SGD. Authors build on two insights from prior work: public pretraining helps downstream private fine-tuning, and data mixture optimization can be reduced to a convex optimization problem without expensive retraining on every candidate mixture. Authors propose an approach that guarantees end-to-end differenttial privacy by splitting the dataset into two parts: one for learning the mixture, another for training the target model. They then use DP-SGD at both stages.

Critical strengths
  • The paper is very clearly written, identifies a reasonable gap in the literature, positions itself well within the field and builds on the prior work. It is very clear what authors are doing, why, and what is the contribution.
  • The algorithm itself is well-designed. As far as I can tell DP claims are correct. Using disjoint datasets to avoid epsilon composition is a clever move.
  • Empirical results (especially Image Classification results) are very strong and convincing
Critical weaknesses

Some of the empirical claims look a little surprising (mostly for language modelling), and I would love to see better ablations and baselines to understand the nature of the phenomena.

For instance, Table 1 shows that for Enron experiments increasing ρ\rho leads to better downstream performance. Generally I would find that quite surprising -- privacy amplification by subsampling in DP-SGD usually has a strong impact of the downstream performance, and I would have expected that larger dataset in the second stage would lead to a better final model. We, however see the opposite.

Additionally, the fact that random search underperforms compared to the uniform version, even though random search omits privacy while exploring options, suggests that better hyperparameter search is possible -- it should recover uniform's performance at least.

At the very least I would like to see the following baselines for Enron data to understand the upper bounds in this setup

  • Non-DP mixture + non-DP training (upper bound)
  • DP-mixture + non-DP training (upper bound for the second stage)
  • non-DP-mixture + DP training (upper bound for the first stage)

Additionally, a baseline that is better that uniform mixture should be possible, e.g. gradient-based alignment from Gu et al. should be applicable for language modelling. Also, method from Yu et al. looks applicable.

I would be happy to raise my score if authors show that DP-MixMin outperforms baselines stronger than uniformly mixing all available data sources.

Questions for authors
  • Why do you think random search does performs worse than the uniform mixture?
3 comments
AuthorsOpenReview ↗

Response to Reviewer nJzy

We thank the reviewer for their time and feedback! Below we discuss questions and concerns brought up by the reviewer.

Why do you think random search does performs worse than the uniform mixture?

Note that we selected the number of samples for random search, mm, to match compute used for MixMin. This results in m=4m=4 for the image classification tasks and m=16m=16 for the language modeling tasks; this is far smaller than needed to exhaustively search the simplex. In fact, note the uniform mixture minimizes worst-case distance to any mixture, so it can often be the case it is closer to the best mixture than the mm randomly selected mixtures. Past work had similarly found that at small compute budgets for mixture selection, a uniform baseline outperformed all data mixing methods except MixMin (see the “Balanced” column in Figure 3 of Thudi et al., 2025 [1]).

[1] Thudi et al., "Mixmin: Finding data mixtures via convex minimization." International Conference on Machine Learning, 2025

For instance, Table 1 shows that for Enron experiments increasing ρ\rho leads to better downstream performance. Generally I would find that quite surprising -- privacy amplification by subsampling in DP-SGD usually has a strong impact of the downstream performance, and I would have expected that larger dataset in the second stage would lead to a better final model. We, however see the opposite.

We actually found that most of the difference in performance comes from the pretraining stage, and so by increasing ρ\rho we get a pre-training mixture closer to optimal. Below are the downstream Enron perplexities (mean ± std over 3 trials) of just the pre-trained model, before fine-tuning:

  1. DP-MixMin ( ϵ=1\epsilon=1, ρ=0.3\rho =0.3): 17.83 ± 1.0135
  2. Uniform: 21.154 ± 1.592

Note that DP fine-tuning reduced the loss for uniform by just under 7% (to perplexity 19.88) , whereas the relative gap between uniform and DP-MixMin after pre-training was already 15%.

At the very least I would like to see the following baselines for Enron data to understand the upper bounds in this setup: Non-DP mixture + non-DP training (upper bound), DP-mixture + non-DP training (upper bound for the second stage), non-DP-mixture + DP training (upper bound for the first stage)

Thank you for that suggestion! Below are the perplexities we obtained from one seed:

  1. Non-DP MixMin pre-training mixture + non-DP finetuning: 4.49
  2. DP MixMin pertaining mixture + non DP finetuning: 4.49 (note we confirmed the pretrained model was slightly different than the Non-DP, probably more seeds would show difference between 1 and 2)
  3. Non-DP MixMin pretraining mixture + DP finetuning: 17.0

We had already found DP MixMin mixture pretraining + DP finetuning had a perplexity of 16.78 ± 0.55, which is within one standard deviation of our one-seed Non-DP MixMin pretraining + DP finetuning. So the DP MixMin mixture already matches the “upper-bound” for the first stage (data mixing). In the paper we already described how the mixtures found by DP-MixMin and non-DP MixMin were comparable (Section 5.3), and the above results confirm any slight differences in mixtures did not lead to significant differences in performance for language modeling.

I would be happy to raise my score if authors show that DP-MixMin outperforms baselines stronger than uniformly mixing all available data sources.

As mentioned in the above response, DP-MixMin already recovers the non private MixMin mixtures, which is the best known data mixing procedure (as far as we are aware). Comparisons in mixture weights and final training performance for image classification are described in Section 5.3 with results in the Appendix. We additionally included direct performance comparisons for language modeling in our earlier response and found they were comparable. So we have shown that DP-MixMin matches the “upper-bound” for data mixing performance, and can do so despite imposing DP guarantees; we attribute this to the problem being a convex learning problem, which comes with relatively little utility degradation when doing DP [2]. Let us know if this did not address the concerns.

[2] Bassily, et al. "Private stochastic convex optimization with optimal rates." Advances in neural information processing systems, 2019

I thank the authors for the response. It covers my concerns and I will raise my score

AuthorsOpenReview ↗

Response to Reviewer nJzy

Thank you for raising your score!

We have also implemented both of the other suggested baselines, and ran each through the same pipeline as DP-MixMin: pretraining GPT-2 medium from random initialization on 7.1B CommonPile tokens, followed by user-level DP fine-tuning at ϵ=1\epsilon = 1 and δ=1e−5\delta = 1e-5 on the same 84 fine-tuning users, with ρmixing=0.3\rho_{mixing} = 0.3 so 70% is left for fine-tuning, over two data splits.

We found DP-MixMin also beats these baselines for language modeling (Gu et al. [3], and Yu et al.,[4]), with no overlapping error bars: see the table below. Note we also applied Gu et al [3] without DP guarantees, favoring them. See Appendix A.11 for details on implementation.

Enron test perplexity (mean ± std, lower is better)

MethodSeedsAfter pretrainingAfter DP fine-tuning
DP-MixMin (ρ=0.3\rho = 0.3)317.83 ± 1.0116.78 ± 0.55
Uniform321.15 ± 1.5919.88 ± 1.10
Yu et al. (private)219.94 ± 1.2419.11 ± 1.07
Gu et al., GSD on github_archive (all non-test users)227.68 ± 0.0926.38 ± 0.08
OpenAI GPT-2 medium (all non-test users)321.5 ± 1.4720.18 ± 1.28

[3] Gu et al., "Choosing Public Datasets for Private Machine Learning via Gradient Subspace Distance." IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), 2025.

[4] Yu et al., "Selective Pre-training for Private Fine-tuning." arXiv:2305.13865, 2023.

Reviewer_f4s3OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces a private version of a pre-training data selection algorithm. It is based on an existing non-private algorithm called MixIn. Their method significantly improves utility on challenging real-world datasets. The method is much more data-efficient compared to existing DP methods based on private hyperparameter search. The method privately minimizes an (convex) objective that results in an efficient mixture of public datasets that can then be used for public pretraining needed for fine-tuning a DP model.

Critical strengths
  1. The utility improvements the paper shows are very impressive (for example, several percentage points in accuracy).
  2. The method is evaluated using sample-level and user-level DP, both showing utility improvements.
  3. The experiment baselines are well designed, especially uniform pretraining and random search makes sense.
  4. The experiments use real-world medical datasets (as opposed to standard weak benchmarks, such as CIFAR-10). Furthermore, the experiments also include natural language tasks.
Critical weaknesses
  1. The novelty is somewhat limited; the authors just privatize an existing implementation. However, the empirical results significantly improve their baselines.
  2. The noise injection in Algorithm 1 (Line 9) is not properly explained (e.g. why does not the noise variance depend on CC?). This is critical part of DP and deserves a careful explanation.
  3. The method claims end-to-end privacy, but yet the hyperparameter search is not private.
  4. The method requires more private data than standard methods, as some if it is used for selecting the mixture. The paper should clarify how much more data this requires.
Questions for authors
  1. In Algorithm 1, it's not clear how much noise is injected to the model. The noise variance σ\sigma is not given as an input to the model. How is σ\sigma calculated? I am confused, as the injected noise does not include any contribution of CC or BB.
  2. Can you add a proper caption to Algorithm 1?
  3. How are the hyperparameters chosen for the baselines?
  4. How do you know you reached the Bayes optimality requirement?
4 comments
AuthorsOpenReview ↗

Response to Reviewer f4s3

We thank the reviewer for their time and feedback! Below we discuss questions and concerns brought up by the reviewer.

In Algorithm 1, it's not clear how much noise is injected to the model. The noise variance σ\sigma is not given as an input to the model. How is σ\sigma calculated? I am confused, as the injected noise does not include any contribution of CC or BB.

Thank you for catching this. In Algorithm 1, σ denotes the DP-SGD noise multiplier, and the added noise should be z∼N(0,σ2C2I)z \sim N(0,\sigma^2C^2I). The factor C2C^2 was missing from line 9, and we have fixed this in both Algorithm 1 and Algorithm 2 (user-level). Our σ\sigma is computed with the Opacus RDP accountant for composing the sampled Gaussian mechanism [1,2] from the target (ϵ,δ\epsilon,\delta), the sampling rate q=B/∣Dλpriv∣q= B / |D_{\lambda}^{priv}| and the number of steps nn. CC does not enter this computation; it only scales the noise. Previously this was stated only in a footnote (page 6), but we now incorporated the footnote into the main text, under the “Making MixMin Differentially Private” paragraph. We have also edited Algorithm 1 to have the DP-SGD accountant as an explicit input and make the second line of the algorithm Accountant((ϵ,δ),q,n(\epsilon,\delta),q,n). We also make the mini-batch sampling explicit (batch size B, as in our experiments, Appendix A.3.1): we updated line 10 to state gsumg_{sum} is summed over the sampled batch and normalized by BB rather than ∣Dλpriv∣|D^{priv}_{\lambda}|.

[1] Mironov et al., 2019: https://arxiv.org/abs/1908.10530 [2] Yousefpour et al., 2021: https://arxiv.org/abs/2109.12298

Can you add a proper caption to Algorithm 1?

We expanded the algorithm caption to “Optimizing the MixMin objective privately by a post-processing of DP-SGD updates.” We also implemented edits to Algorithm 1 to make the σ\sigma calculation and use explicit (see our previous response). Let us know if there is anything else you think would help with clarity.

How are the hyperparameters chosen for the baselines?

All methods share the same training hyperparameters (Appendix A.3) except the pretraining epochs for image classification (i.e., early stopping), and were tuned on the Uniform baseline; we have updated Appendix A.3 to be more clear about this. In particular, for NIH the batch size, fine-tuning steps, learning rate and clipping norm were tuned once for Uniform pre-training on a single pilot configuration using a held-out validation set (Atelectasis, CheXpert, ε = 1, validation set was 6000 samples), then held fixed across all methods, tasks, seeds and privacy budgets (l. 765–773). Pre-training epochs were tuned for each task and pre-training mixture using the held-out validation set. The language-modelling settings are likewise shared by all methods (l. 781–788), and used a held out validation set that was 2% of the whole dataset. Beyond these, Uniform and Random Init have no extra hyperparameters. Random Search has only the number of candidates mm, which we set to match or exceed DP-MixMin’s compute (Appendix A.10), and it selects the best candidate non-privately which favours the baseline. The only method-specific tuning is DP-MixMin’s mixture-learning step (clipping norm, batch size and step size, l. 760–764) which was selected using the same held out validation set as the training parameters.

How do you know you reached the Bayes optimality requirement?

While we do not empirically know we reached Bayes optimality for proxy models, the original MixMin paper conducted both a formal and empirical study into how performance degrades with errors in the proxy models. It was formally shown that the error from using imperfect proxy models degrades smoothly (see Lemma 3.2 in the MixMin paper [3]). Empirically, the MixMin paper also conducted ablations where they severely under-trained the proxy models (from 100% to <1% the compute for a single chinchilla optimal training run) and saw the performance was stable even for severely under-trained models across language modeling tasks: see Figures 3,9,10 in [3]. Note, our language proxy models are also severely undertrained, using 3.5% the compute relative to our final pre-training run, and yet we also still outperform the baselines.

To further support the stability of DP-MixMin to under-trained proxy models, but now for image classification, we ablated the compute used for our image classification proxy models. See Table 3 in the Appendix where we show final performance (i.e., after fine-tuning the pretrained model on the DP-MixMin mixture) on Atelectasis was stable across proxy model training budgets. In Table 5 we also show across the 14 NIH tasks the mixtures found by DP-MixMin were stable across proxy model training epochs (often cosine similarity >0.9>0.9).

[3] Thudi, Anvith, et al. "Mixmin: Finding data mixtures via convex minimization." International Conference on Machine Learning, 2025

AuthorsOpenReview ↗

Response to Reviewer f4s3 Continued

The method requires more private data than standard methods, as some if it is used for selecting the mixture. The paper should clarify how much more data this requires.

The entire DP-MixMin pipeline uses the same amount of private data as our other baselines. In particular, we found that splitting the private data into a small set for finding a data mixture and a larger set for fine-tuning was preferable to using all the data for fine-tuning. However, we agree future work could further push the data efficiency of DP learning and data mixing. Towards this, in tables 11-16 in the Appendix we ablated the ρ\rho fraction used for mixing and report how the DP-MixMin mixtures converge to the non-private MixMin mixture (which also used the whole private set). We found often after ρ>0.1\rho >0.1 for ϵ>1\epsilon > 1 we had cosine similarity >0.9>0.9 to the non private MixMin weights; the main exception is user-level DP on Enron, and future work may study how its trade-off could be improved. We have some preliminary thoughts in predicting this in Appendix A.2.

Could the authors still expand on the end-to-end privacy claims? This was also noted as a weakness by reviewer pLSY.

AuthorsOpenReview ↗

Response to Reviewer f4s3

Our pipeline (DP data mixing + finetuning) satisfies DP guarantees with respect to the dataset used for mixture finding and fine-tuning, for a given set of training hyperparameters: e.g., the learning rate, batch size, number of steps for pre-training and fine-tuning.

However, the performance of the baselines, and our method, is sensitive to the choice of training hyperparameters. So in order to control for the effect of training hyperparameters vs. the effect of the pre-trained mixture, we used a disjoint small validation set to tune these parameters, and treated it as public . This does not affect the DP guarantees for the rest of the task data (the mixture finding dataset and fine-tuning dataset), but means this small validation set does not have DP guarantees. In the case of image classification, this set was 6000 samples (5.3% of the original dataset), and for language modeling this was always a set less than 2% of the original task dataset: see Appendix 3 for details on these set sizes.

So, our stated DP guarantees are with respect to the dataset used for mixing and fine-tuning, but not with respect to the much smaller set used for hyperparameter tuning. Future work could apply differential privacy to also the selection of training parameters (I.e. DP-SGD hyperparameters); this can be done by applying existing work on private hyperparameter selection [1], and composing the guarantee using parallel composition as the validation set is disjoint (so we just need the (ϵ,δ)(\epsilon,\delta)-DP of hyperparameter selection to be less than mixture finding and fine-tuning).

In the end, we wanted to focus on how, given these hyperparameters, our mixtures outperformed baselines. We agree we can make how we used a validation set clearer. We updated sections 5.1.1 and 5.2.1 to mention that we treat the validation sets used for tuning training hyperparameters as public. We also added the following to the opening blurb of section 5 to highlight this:

“ In order to control for the effect of hyperparameters versus pretraining mixtures, we assumed a small public validation set to tune hyperparameters (see Appendix A.3)”

We also state the following in the conclusion as part of our list of limitations:

“Furthermore, in our experiments, we tuned training hyperparameters assuming a public validation set: future work could apply private hyperparameter selection techniques alongside parallel composition to privatize this and achieve fully end-to-end differentially private deployments.”

Let us know if this answered your question.

[1] Nicolas Papernot and Thomas Steinke. “Hyperparameter tuning with renyi differential privacy”. In International Conference on Learning Representations, 2022

Reviewer_pLSYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper is about choosing, under DP, how to weight kk public pretraining sources for a given sensitive downstream task. It builds on MixMin (Thudi et al. 2025), where near Bayes optimality the mixture problem becomes a convex problem in the weights λ∈Δk\lambda \in \Delta_k over per-source proxy models f^i\hat f_i. DP-MixMin splits the private data into a piece DλprivD^{\mathrm{priv}}_\lambda (fraction ρ\rho) for learning λ\lambda and a piece DftprivD^{\mathrm{priv}}_{\mathrm{ft}} for finetuning, runs entropic mirror descent on λ\lambda with per-example clipping and Gaussian noise (Algorithm 1; user-level variant in Algorithm 2), pretrains on the learned mixture and DP-finetunes on DftprivD^{\mathrm{priv}}_{\mathrm{ft}}; parallel composition gives (max⁡iεi,max⁡iδi)(\max_i \varepsilon_i, \max_i \delta_i). Lemma 4.1 shows that private hyperparameter search over random mixtures needs ε=Ω(k)\varepsilon = \Omega(k) on a hard instance while DP-SCO on the convex objective needs ε=O(k)\varepsilon = O(\sqrt{k}). Experiments: the 14 NIH ChestX-ray14 tasks with the 14 CheXpert or MIMIC-CXR label columns as sources (ResNet-50 proxy, ResNet-18 downstream), macro AUC +0.029+0.029 to +0.037+0.037 (CheXpert) and +0.024+0.024 to +0.033+0.033 (MIMIC) over uniform pretraining at ε∈{1,2,5}\varepsilon \in \lbrace 1, 2, 5 \rbrace; Enron under user-level DP with 16 Common Pile domains (GPT-2 Small proxies, GPT-2 Medium downstream), perplexity 16.7816.78 vs 19.8819.88 for uniform at ε=1\varepsilon = 1; OpenWebMath 12.2212.22 vs 14.0514.05. There are also recovery experiments (cosine similarity ≥0.93\ge 0.93 to non-private MixMin at ε=1\varepsilon = 1, downstream AUC within 0.0060.006 of it), canary audits (εLB≤0.131\varepsilon_{\mathrm{LB}} \le 0.131), a proxy-quality ablation and an empirical k\sqrt{k} scaling check.

Critical strengths

The idea is simple and the paper shows it is nearly free. Mixture selection becomes a kk-dimensional convex problem over frozen public proxies, so the private part is cheap, and because of parallel composition it costs data rather than budget. The recovery experiments then show that the private weights are essentially the non-private ones (cosine 0.930.93 to 0.970.97 at ε=1\varepsilon = 1, Table 11) and the end-to-end pipeline is within 0.0010.001 to 0.0060.006 macro AUC of non-private MixMin (Tables 18, 19). The canary audit (Tables 16, 17) is something most papers in this area skip, and I was glad to see it.

The gains hold up across modalities and budgets, including the hardest case: user-level DP on Enron with about 35 users for mixing and roughly 80 for finetuning, where the 16% perplexity drop at ε=1\varepsilon = 1 is well outside the spread over seeds. OpenWebMath gives 13%. The ρ\rho ablation (Table 1 / Fig. 6) and the proxy-epochs ablation (Table 3) are the right checks to have run.

Critical weaknesses
  • The vision baselines are weaker than what people actually do, so the X-ray gains may be inflated. Pretraining uses a scalar output regressed onto the λ\lambda-weighted label average ∑c=114λcyc\sum_{c=1}^{14} \lambda_c y_c (Appendix A.3.1). For the uniform baseline that means predicting the mean of 14 binary labels, which is not how anyone pretrains on CheXpert or MIMIC. Three baselines are missing: standard 14-way multi-label pretraining on the whole public set; the matched-label single source (Table 9 shows 7 of the 14 NIH classes have a direct match, and the biggest win, Cardiomegaly at 0.8720.872 vs 0.7100.710 in Table 7, is one of them); and private selection of one source via the exponential mechanism over k=14k = 14 candidates, which is the cheap private competitor here, much more so than random search. Also, the vision "sources" are label columns of a single image set, so this is label-space mixing, a narrower thing than mixing public datasets. Only the language experiments test heterogeneous corpora.

  • End-to-end privacy is leakier than the paper suggests. Pretraining checkpoints (3/6/9 epochs) were picked by downstream validation AUC on the private validation split, and the clipping norm, batch size and learning rate were tuned on a pilot NIH task, as far as I can tell without any accounting (A.3.1). This is common practice, but it sits oddly in a paper whose selling point is a single end-to-end guarantee and which audits the mixing step. Smaller things: mm for Random Search is never given, and RS underperforms uniform in Tables 7 and 8, which is strange for a baseline that selects non-privately; Table 1 says std 0.550.55 at ρ=0.3\rho = 0.3 while Section 5.2.2 and the Fig. 6 caption say 0.450.45; the sequential vs. parallel composition comparison (Table 6) is one class with three seeds.

Questions for authors
  1. Can you add the three baselines above (multi-label pretraining, matched-label single source, exponential-mechanism source selection), at least for CheXpert →\to NIH at ε=1\varepsilon = 1? If the gap to the single-source baselines turns out small the paper still has a contribution, but it should be framed as "mixtures beat the best single source by XX".

  2. Is the NIH validation split treated as public? If not, how do checkpoint selection and hyperparameter tuning fit the stated (ε,δ)(\varepsilon, \delta)?

  3. What mm was used for Random Search in each setting, and why is it worse than uniform? For Enron, what is the test perplexity of each pretrained model before any DP finetuning? With about 80 users at ε=1\varepsilon = 1 I suspect most of the gap is already there before finetuning.

  4. Does the no-covariate-shift assumption (di(x)=dj(x)d_i(x) = d_j(x)) need to hold only among the public sources, or also between public and private inputs? CheXpert and NIH images are not the same distribution. What does the theory need versus what the experiments satisfy? 0.450.45 or 0.550.55 (Table 1 vs text)?

3 comments
AuthorsOpenReview ↗

Response to Reviewer pLSY

We thank the reviewer for their time and feedback! Below we respond to the questions and concerns brought up by the reviewer. We apologize for the delay in responding, the experiments are taking much longer due to cluster availability as these medical datasets required licenses on a specific cluster, but we are happy to share our current results with you.

Can you add the three baselines above (multi-label pretraining, matched-label single source, exponential-mechanism source selection), at least for CheXpert -> NIH at ϵ=1\epsilon=1?

We thank the reviewer for this suggestion. Below are the results after pre-training; we are currently running the fine-tuning but believe it will not be done before the rebuttal, as each fine-tuning run takes 10 hours and we have been having cluster availability issues.

One thing to note is that DP-MixMin minimizes downstream loss, and so when we look at the NIH task loss, we find DP-MixMin consistently beats the baselines. Here for the multi-label pretrained model, we picked the output index which had the best results for the NIH downstream task if there was not a matched label. For the 7 matched NIH tasks we got the following mean losses over the tasks (also averaged over 3 trials per task):

  1. DP-MixMin: 0.171
  2. Multi-label pretraining: 0.181
  3. Matched Single-Source: 0.196

For the 7 unmatched NIH tasks, we selected the best single-source, and got:

  1. DP-MixMin: 0.188
  2. Multi-label pretraining: 0.220
  3. Best Single-Source: 0.237

We are added the full table (with results per task) to the paper: see Table 20 in the Appendix.

However note that lower loss does not always imply lower AUROC, and we found at the pre-training stage, despite lower loss the DP-MixMin can have worse AUROC.

For the 7 matched NIH tasks we got AUROC:

  1. DP-MixMin: 0.715
  2. Multi-label pretraining: 0.756
  3. Matched Single-Source: 0.735

For the 7 unmatched NIH tasks we got AUROC:

  1. DP- MixMin: 0.583
  2. Multi-label pretraining: 0.661
  3. Best Single-Source: 0.633

See Table 21 in the Appendix for the per-task results. Note DP-MixMin has comparable AUROC for almost all tasks, with just a few tasks where the DP-MixMin is worse on AUROC. In fact these also coincide with the tasks with the lowest positive rate. When instead evaluating AUPRC, which is known to be less sensitive to low positive rates, we found the differences between the methods becomes mostly negligible: see Table 22 in the Appendix.

These results do not answer what the differences will be after finetuning for these image classification comparisons, and we are sorry we could not answer this in time. Based on the language modeling results, we found lower pretraining loss led to also lower loss after finetuning (see below response). If our finetuning runs finish before the rebuttal we will respond with those results.

Regardless, once the results are done we will update Section 5 of the paper to make clear that DP-MixMin only directly improves downstream loss, and depending on the evaluation metric lower loss may not mean higher performance.

Is the NIH validation split treated as public? If not, how do checkpoint selection and hyperparameter tuning fit the stated (ϵ,δ)(\epsilon,\delta)?

Yes we were treating the validation set (which was 6000 samples) as public to tune training hyperparameters and so not affecting the (ϵ,δ)(\epsilon,\delta) guarantees on the rest of the task data. However we want to clarify that the pre-training and fine-tuning hyperparameter tuning was only done for the uniform baseline and then applied to all other methods (DP-MixMin and random search), and furthermore was only done for a held out validation set on one task (Atelectasis, CheXpert, ε = 1) with the same hyperparameters then applied to all tasks. Similarly, DP-MixMin parameters were only tuned on the same held out validation set on one task and used for all tasks. The only exception was the pretraining epochs for image classification (i.e., early stopping), which was done per task and mixture using the held out validation set. We have updated Appendix A.3 to make all this clearer.

We have updated section 5.1 to clarify that we assumed a small public validation set for tuning training hyperparameters. Ultimately the paper only addresses how to select one specific hyperparameter (mixture weights) with privacy for almost free.

AuthorsOpenReview ↗

Response to Reviewer pLSY Continued

What mm was used for Random Search in each setting, and why is it worse than uniform?

We selected mm to match the compute used for MixMin, and this was m=4m= 4 for image classification tasks and m=16m=16 for language modeling tasks. This is far smaller than what is needed to exhaustively search the simplex, and note the uniform mixture minimizes worst-case distance to any mixture; so it is often likely the mm mixtures were all further away to the best mixture than uniform. We note past work had also observed uniform often outperforms random search at low compute budgets [1]. Also, as mentioned in our earlier response, the hyperparameters for all pre-training and fine-tuning were tuned once only for the uniform baselines and then shared by all other methods, which may also account for the slight boost in performance for uniform (however note that uniform and random search performance are often within a standard deviation).

[1] Thudi, Anvith, et al. "Mixmin: Finding data mixtures via convex minimization." International Conference on Machine Learning, 2025

For Enron, what is the test perplexity of each pretrained model before any DP finetuning? With about 80 users at ϵ=1\epsilon =1 I suspect most of the gap is already there before finetuning.

Thank you for mentioning to include this! Indeed we found that pretraining attributed most of the difference. Below are the perplexities (mean ± std over 3 trials) of the pretrained model, before finetuning, for Enron:

  1. DP-MixMin ( ϵ=1\epsilon=1, ρ=0.3\rho =0.3): 17.83 ± 1.0135
  2. Uniform: 21.154 ± 1.592

Does the no-covariate-shift assumption (di(x)=dj(x)d_i(x) = d_j(x)) need to hold only among the public sources, or also between public and private inputs? CheXpert and NIH images are not the same distribution. What does the theory need versus what the experiments satisfy?

The no covariate shift needs to only hold among the public sources, not between the public and private inputs. Specifically, the theory only needs for the mixture conditional probability learned by CE to satisfy pλ(y∣x)=∑λipi(y∣x)p_{\lambda}(y|x) = \sum \lambda_i p_{i}(y|x); this is true if pi(x)=pj(x)p_i(x) = p_j(x). MixMin puts no constraint on the target distribution, only that the downstream objective is convex over predicted probabilities and we can compute its gradients (e.g., CE).

0.550.55 or 0.450.45 (Table 1 vs text)?

Sorry for the typo, it should be 0.550.55 and we have fixed the text accordingly.

The authors have addressed my concerns. I remained my score for a weak acceptance.

Comments on the paper

1 comment

Dear reviewers,

As we are approaching the deadline of the discussion phase, please read the authors’ responses and share your thoughts in the discussion as soon as possible. In particular, please clarify whether the responses have addressed your main concerns. Please also update your review and scores if appropriate.

Your timely input will help us reach a well-informed decision. Thank you for your time and effort!

Best regards,

Area Chair

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.